Singapore
All work04 / 07

SEO crawler

Context
OOm
Year
2025
Type
Site audits at scale
Links
Concept preview · illustrative data

Overview

A FastAPI crawler using Playwright and Crawl4AI to extract metadata, structured data, security and backlink signals, with audits queued as background jobs on Redis/RQ.

What I built

  1. 01

    Playwright and Crawl4AI extract metadata, structured data, security and backlink signals.

  2. 02

    Audits are queued as background jobs on Redis/RQ.

  3. 03

    Built as a FastAPI service.

Stack

  • FastAPI
  • Playwright
  • Crawl4AI
  • Redis/RQ
Next projectAuction recommendationsApplied ML · Final year project
Pages
HomePage
WorkPage
AboutPage
ContactPage
Projects
AccioDev tool · LLM wiki
AgentEvalAgent evaluation
GEO platformAI search visibility
SEO crawlerSite audits at scale
Auction recommendationsApplied ML
Ahoy ClubYacht charter website
EZTravelAI trip planner
Actions
Copy email addresssup@danielpyae.tech
Write an emailOpens your mail app
Open the résuméPDF
Light themeAppearance
Dark themeAppearance
Match my deviceAppearance
Elsewhere
LinkedInlinkedin.com
GitHubgithub.com
X (Twitter)x.com
Magic
LumosLight your wand
Draw the Elder WandElder · the legendary wand
Draw Harry Potter’s wandHolly · phoenix feather
Draw Voldemort’s wandYew · phoenix feather
Summon HedwigSeven letters to catch
Play QuidditchThirty seconds as Seeker
Find the golden SnitchOff the map
Up Down to moveEnter to openThere’s a little magic at the bottom