I Built a Website Crawler Because “It Works in the Browser” Isn’t Enough
When you build a website, one of the easiest things to assume is: If the page looks fine in the browser, the website is fine. But a browser session only shows you one path through the application. A website can look completely normal while still having broken links, missing metadata, invalid structured data, crawlability problems, or performance issues across pages you never opened. That was the idea behind a project I’ve been building called sitebot. As a frontend developer, I spend a lot of ti
