robots.txt Viewer

An online robots.txt viewer. Enter a domain to fetch and parse the site's robots.txt, showing Allow/Disallow rules grouped by User-agent along with Sitemap URLs and crawl delay. It's a handy tool for checking crawler rules and doing SEO checks.