User-agent: *
Allow: /
Disallow: /auth
Disallow: /dashboard
Disallow: /settings
Disallow: /admin

Sitemap: https://scrapewhale.dev/sitemap.xml