User-agent: * Allow: / Disallow: /i18n-reference.json Disallow: /404.html Disallow: /offline.html # Obsolete *.html URLs are 308-redirected to the clean URL by Vercel's # cleanUrls setting. We let crawlers follow the redirect so Google can # discover the canonical and gradually drop the .html variant from the # index. Blocking would freeze the legacy entries in place. # Google AdSense crawlers, explicitly allow so ads can be targeted properly # (Mediapartners-Google and AdsBot don't follow the wildcard rules by # default, so they need their own entries to see the same pages users see). User-agent: Mediapartners-Google Allow: / User-agent: AdsBot-Google Allow: / User-agent: AdsBot-Google-Mobile Allow: / Sitemap: https://absolutool.com/sitemap.xml