# Azul by Omar - https://azulspit.com.au # Open to all major search and AI crawlers. The marina, the menu, the # music programme — all of it is meant to be found. User-agent: * Allow: / Disallow: /partials/ Disallow: /.well-known/ # AI search crawlers - explicitly welcome User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # Scrapers that ignore robots.txt anyway - documented for transparency User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / Sitemap: https://azulspit.com.au/sitemap.xml