November 26, 2024

Robots.txt for betterhealth.vic.gov.au

        User-agent: *
Disallow: /js
Disallow: /img
Disallow: /_nuxt
Disallow: /_nuxt/*
User-agent: bingbot
Crawl-delay: 10
User-agent: Baiduspider
Crawl-delay: 10
User-agent: Mediapartners-Google
Crawl-delay: 10
User-agent: www.proximic.com/info/spider.php
Crawl-delay: 10
User-agent: ia_archiver ( crawler@alexa.com)
Crawl-delay: 10
User-agent: GrapeShotCrawler
Crawl-delay: 10
User-agent: msnbot
Crawl-delay: 10
User-agent: YandexBot
Crawl-delay: 10
User-agent: Yahoo! Slurp
Crawl-delay: 30
User-agent: Slurp
Crawl-delay: 30
User-agent: dotbot
Disallow: /
User-agent: SemrushBot
Disallow: /
User-agent: AhrefsBot
Disallow: /
User-agent: MauiBot
Disallow: /
User-agent: Swiftbot
Crawl-delay: 10
Sitemap: https:/www.betterhealth.vic.gov.au/sitemap.xml