November 26, 2024

Robots.txt for amarujala.com

        User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: CCBot 
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: Omgili
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: Diffbot
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: ImagesiftBot
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: Meta-ExternalAgent
Disallow: /
User-agent: Meta-ExternalFetcher
Disallow: /
User-agent: Applebot
Disallow: /
User-agent: Applebot-Extended
Disallow: /
User-agent: Amazonbot
Disallow: /

User-agent: *
Allow: /.well-known/amphtml/apikey.pub
Disallow: /blocked.html
Disallow: /entertainment/bollywood/poonam-pandey-sami-nude-pics
Disallow: /static-ad/home
Disallow: /ucv1/
Disallow: /ajax/
Disallow: /cdn-cgi/
Disallow: /sports/parimatch-betting-five-qualities-of-modern-betting-all-you-need-to-know
Disallow: /lifestyle/kapiva-himalayan-shilajit-from-18000-feet-himalaya-for-good-physical-and-mental-health
Disallow: /tags/betting
Disallow: /channels/amp_downloads*
Disallow: /tags/sex*
Disallow: /games/*

Sitemap: https://www.amarujala.com/sitemap.xml
Sitemap: https://www.amarujala.com/sitemap-news-v1.xml
Sitemap: https://www.amarujala.com/sitemap/sitemap-index.xml