Free tools

Robots.txt Examples

Robots.txt file content for alltrails.com.

Robot.txt file for: alltrails.com

      # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
#
User-agent: Butterfly
Disallow: /
User-agent: Exabot
Disallow: /
User-agent: Ginxbot
Disallow: /
User-agent: MetaURI
Disallow: /
User-agent: Postrank
Disallow: /
User-agent: ScoutJet
Disallow: /
User-agent: ThingFetcher
Disallow: /
User-agent: TweetmemeBot
Disallow: /
User-agent: Tagoobot
Disallow: /
User-agent: mxbot
Disallow: /
User-agent: Twitturls
Disallow: /
User-agent: radian6
Disallow: /
User-agent: Viralheat
Disallow: /
User-agent: MLBot
Disallow: /
User-agent: Twiceler
Disallow: /
User-agent: Twingly Recon
Disallow: /
User-agent: Moreoverbot
Disallow: /
User-agent: Purebot
Disallow: /
User-agent: Twitturly
Disallow: /
User-agent: OneRiot
Disallow: /
User-agent: Feedtrace
Disallow: /
User-agent: FairShare
Disallow: /
User-agent: NjuiceBot
Disallow: /
User-agent: semiocast
Disallow: /
User-agent: Typhoeus
Disallow: /
User-agent: my6sense
Disallow: /
User-agent: twitmatic
Disallow: /
User-agent: BlogPulseLive
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: discobot
Disallow: /
User-agent: Ezooms
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: SiteBot
Disallow: /
User-agent: TweetedTimes Bot
Disallow: /
User-agent: NHN Corp
Disallow: /
User-agent: Bender
Disallow: /
User-agent: proximic
Disallow: /
User-agent: netseer
Disallow: /
User-agent: YodaoBot
Disallow: /
User-agent: Birubot
Disallow: /
User-agent: Showyoubot
Disallow: /
User-agent: suggybot
Disallow: /
User-agent: BOTW Spider
Disallow: /
User-agent: Summify
Disallow: /
User-agent: TANNER
Disallow: /
User-agent: gosospider
Disallow: /
User-agent: expo9
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: Crowsnest
Disallow: /
User-agent: DoCoMo
Disallow: /
User-agent: percbotspider
Disallow: /
User-agent: trendictionbot
Disallow: /
User-agent: knowaboutBot
Disallow: /
User-agent: Atomic_Email_Hunter
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: *
Allow: /*/maps/*/share_image
Disallow: /static2/
Disallow: /stob-dab/
Disallow: /register/
Disallow: /users/auth/
Disallow: /api/
Disallow: /api-v4/
Disallow: /api-v5/
Disallow: /*/api/
Disallow: /*/api-v4/
Disallow: /*/api-v5/
Disallow: /*?lat=
Disallow: /members/

Sitemap: https://www.alltrails.com/sitemap/secure/rails/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/en-gb/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/da-dk/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/de/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/es/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/es-mx/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/fr/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/it-it/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/nl-nl/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/nb-no/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/pl-pl/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/pt-br/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/pt-pt/index.xml
Sitemap: https://www.alltrails.com/sitemap/secure/rails/sv-se/index.xml