November 26, 2024

Robots.txt for bab.la

textile

User-agent: *
Disallow: /auth/xshare
Disallow: /*/spellcheck/

User-agent: Bingbot
User-agent: ias_crawler
User-agent: ias_wombles
User-agent: SirdataBot
Crawl-delay: 1

User-agent: Verity/1.1
Crawl-delay: 3

User-agent: Keybot Translation-Search-Machine
User-agent: Covario-IDS
User-agent: 008
Disallow:/