Free tools

Robots.txt Examples

Robots.txt file content for mercadolivre.com.br.

Robot.txt file for: mercadolivre.com.br

      User-agent: Amazonbot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: FacebookExternalHit
User-agent: FacebookBot
User-agent: Twitterbot
User-agent: LinkedInBot
Disallow: 

User-agent: *
Disallow: /HOME/
Disallow: /gz/merch/
Disallow: /gz/menu
Disallow: /gz/webdevice/config
Disallow: /gz/referidos
Disallow: /*www.siteinfo.cf
Disallow: /gz/cart/
Disallow: /gz/checkout/
Disallow: /gz/user-logged
Disallow: /gz/shipping-selector
Disallow: /gz/navigation/searches/last
Disallow: /perfil/vendedor/
Disallow: /perfil/comprador/
Disallow: /perfil/profile/
Disallow: /perfil/jm/profile
Disallow: /perfil/ALEXSETHMS
Disallow: /noindex/
Disallow: /navigation/
Disallow: /*itemid
Disallow: /*/jm/item
Disallow: /recommendations*
Disallow: /org-img/html/
Disallow: /registration?confirmation_url*
Disallow: /home/recommendations
Disallow: /social/
Disallow: /adn/api*
Disallow: /product-fe-recommendations/recommendations*
Disallow: /*.js
Disallow: /*/p/MLB*/s
Disallow: /*/p/MLB*/seller-info
Disallow: /*/p/MLB*/specifications
Disallow: /anuncie
Disallow: /gz/
Disallow: /blog/cg-*
Disallow: /mais-vendidos/MLB
Disallow: /ofertas?
Allow: /home/recommendations/pom