Free tools

Robots.txt Examples

Robots.txt file content for mercadolibre.com.co.

Robot.txt file for: mercadolibre.com.co

      User-agent: Amazonbot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: FacebookExternalHit
User-agent: FacebookBot
User-agent: Twitterbot
User-agent: LinkedInBot
Disallow: 

User-agent: *
Disallow: /HOME/
Disallow: /gz/merch/
Disallow: /gz/menu
Disallow: /gz/webdevice/config
Disallow: /gz/referidos
Disallow: /*www.siteinfo.cf
Disallow: /gz/cart/
Disallow: /gz/checkout/
Disallow: /gz/user-logged
Disallow: /gz/shipping-selector
Disallow: /gz/navigation/searches/last
Disallow: /perfil/vendedor/
Disallow: /perfil/comprador/
Disallow: /perfil/profile/
Disallow: /perfil/jm/profile
Disallow: /perfil/ALEXSETHMS
Disallow: /noindex/
Disallow: /navigation/
Disallow: /*itemid
Disallow: /*/jm/item
Disallow: /recommendations*
Disallow: /org-img/html/
Disallow: /registration?confirmation_url*
Disallow: /home/recommendations
Disallow: /social/
Disallow: /adn/api*
Disallow: /product-fe-recommendations/recommendations*
Disallow: /*.js