Free tools
Robots.txt Examples
Robots.txt file content for olx.ua.
Robot.txt file for: olx.ua
# sitecode:olxua-desktop Host: https://www.olx.ua Sitemap: https://www.olx.ua/sitemap.xml User-agent: * Disallow: */ajax/ Disallow: */adminpanel/ Disallow: */facebook/ Disallow: */rss/ Disallow: */account/ Disallow: */myaccount/ Disallow: */print/ Disallow: */chat/ Disallow: */i2/obyavlenie/abuse/ Disallow: */m/obyavlenie/abuse/ Disallow: */bundles/ Disallow: */obyavlenie/flyer/ Disallow: */obyavlenie/contact/ Disallow: */payment/ Disallow: */post-new-ad Disallow: */favorites Disallow: */abuse/ Disallow: */search/ Disallow: */searchform/ Disallow: */observed Disallow: */adding Disallow: */list/next/ Disallow: */list/prev/ Disallow: */api/v1/users/me/ Disallow: */api/v1/offers/*/page-views/ Disallow: */api/v1/offers/*/suggested/ Disallow: */api/v1/targeting/ Disallow: */api/v1/offers/metadata/ Disallow: */i2/obyavlenie/contact/* Disallow: /api/open/oauth/token/ Disallow: */i2/* Allow: /