Free tools
Robots.txt Examples
Robots.txt file content for bigl.ua.
Robot.txt file for: bigl.ua
User-Agent: * Disallow: /*? Disallow: /search Disallow: /redirect Disallow: /debug/ Disallow: /iframe.html Disallow: /remote/ Disallow: /catalog/ Disallow: /js/ckeditor_latest/ Disallow: /opensearch.xml Disallow: /o0* Disallow: /o1* Disallow: /o2* Disallow: /o3* Disallow: /o4* Disallow: /o5* Disallow: /o6* Disallow: /o7* Disallow: /o8* Disallow: /o9* Disallow: /R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw== Disallow: /discount User-Agent: YadirectBot Disallow: / User-Agent: YandexDirect Disallow: / User-agent: Yandex Disallow: / User-Agent: YandexBot Disallow: / User-agent: Yahoo Disallow: / User-agent: MJ12bot Disallow: / User-agent: baidu Disallow: / User-agent: dotbot Disallow: / User-agent: grapeshot Disallow: / User-agent: oBot Disallow: / User-agent: UptimeRobot Disallow: / User-agent: Wget Disallow: / User-agent: Scrapy Disallow: / User-agent: Twitterbot Disallow: / User-agent: GuzzleHttp Disallow: / User-agent: X11 Disallow: / User-agent: SemrushBot Disallow: / User-agent: Rambler Disallow: / User-agent: Linguee Bot Disallow: / User-agent: Webcrawler Disallow: / User-agent: proximic Disallow: / User-agent: Trident Disallow: / User-agent: bingbot Disallow: / User-agent: BLEXBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: CriteoBot Disallow: / User-agent: Barkrowler Disallow: / Host: https://bigl.ua Sitemap: https://bigl.ua/sitemap.xml