Robots.txt for free.fr
User-Agent: Googlebot User-Agent: Googlebot-Image User-Agent: Mediapartners-Google User-Agent: Googlebot-News User-Agent: Googlebot-Video User-Agent: AdsBot-Google User-Agent: AdsBot-Google-Mobile User-Agent: AdsBot-Google-Mobile-Apps User-Agent: Storebot-Google User-Agent: Bingbot User-Agent: AdIdxBot User-Agent: DuckDuckBot User-Agent: Slurp User-Agent: Qwantify User-Agent: Pinterest Allow: / User-Agent: * Disallow: /freebox/informations/avis-freebox/?page= Allow: /freebox/informations/avis-freebox/?page=1 Disallow: /apps/ Disallow: /*contentOnly* User-Agent: buenibot Disallow: / User-Agent: EtaoSpider Disallow: / User-Agent: Yandex Disallow: / User-Agent: Baiduspider Disallow: / User-Agent: Baiduspider-image Disallow: / User-Agent: 360Spider Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: CazoodleBot Disallow: / User-Agent: HaosouSpider Disallow: / User-Agent: ia_archiver Disallow: / User-Agent: Wget Disallow: / User-Agent: TurnitinBot Disallow: / User-Agent: HTTrack Disallow: / Sitemap: https://www.free.fr/sitemap.xml