Free tools
Robots.txt Examples
Robots.txt file content for manta.com.
Robot.txt file for: manta.com
User-agent: Googlebot Disallow: User-agent: AdsBot-Google Disallow: User-agent: Googlebot-Image Disallow: User-agent: Google-InspectionTool Disallow: User-agent: Googlebot-Video Disallow: User-agent: 008 Disallow: / User-agent: Bender Disallow: / User-agent: Baiduspider Disallow: / User-agent: Slurp/3.0 Disallow: / User-agent: Mediapartners-Google Disallow: /claim/ Disallow: /member Disallow: /profile/ Disallow: /cart/ Disallow: /manta. User-agent: ccbot Disallow: / User-agent: * Disallow: /urlverify Disallow: /api/v1/urlverify Disallow: /mdir Disallow: /jobs Disallow: /manta/sample-pdf Disallow: /coms2/dnbdescription Disallow: /coms2/page_cse Disallow: /video Disallow: /coms2/page_radlink_results Disallow: /comsite5/bin/ Disallow: /claim/ Disallow: /member Disallow: /g/ Disallow: /p/*/*/recommends Disallow: /p/*/*/following Disallow: /m/*/*/recommends Disallow: /m/*/*/following Disallow: /mb_5*_/ Allow: /mb_5*0/*_/ Allow: /mb_5*1/*_/ Allow: /mb_5*2/*_/ Allow: /mb_5*3/*_/ Allow: /mb_5*4/*_/ Allow: /mb_5*5/*_/ Allow: /mb_5*6/*_/ Allow: /mb_5*7/*_/ Allow: /mb_5*8/*_/ Allow: /mb_5*9/*_/ Allow: /mb_5*A/*_/ Allow: /mb_5*B/*_/ Allow: /mb_5*C/*_/ Allow: /mb_5*D/*_/ Allow: /mb_5*E/*_/ Allow: /mb_5*F/*_/ Allow: /mb_5*G/*_/ Allow: /mb_5*H/*_/ Allow: /mb_5*I/*_/ Allow: /mb_5*J/*_/ Allow: /mb_5*K/*_/ Allow: /mb_5*L/*_/ Allow: /mb_5*M/*_/ Allow: /mb_5*N/*_/ Allow: /mb_5*O/*_/ Allow: /mb_5*P/*_/ Allow: /mb_5*Q/*_/ Allow: /mb_5*R/*_/ Allow: /mb_5*S/*_/ Allow: /mb_5*T/*_/ Allow: /mb_5*U/*_/ Allow: /mb_5*V/*_/ Allow: /mb_5*W/*_/ Allow: /mb_5*X/*_/ Allow: /mb_5*Y/*_/ Allow: /mb_5*Z/*_/ Disallow: /c/*/*/following Disallow: /cr/ Disallow: /profile/ Disallow: /ci/ Disallow: /cs/ Disallow: /cmap/ Disallow: /icr Disallow: /icmap Disallow: /ics Disallow: /cart/ Disallow: /world/Africa Disallow: /world/Asia Allow: /world/Europe/United+Kingdom Disallow: /world/Europe Disallow: /world/Oceania Allow: /world/Oceania/Australia Allow: /world/Oceania/New+Zealand Disallow: /world/South+America Disallow: /manta. Disallow: /inv/ Disallow: /cpl/ Disallow: /cpl-edit/ Disallow: /cp-edit/ Disallow: /getlisted/ Disallow: /suggestions/ Disallow: /6009/* Disallow: /ng* Disallow: /fapi/ Disallow: /napi/ Disallow: /content-only/ Disallow: /search Disallow: /track? Disallow: *.html Disallow: /stats/c/ Disallow: /scan/c/ Disallow: /edit/c/ Disallow: /build/c/ Disallow: /ebook Disallow: /empty Disallow: /unsub/