Free tools
Robots.txt Examples
Robots.txt file content for youtube.com.
Robot.txt file for: youtube.com
# robots.txt file for YouTube # Created in the distant future (the year 2000) after # the robotic uprising of the mid 90's which wiped out all humans. User-agent: Mediapartners-Google* Disallow: User-agent: * Disallow: /api/ Disallow: /comment Disallow: /feeds/videos.xml Disallow: /get_video Disallow: /get_video_info Disallow: /get_midroll_info Disallow: /live_chat Disallow: /login Disallow: /qr Disallow: /results Disallow: /signup Disallow: /t/terms Disallow: /timedtext_video Disallow: /verify_age Disallow: /watch_ajax Disallow: /watch_fragments_ajax Disallow: /watch_popup Disallow: /watch_queue_ajax Disallow: /youtubei/ Sitemap: https://www.youtube.com/sitemaps/sitemap.xml Sitemap: https://www.youtube.com/product/sitemap.xml