Free tools

Robots.txt Examples

Robots.txt file content for twitch.tv.

Robot.txt file for: twitch.tv

      User-Agent: *
Allow: /
Allow: /directory
Allow: /directory/all
Allow: /directory/*
Allow: /videos/week
Allow: /.well-known/assetlinks.json
Disallow: /admin/*
Disallow: /email-unsubscribe/
Disallow: /login$
Disallow: /message/*
Disallow: /signup$
Disallow: /user/*

Sitemap: https://www.twitch.tv/sitemapv2_index.xml.gz