Free tools

Robots.txt Examples

Robots.txt file content for line.me.

Robot.txt file for: line.me

      User-agent: Twitterbot
Disallow: *
Allow: /ti/g2/

User-agent: Mediapartners-Google
Disallow: /app/
Disallow: /cs/
Disallow: /R/
Disallow: /ti/
Disallow: /msg/
Disallow: /au/
Disallow: /run/
Allow: /ti/p/
Allow: /R/ti/p/
Allow: /ti/g2/
Allow: /ads.txt
Allow: /app-ads.txt

User-agent: *
Disallow: /app/
Disallow: /cs/
Disallow: /R/
Disallow: /ti/
Disallow: /msg/
Disallow: /au/
Disallow: /run/
Allow: /ti/p/
Allow: /R/ti/p/