Free tools

Robots.txt Examples

Robots.txt file content for people.com.

Robot.txt file for: people.com

      User-agent: *

Disallow: *quizResult=
Disallow: *.pdf
Disallow: *globeTest_
Disallow: *globeNoTest
Disallow: *globeResource
Disallow: *?kw
Disallow: /embed?
Disallow: /shop/
Disallow: /authentication/
Disallow: /newsletters/preferences/manage
Disallow: /newsletters/preferences/unsubscribe

User-agent: Pinterest
Disallow:

User-agent: Pinterestbot
Disallow:

User-agent: CCBot
Disallow: /

User-agent: GPTBot
Disallow: /

Sitemap: https://people.com/sitemap.xml

Sitemap: https://people.com/google-news-sitemap.xml