Free tools

Robots.txt Examples

Robots.txt file content for ndtv.com.

Robot.txt file for: ndtv.com

      User-agent: *
Disallow: /images/ndtvvideo/*
Disallow: /news/redirect/*
Disallow: /news/feeds/*
Disallow: /page/send-email/*
Disallow: /mb/
Disallow: /usopen09/*
Disallow: /ndtvfuture/*
Disallow: /ndtvfuture/ndtv/postcomments.aspx
Disallow: /video/embed-player

User-agent: Googlebot-News
Disallow: /partner-content/*
Disallow: /brandcontent/*

User-agent: GPTBot
Disallow: /

Sitemap: https://www.ndtv.com/sitemap.xml
Sitemap: https://www.ndtv.com/sitemap/google-news-sitemap
Sitemap: https://www.ndtv.com/sitemap.xml?years=2023
Sitemap: https://www.ndtv.com/entertainment/sitemap-2023
Sitemap: https://www.ndtv.com/entertainment/sitemap.xml?years=2023