Free tools

Robots.txt Examples

Robots.txt file content for cheatsheet.com.

Robot.txt file for: cheatsheet.com

      User-agent: *
Disallow: /wp-admin/

Disallow: /wp-includes/certificates
Disallow: /wp-includes/SimplePie
Disallow: /wp-includes/Text
Disallow: /wp-includes/*.php$
Disallow: /eyeblaster/addineyeV2.html
Disallow: /wp-content/plugins
Disallow: /Just-For-Test
Disallow: /test/
Disallow: /recipes/
Disallow: /wine/
Disallow: /wine-jargon/
Disallow: /archive/

Disallow: /forum/
Disallow: /watchdog/
Disallow: /cronjob/
Disallow: /wp-content/uploads/wpsc/
Disallow: /downloads

Disallow: /*Insert_Click_Track_URLhttp://ad.wsod.com/

Disallow: /*attachment/

Disallow: /*/*...

Disallow: /1
Disallow: /pubclick

Disallow: /newsletters/p/
Disallow: /members/p/
Disallow: /premium/p/
Disallow: /members-st/

Disallow: /?p=*

Disallow: /mcs/*

Disallow: /*/entertainment/page/*/

Allow: /ads.txt

User-agent: Mediapartners-Google
Allow: /

User-agent: Googlebot-News
Allow: /

# Sitemap archive
Sitemap: https://www.cheatsheet.com/news-sitemap.xml

Sitemap: https://www.cheatsheet.com/sitemap.xml


# START YOAST BLOCK
# ---------------------------
User-agent: *
Disallow:

# ---------------------------
# END YOAST BLOCK