Free tools

Robots.txt Examples

Robots.txt file content for ncsu.edu.

Robot.txt file for: ncsu.edu

      User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: https://www.ncsu.edu/wp-sitemap.xml