Free tools

Robots.txt Examples

Robots.txt file content for lawinsider.com.

Robot.txt file for: lawinsider.com

      User-agent: *
Disallow: /contracts/tagged/*+*
Disallow: /contracts/tagged/*%2B*
Disallow: /signin*
Disallow: /signup*
Disallow: /user/*
Disallow: /api/search
Disallow: /contracts/*.pdf$
Disallow: /contracts/*.docx$
Disallow: /contracts/*.drive$
Disallow: /clause/*.pdf$
Disallow: /clause/*.docx$
Disallow: /clause/*.drive$
Disallow: /search*
Sitemap: https://www.lawinsider.com/sitemap.xml