Free tools

Robots.txt Examples

Robots.txt file content for artstation.com.

Robot.txt file for: artstation.com

      User-agent: *
Disallow: /*/likes
Disallow: /*/following
Disallow: /*/followers
Disallow: /*/collections
Disallow: /*/collections/likes
Disallow: /*/collections/*
Disallow: /registration/*
Disallow: /studentpro
Disallow: /2fa
Disallow: /users/password/edit

Sitemap: https://www.artstation.com/sitemap.xml
Sitemap: https://www.artstation.com/api/v2/marketplace/product-sitemap-index.xml
Sitemap: https://www.artstation.com/api/v2/prints/printed-product-sitemap-index.xml
Sitemap: https://www.artstation.com/api/v2/prints/curated-pages-sitemap-index.xml
Sitemap: https://www.artstation.com/api/v2/blogging/blog-sitemap-index.xml
Sitemap: https://www.artstation.com/api/v2/jobs/jobs-sitemap-index.xml
Sitemap: https://www.artstation.com/api/v2/learning/sitemap-index.xml
Sitemap: https://www.artstation.com/api/v2/seo/sitemap-seo-pages-index.xml
Sitemap: https://www.artstation.com/api/v2/competition/challenges-sitemap-index.xml