Free tools
Robots.txt Examples
Robots.txt file content for artstation.com.
Robot.txt file for: artstation.com
User-agent: * Disallow: /*/likes Disallow: /*/following Disallow: /*/followers Disallow: /*/collections Disallow: /*/collections/likes Disallow: /*/collections/* Disallow: /registration/* Disallow: /studentpro Disallow: /2fa Disallow: /users/password/edit Sitemap: https://www.artstation.com/sitemap.xml Sitemap: https://www.artstation.com/api/v2/marketplace/product-sitemap-index.xml Sitemap: https://www.artstation.com/api/v2/prints/printed-product-sitemap-index.xml Sitemap: https://www.artstation.com/api/v2/prints/curated-pages-sitemap-index.xml Sitemap: https://www.artstation.com/api/v2/blogging/blog-sitemap-index.xml Sitemap: https://www.artstation.com/api/v2/jobs/jobs-sitemap-index.xml Sitemap: https://www.artstation.com/api/v2/learning/sitemap-index.xml Sitemap: https://www.artstation.com/api/v2/seo/sitemap-seo-pages-index.xml Sitemap: https://www.artstation.com/api/v2/competition/challenges-sitemap-index.xml