Free tools
Robots.txt Examples
Robots.txt file content for smule.com.
Robot.txt file for: smule.com
User-agent: PiplBot Disallow: / User-agent: piplbot Disallow: / User-agent: EasouSpider Disallow: / User-agent: meanpathbot Disallow: / User-agent: QuerySeekerSpider Disallow: / User-agent: Scrapy Disallow: / User-agent: SemrushBot Disallow: / User-agent: * Disallow: /s/js-log Disallow: /user/ Disallow: /password_reset Disallow: /activate Disallow: /s/playlist_json Disallow: /profile/block Disallow: /profile/unblock Disallow: /profile/is_blocked Disallow: /forum/ Disallow: /s/performance/ Sitemap: https://www.smule.com/assets/sitemaps/daily.xml.gz Sitemap: https://www.smule.com/assets/sitemaps/weekly.xml.gz Sitemap: https://www.smule.com/assets/sitemaps/monthly.xml.gz Sitemap: https://www.smule.com/assets/sitemaps/yearly.xml.gz