Robots.txt for standard.co.uk

        
User-agent: *
Disallow: /api/
Disallow: /internal-api/
Disallow: /*ILC-refresh

Sitemap: https://www.standard.co.uk/sitemaps/googlenews
Sitemap: https://www.standard.co.uk/sitemap.xml

User-agent: Nutch
Disallow: /