Robots.txt for medlineplus.gov

        Sitemap: https://medlineplus.gov/sitemap.xml
User-agent: *
Disallow: /URLchecker/
Disallow: /usp/
Disallow: /cgi/
Disallow: /logs/
Disallow: /glftp/
Disallow: /feeds/topics/
Disallow: /groupfeeds/
Disallow: /spanish/feeds/topics/
Disallow: /spanish/groupfeeds/
Disallow: /xml/
Disallow: /xml/vocabulary/
Disallow: /scwidget/topsearch.html
Disallow: /spanish/scwidget/topsearch.html
Disallow: /tutorials/*.swf$
Disallow: /spanish/tutorials/*.swf$