November 26, 2024
Robots.txt for va.gov
User-agent: usasearch
Allow: /
# existing disallow on va.gov (may not be needed)
User-agent: Synapse
Disallow: /
# existing disallow from vets.gov
User-Agent: *
Disallow: /analytics-opt-out.html
Disallow: /cgi-bin/
Disallow: /drupal
Disallow: /covid19screen
# disallow WIP VAMCs
# make sure to add a trailing slash at the end of the path
# to prevent sub-directories from being indexed
# see https://developers.google.com/search/docs/advanced/robots/create-robots-txt#useful-robots.txt-rules
# sitemap index
Sitemap: https://www.va.gov/sitemap.xml