Free tools

Robots.txt Examples

Robots.txt file content for aarp.org.

Robot.txt file for: aarp.org

      #                         _____    _____
#      /\         /\     |  __ \  |  __ \
#     /  \       /  \    | |__) | | |__) |
#    / /\ \     / /\ \   |  _  /  |  ___/
#   / ____ \   / ____ \  | | \ \  | |
#  /_/    \_\ /_/    \_\ |_|  \_\ |_|
#
# Robots.txt file created by https://www.aarp.org/
# For domain: https://www.aarp.org/
# Created 09-12-2017 Raymond Deschenes
# Updated 10-21-2024 Kasia Lorenc - added /rewards/
# All robots will spider the domain

User-agent: *
Allow: /rewards/$
Disallow: /*.pdf
Disallow: /benefits/
Disallow: /content/aarpe/en/home/
Disallow: /content/aarpe/es/home/
Disallow: /content/aarpe/es/home/vendor/
Disallow: /content/campaigns/
Disallow: /content/dam/aarp/Member-Benefits/AARP-Auto-Buying-Program-For-Member-Communications-Digital-Fulfillment.pdf
Disallow: /content/experience-fragments/
Disallow: /content/specialized-membership/staying-sharp/
Disallow: /espanol/busqueda/
Disallow: /espanol/vendor/
Disallow: /everywhere/
Disallow: /everywhere-configuration/
Disallow: /playground-for-testing-stuff/
Disallow: /qa/
Disallow: /relationships/caregiving/
Disallow: /rewards/
Disallow: /search/
Disallow: /secure-information/
Disallow: /training-aarpe/
Disallow: /vendor/


Sitemap: https://www.aarp.org/sitemap.xml