Free tools
Robots.txt Examples
Robots.txt file content for aarp.org.
Robot.txt file for: aarp.org
# _____ _____ # /\ /\ | __ \ | __ \ # / \ / \ | |__) | | |__) | # / /\ \ / /\ \ | _ / | ___/ # / ____ \ / ____ \ | | \ \ | | # /_/ \_\ /_/ \_\ |_| \_\ |_| # # Robots.txt file created by https://www.aarp.org/ # For domain: https://www.aarp.org/ # Created 09-12-2017 Raymond Deschenes # Updated 10-21-2024 Kasia Lorenc - added /rewards/ # All robots will spider the domain User-agent: * Allow: /rewards/$ Disallow: /*.pdf Disallow: /benefits/ Disallow: /content/aarpe/en/home/ Disallow: /content/aarpe/es/home/ Disallow: /content/aarpe/es/home/vendor/ Disallow: /content/campaigns/ Disallow: /content/dam/aarp/Member-Benefits/AARP-Auto-Buying-Program-For-Member-Communications-Digital-Fulfillment.pdf Disallow: /content/experience-fragments/ Disallow: /content/specialized-membership/staying-sharp/ Disallow: /espanol/busqueda/ Disallow: /espanol/vendor/ Disallow: /everywhere/ Disallow: /everywhere-configuration/ Disallow: /playground-for-testing-stuff/ Disallow: /qa/ Disallow: /relationships/caregiving/ Disallow: /rewards/ Disallow: /search/ Disallow: /secure-information/ Disallow: /training-aarpe/ Disallow: /vendor/ Sitemap: https://www.aarp.org/sitemap.xml