Scenario

AI Crawler Policy for Ecommerce Stores

Stores usually want product pages discoverable, but cart, checkout, account, search result filters, and internal campaign URLs should be handled carefully.

Suggested protected paths

/cart/
/checkout/
/account/
/login/
/admin/
/search?

Use robots.txt for crawl preference and noindex or canonical rules for search quality. Real customer data must remain behind authentication.