This section lets you control how search engines, AI bots, and crawlers access your store through the robots.txt file. From here you decide the behavior toward artificial intelligence bots, add your own rules, review backups of previous versions, and schedule automatic regeneration via cron. The section is organized into three sub-tabs: Configuration, Backups, and Cron.
Configuration
In this sub-tab you adjust the entire behavior of the robots.txt. The options are grouped into three blocks (AI bots, Catalog and languages, SEO crawling), followed by your custom rules and a preview of the resulting file.
Robots.txt File URL
Shows the direct URL of your robots.txt file. It has two buttons: one to copy the URL to the clipboard and another to open it in a new browser tab.
AI bots mode
Controls how ChatGPT, Claude, Perplexity, and other AI bots access your content. You can choose between four modes:
- Allow citation (no training): asks AI bots to be able to cite and show your store in their answers, but not to use your content to train their models. This is the recommended mode and the one enabled by default.
- Allow all (maximum exposure): allows both citation and training. Offers the highest possible visibility in AI tools.
- Block all AI bots: asks any artificial intelligence bot not to access your content.
- Disabled (no AI blocking): does not add any AI-specific rules to the file.
Note: Both the robots.txt file and the Content-Signal directive state a preference: it is honoured by each company and each crawler that decides to respect it, and there is no technical way to enforce it from the shop. Choosing "no training" does not guarantee that no system will use your content for training.
Note: Blocking only Google-Extended/Applebot-Extended affects AI training (Gemini/Apple Intelligence), not your ranking in Google/Bing search or your AI preview.
Content-Signal
Declares how AI-powered search engines may use your content via the Content-Signal directive. The available options are: Disabled, Search yes, AI training no, and Search yes, AI training yes.
Catalog images
When this option is active, it allows crawlers to index your product, category, and manufacturer images (/img/p/, /img/c/, /img/m/). If you disable it, those paths are blocked.
Rules per language prefix
When you enable this option, your private rules are automatically replicated for each active language prefix of the store (/es/, /pt/, etc.).
Social bots
Allows Facebook and Twitter/X to fetch link previews of your pages when they are shared on social networks.
Crawl delay for SEO crawlers
If you enable this option, a pause is added between requests from crawlers that honor it, reducing the server load caused by aggressive SEO bots. When enabled, two additional fields appear: Seconds (how many seconds of pause, 10 by default) and SEO crawlers (comma-separated) (the list of affected crawlers, by default AhrefsBot, SemrushBot, MJ12bot).
Note: Google ignores crawl delay; it only affects crawlers that honor it (Bing, Yandex, Ahrefs, Semrush...).
Add rule
Adds a new custom rule to the robots.txt. Each rule consists of a type selector (Allow / Deny) and a path field where you specify the URL or pattern affected. To delete an existing rule use the trash button in its row. If there are no rules configured, the message "No rules configured." is shown.
Save changes
Saves the robots configuration settings, regenerating only the blocks managed by the module. The button is only activated when there are pending changes and no validation errors.
Generate
Rebuilds the complete robots.txt file (PrestaShop base, your rules, and the sitemap) and automatically saves a backup of the previous version before applying the changes.
Note: Generating rebuilds the full robots.txt (PrestaShop base, your rules, and sitemap) and first saves a backup of the current version.
Robots .txt Preview
View in real time how your generated robots.txt file will look with all the rules and configurations you have established. If there are invalid directives, they are shown in an error block that you must fix before saving.
URL parameters
Decide, parameter by parameter, whether URLs carrying it stay blocked from crawling or become readable so search engines can see the canonical your shop already serves. The table shows three columns: Parameter, Crawling (Blocked or Allowed) and Page behind it, which tells you whether the parameter leads to a Real content page or to a Private page.
The parameters covered are the ones PrestaShop adds on its own: order, tag, id_currency and n on content pages, and search_query and back on private pages.
Note: Allowing crawling is only correct when the page serves a correct canonical or is marked as non-indexable. Without either, you multiply duplicates.
Private pages
Manages three separate signals for each private page of the shop: whether it is advertised in the sitemap, whether a crawler may read it, and whether you ask for it not to be indexed. The table has four columns: Page, In the sitemap, Crawling and Non-indexable.
The distinction matters because the three are often confused: a page can be out of the sitemap and still be crawled, and blocking its crawling does not stop it from showing up in the results if another page links to it.
Note: Marking a page as non-indexable has no effect while its crawling stays blocked: a search engine cannot read a directive on a page it may not fetch. Free its crawling too.
Note: When you free the crawling of a private page that is not marked as non-indexable, you are inviting search engines to index it. Mark it first.
Note: Before applying any change from these two panels, the module automatically saves a backup of the robots.txt file.
Note: Private and transactional pages —login, cart, orders, my account, search and password recovery— are no longer marked for the sitemap. They stay in the list with their checkbox in case you want to add them. And when you update the module they are unmarked in your shop too, once: until now the sitemap advertised your login page while robots.txt forbade reading it, which is what Google reports as "Indexed, though blocked by robots.txt".
Note: What you decide in these two panels survives the regeneration of robots.txt, whether you trigger it from the module or with the "Generate robots.txt file" button on PrestaShop's Preferences > SEO & URLs screen. Without that, every native regeneration would bring back the rules you have just removed.