Crawl budget management for large Wix dynamic page collections

When a Wix CMS collection grows to hundreds or thousands of records, crawl budget becomes a real constraint. Google allocates a crawl budget to every site based on its authority and server speed. If you have 5,000 dynamic pages and a modest crawl budget, Google may take weeks to crawl and index all of them — and will prioritise the pages it finds most valuable. Managing this correctly is essential for large dynamic page sets.

How Google Allocates Crawl Budget to Wix Dynamic Pages

Google does not crawl every page on a site every day. It uses crawl frequency signals including the age of pages, their update frequency, their internal link depth, and the overall site authority to determine which pages to crawl and how often. New dynamic pages added to a large collection may wait days or weeks for their first crawl if the collection is not well-architected.

Strategies for Efficient Crawl Budget on Dynamic Pages

• Add the highest-priority dynamic pages to internal links on high-authority pages to attract crawl attention faster
• Use robots.txt to block crawling of filter and sort URL variants that duplicate dynamic content
• Noindex thin or near-duplicate records rather than letting Google waste budget crawling them
• Keep your XML sitemap updated and accurate — remove deleted records promptly
• Limit URL parameter variants by configuring canonical tags or parameter handling

Monitoring Dynamic Page Indexing in Google Search Console

In Google Search Console, the Pages report under Indexing shows how many of your site pages are indexed. For dynamic pages, filter by the URL prefix of your dynamic page set to see the indexing rate. If a large collection is showing many "discovered but not indexed" pages, this is a sign of crawl budget strain combined with low perceived value of the content.

Step-by-step: How to manage crawl budget for large Wix dynamic page collections

• Open Google Search Console and navigate to the Pages report under Indexing to see how many dynamic pages have been indexed versus discovered but not indexed.
• Filter the Pages report by the URL prefix of your dynamic page set to isolate the collection-specific indexing data.
• If many pages show "Discovered — currently not indexed", check whether the content quality meets Google's bar by reviewing the pages manually.
• Add your highest-priority dynamic pages to prominent internal links on high-authority pages to attract Googlebot attention to them faster.
• In your CMS collection, identify any records with thin content (short descriptions, missing images, duplicate boilerplate) and add a boolean "noindex" field set to true for those records.
• Configure the dynamic page template to output a noindex robots meta tag when the collection record's noindex field is set to true.
• Review your sitemap at yoursite.com/sitemap.xml and confirm it only lists pages you actually want indexed — remove deleted or noindexed records promptly.
• Check your Wix robots.txt to ensure no crawl directives are accidentally blocking Googlebot from your dynamic page set.
• Monitor the indexing rate monthly in Search Console and prioritise enriching collection records that remain in "Discovered but not indexed" status.

How to Audit and Optimise Crawl Budget for Large Wix CMS Collections

Auditing crawl budget efficiency for large Wix CMS collections ensures Google prioritises your highest-value dynamic pages and does not waste crawl allocation on thin or low-quality records.

How to audit and manage crawl budget allocation for Wix dynamic page collections

• Open Google Search Console and navigate to the Pages report under Indexing — click "Not indexed" to see pages Google has discovered but not indexed.
• Filter the Pages report by the URL prefix of your dynamic page set to isolate the indexing status of that specific collection.
• Export the full list of "Discovered — currently not indexed" URLs from the dynamic page set and open a sample of them to manually assess content quality.
• For each thin or low-quality record identified, open Wix Dashboard > CMS > Collections and add a boolean "noindex" field — set it to true for records that do not meet content quality standards.
• Configure the dynamic page template in the Wix Editor to output a noindex robots meta tag when the collection record's noindex field is set to true.
• Review your XML sitemap at yoursite.com/sitemap.xml and confirm it includes only pages you want Google to prioritise — remove deleted or noindexed records promptly.
• Add your highest-priority dynamic pages to internal links on your most authoritative pages (homepage, main service pages) to attract Googlebot crawl attention.
• Check that robots.txt at yoursite.com/robots.txt does not accidentally block the dynamic page URL prefix with a Disallow directive.
• In Wix Dashboard > Marketing & SEO > SEO Tools, confirm the sitemap submission is up to date and resubmit if collection records have been recently added or removed.
• Set a monthly reminder to review the Pages report in Search Console filtered to the dynamic page prefix and track the ratio of indexed to discovered pages over time.
• Aim to maintain an indexing ratio above 80% for well-structured collections — a persistently low ratio signals content quality issues that need addressing.