Crawl budget management for large Wix dynamic page collections

When a Wix CMS collection grows to hundreds or thousands of records, crawl budget becomes a real constraint. Google allocates a crawl budget to every site based on its authority and server speed. If you have 5,000 dynamic pages and a modest crawl budget, Google may take weeks to crawl and index all of them — and will prioritise the pages it finds most valuable. Managing this correctly is essential for large dynamic page sets.

How Google Allocates Crawl Budget to Wix Dynamic Pages

Google does not crawl every page on a site every day. It uses crawl frequency signals including the age of pages, their update frequency, their internal link depth, and the overall site authority to determine which pages to crawl and how often. New dynamic pages added to a large collection may wait days or weeks for their first crawl if the collection is not well-architected.

Strategies for Efficient Crawl Budget on Dynamic Pages

• Add the highest-priority dynamic pages to internal links on high-authority pages to attract crawl attention faster
• Use robots.txt to block crawling of filter and sort URL variants that duplicate dynamic content
• Noindex thin or near-duplicate records rather than letting Google waste budget crawling them
• Keep your XML sitemap updated and accurate — remove deleted records promptly
• Limit URL parameter variants by configuring canonical tags or parameter handling

Monitoring Dynamic Page Indexing in Google Search Console

In Google Search Console, the Pages report under Indexing shows how many of your site pages are indexed. For dynamic pages, filter by the URL prefix of your dynamic page set to see the indexing rate. If a large collection is showing many "discovered but not indexed" pages, this is a sign of crawl budget strain combined with low perceived value of the content.