Robots meta directives give you page-by-page command over how search engines crawl, index, and present your content. Most site owners are aware of the noindex option, but the full directive set includes seven controls that govern everything from snippet length to image preview size. Mastering these settings prevents accidental deindexing and lets you fine-tune your search appearance.
The Complete Directive Reference
noindex
Instructs crawlers to exclude the page from search results entirely. The page can still be crawled, but it will not appear for any query. Apply this to utility pages with no search value: thank-you confirmations, internal dashboards, login screens, or content duplicates you cannot resolve through canonical tags.
nofollow
Tells crawlers not to follow or pass ranking equity through any link on the page. This is a blanket page-level instruction, distinct from the per-link rel="nofollow" attribute. Use it sparingly because it prevents all internal links on the page from distributing authority to their destinations.
nosnippet
Blocks the search engine from generating a text snippet or video preview for the page in results. The listing will show only the title and URL with no descriptive text. This also prevents the page from being included in AI-generated answer panels. Apply it only when you have a specific reason to suppress preview content.
noarchive
Prevents the engine from storing or displaying a cached snapshot of the page. Visitors will not be able to access a saved version through the cache link. Useful for pages with frequently changing prices, time-sensitive offers, or content you want to control access to strictly through your live site.
noimageindex
Stops images on the page from appearing in image search results. The page itself can still be indexed normally. Apply this when you use licensed visuals that should not circulate outside your site, or on pages where image search traffic holds no value.
Display Control Directives
max-snippet
Sets a character ceiling for the text snippet shown in results. A value of 0 suppresses the snippet, -1 lets the engine choose freely, and any positive number caps the length at that many characters. Leaving it unset or using -1 is the right choice for the vast majority of pages.
max-image-preview
Caps the size of image thumbnails in search results. The options are "none" (no thumbnail), "standard" (small default size), and "large" (full-width preview). Set this to "large" on every page to maximise visual prominence, especially on mobile where thumbnails heavily influence click behaviour.
max-video-preview
Limits the duration of animated video previews in results. A value of 0 shows a static still frame, -1 allows full-length previews, and a positive number caps the preview at that many seconds. If you embed video content, generous preview lengths tend to increase engagement.
Applying Directives to Individual Pages
Setting robots directives on a single page
- 1Open the page in your editor
- 2Navigate to the SEO settings through the page menu
- 3Switch to the Advanced tab
- 4Locate the robots meta tag controls
- 5Select or configure the directives you need: noindex, nofollow, nosnippet, and so on
- 6For the display controls (max-snippet, max-image-preview, max-video-preview), enter the appropriate values
- 7Save and publish your changes
Setting Defaults for Entire Page Categories
When you have page types with many entries (blog articles, product listings, service pages), you can set default directives through the SEO configuration in your Dashboard. Select the page category, adjust the robots settings, and the defaults apply to every page of that type. Individual page overrides still take precedence.
High-Risk Setting
Enabling noindex at the page-category level will hide every page of that type from search results. Always double-check category-level settings before saving. Accidentally noindexing your entire blog or product catalogue can take weeks to recover from.
Mistakes That Silently Remove Pages
- Toggling noindex on key pages during testing and forgetting to reverse it
- Applying noindex to an entire page category in the SEO defaults without realising the scope
- Using nofollow on internal hub pages, cutting off equity flow to the rest of the site
- Setting max-snippet to 0, which strips descriptive text from all listings for the affected pages
- Leaving noindex active after finishing a staging or pre-launch phase
- Combining directives incorrectly when only a single directive was intended
Quarterly Audit
Crawl your entire site with a tool like Screaming Frog or use the platform inspection features once a quarter. Filter for any pages carrying unexpected noindex directives. Catching a stray noindex before it does damage is far easier than recovering from weeks of invisible deindexing.
Key Takeaways
- Seven directives are available: noindex, nofollow, nosnippet, noarchive, noimageindex, max-snippet, max-image-preview, and max-video-preview
- Set max-image-preview to "large" across all pages for maximum thumbnail visibility in results
- Never apply noindex at the page-category level without fully understanding how many pages it affects
- Use individual page settings for critical content and category defaults for bulk page types
- Audit robots directives quarterly to catch accidental configurations before they cause ranking loss
- Misconfigured directives are among the most common causes of pages silently disappearing from search results
