On October 26, Google added a new section, “Only index content meant for Search,” to its spam prevention guide: a page made for sharing by link, collaboration, or utility, rather than for search, should get a noindex by default (a robots meta tag or an X-Robots-Tag header) or be blocked in robots.txt. For a small business, that covers pages generated on the fly: online quotes, calculators, configurators, shared wishlists, client accounts. Blocking with robots.txt alone does not always work: Google only documents it for images and videos, not pages, and a page blocked from crawling can still get indexed if other sites link to it. A noindex rule, by contrast, requires Google to be able to crawl the page to read it.
Contents
On October 26, 2026, Google updated its guide “Prevent user-generated spam on your site and platform,” which covers preventing spam created by users.1 The guide gained a new section, “Only index content meant for Search”: a page made for sharing, collaboration, or utility, rather than to be found through public search, should get a noindex or be blocked in robots.txt by default.1 Google says the update is meant to help platforms keep user-created pages that were never meant for search out of its index, and to detect spam earlier.2 This article covers what the section says, which pages on a small business site can be affected, and how to apply a noindex that actually works.
01What Google changed on October 26
The new section sits in the “Prevent user-generated spam on your site and platform” guide, part of Search Central's documentation on preventing and monitoring abuse.1 The documentation changelog sums up why: to help platforms keep user-created pages that aren't meant for search out of the index, and detect spam earlier.2 Search Engine Roundtable describes the new section as a significant addition, one that asks platforms to distinguish content meant for public discovery from content meant only for direct access.3
What the new section says
The text runs to two short paragraphs. First, a question to ask: if a service lets users create shareable pages, generated outputs, or interactive tools, it should be decided whether that content is meant to be found through search engines, or only accessed directly by people with the link.1 Then the default rule: when pages are created for direct sharing, collaboration, or utility rather than public search discovery, apply the noindex rule (a robots meta tag or an X-Robots-Tag HTTP response header) or block crawling in robots.txt by default.1
PPC Land points out that everything hinges on the last two words, “by default”: the guide doesn't say to keep this content out of search forever, nor does it say what should happen when a user wants a shared page to become findable. Exclusion is the starting position; what comes after is left open.4
A scope widened beyond comments
The change doesn't stop at that section. The guide's introduction, which used to speak only of hosting platforms, now explicitly widens to tools that let users publish shared pages, generated content, or web apps.4
| Before October 26 | Since October 26 |
|---|---|
| “Hosting platforms may be similarly open to abuse” | “Hosting platforms and tools that let users publish shared pages, generated content, or web apps may be similarly open to abuse” |
| Unit of abuse named: “sites” | Unit of abuse named: “pages or sites” |
Google names no product or technology in this guide.4 The rule applies to any service that lets users produce pages, including a small business site that offers an internal tool to its visitors rather than an open publishing space.
02Which pages on a small business site can be affected
The guide targets services that let users create pages, but a page made for direct use isn't limited to large community platforms. A brochure site that generates a page for every request, for one specific visitor, falls under the same logic as soon as that page is made to be viewed or shared by link rather than found through a search.1
Pages made for a use, not for an audience
What these pages have in common: they exist so that the person who made the request, or whoever it's forwarded to, can view it, not to show up in search results on the topic.
| Page type on a small business site | Why it fits the case Google describes |
|---|---|
| Online quote generated for a client | made for one person, sent or shared by link, not meant to be searched for1 |
| Calculator or cost simulator | a result generated on the fly from a specific input, an “interactive tool” in the guide's own words1 |
| Product configurator | a combination of options unique to one visitor, of little interest for a public search1 |
| Shared wishlist or cart | “collaboration” content, made to be passed along by link1 |
| Client account or order tracking page | direct access meant for one person, never a discovery page1 |
A page generator can create hundreds of these without anyone noticing, and each one gets its own address. That's exactly the risk Google is targeting: pages unrelated to the site's subject ending up in its index and diluting the rest.14 This kind of page often slips past a first standard technical audit, because it isn't listed anywhere and only appears when a visitor triggers its creation.
Keep them out of the sitemap
A page made for direct use has no place in the sitemap, which should only list the canonical addresses meant to be found.1 What the sitemap is actually meant to hold follows the same principle: listing a quote or cart page there means asking Google to discover it, the opposite of what this guide recommends.
03Applying a noindex that works, then monitoring
The guide offers two methods, and they are not equivalent.14
The robots meta tag or the X-Robots-Tag header
For an HTML page, the rule goes in the page's head: <meta name="robots" content="noindex">. For a document that isn't an HTML page, such as a quote exported as a PDF, Google recommends the X-Robots-Tag HTTP response header, which carries the same rule server-side: X-Robots-Tag: noindex.5
A point Google makes directly on the page documenting both methods: the rule is only read if crawlers are allowed to access the page carrying it.5 In other words, a noindex placed on a page that is also blocked in robots.txt does nothing, since Google never crawls it to read the rule.
Why robots.txt alone isn't enough for a page
The guide lists blocking via robots.txt as an option, but Google's own page on controlling shared content, updated five days earlier on October 21, lists robots.txt as a method only for images and videos, not for pages.6 Noindex, by contrast, applies to every content type.6
PPC Land explains why the distinction matters: a share link, by design, gets pasted elsewhere — into a message, a post, an email — and that kind of external link can be enough to get an address blocked by robots.txt indexed anyway, without its content. John Mueller explained this back in September 2024: a page blocked by robots.txt can still get indexed if other pages link to it, and a crawler kept out by robots.txt will never see a noindex rule on that blocked page. Combining the two methods on the same page can cancel out the noindex instead of reinforcing it.4
Monitoring the Search Console indexing report
The guide adds two monitoring recommendations, placed at the top of its list. The first: verify the site or subdomain in Search Console, to get an alert if Google issues a manual action or detects a security issue, and regularly check the “Page Indexing” and “Search performance” reports for unusual spikes in indexed pages, impressions, or queries unrelated to the site's purpose.1 The second: track creation and indexing metrics internally, and set up alerts for anomalies such as a sudden spike in newly published public pages.1 Neither comes with a numeric threshold.4
This kind of monitoring is the same one a website redesign already requires: indexing coverage is part of what needs tracking in the weeks after a launch, to check that only the right addresses are entering the index.