What indexing means
Google's index is the large database of pages Google can show in search results. A page enters the index after Google discovers it, crawls it, renders it and decides it is worth keeping. If any step fails, the page stays out.
Indexing is not automatic. Google does not index every page it finds. It chooses based on whether a page is accessible, whether it duplicates another page, and whether it seems useful enough to include. That is why a site can have 100 pages while Google indexes only 60.
How to see what Google has indexed
The most reliable source is Google Search Console's Page indexing report. It shows how many pages are indexed, how many are not, and a reason for every excluded page. The URL Inspection tool goes further for a single page: it shows whether the page is indexed, when Google last crawled it, which canonical URL Google chose and how Google rendered it.
A site: search in Google gives a rough picture, but its counts are estimates. For decisions, we rely on Search Console.
Why pages are not indexed
| Status in Search Console | What it usually means | Typical fix |
|---|---|---|
| Excluded by noindex tag | The page tells Google not to index it | Remove the tag if the page should rank |
| Blocked by robots.txt | Crawling is blocked | Change the robots.txt rule |
| Crawled – currently not indexed | Google saw the page but chose not to index it | Improve content quality, uniqueness and internal links |
| Discovered – currently not indexed | Google knows the URL but has not crawled it yet | Strengthen internal links, check server speed and site quality |
| Duplicate without user-selected canonical | Google sees the page as a copy of another | Make the content distinct, or set a canonical and consolidate |
| Alternate page with proper canonical tag | The page points to another URL as the main version | Usually fine, if intended |
| Page with redirect | The URL redirects elsewhere | Fine for old URLs; fix if a live page redirects by mistake |
| Not found (404) / Soft 404 | The page does not exist or looks empty | Restore, redirect, or add real content |
Some exclusions are healthy. Thank-you pages, internal search results, login pages and old redirected URLs should not be indexed. The goal is not to index everything, but to index everything valuable.
Technical causes we fix
Accidental noindex tags
A noindex tag left over from a development site, or a setting in a website builder that hides pages from search engines, can silently remove pages from Google. We check every important page for noindex in its HTML and in HTTP headers.
robots.txt blocks
A rule meant to block one folder can block far more. We review robots.txt line by line and test important URLs against it.
Canonical mistakes
A canonical tag that points every page to the homepage, or to an http version of the page, tells Google to ignore the page. We correct canonicals so each indexable page points to itself unless it is a genuine duplicate.
Rendering problems
If a page's content loads only through JavaScript that fails for Google, the page can look empty and be treated as a soft 404. We check the rendered version in the URL Inspection tool.
Orphan pages
Pages that no other page links to are hard for Google to find and look unimportant. We add internal links from relevant hubs, service pages and guides.
Quality causes we fix
When Google crawls a page and still does not index it, the reason is often the content rather than the code:
- Thin pages with a few sentences and little real information.
- Near-duplicate pages, such as city pages with only the place name changed.
- Pages competing with each other, such as two pages about the same service.
- Pages with no clear purpose that answer no particular search.
The fix is to improve, merge or remove these pages. Strong, distinct service pages built from keyword research rarely have indexing problems for long.
Getting new pages indexed faster
When you publish a new page, a few steps help Google find and index it:
- Link to it from relevant existing pages, such as the services hub and related service pages.
- Include it in the XML sitemap, and make sure the sitemap is submitted in Search Console.
- Request indexing for the page through the URL Inspection tool.
- Make sure it is distinct, with content that is clearly different from existing pages.
Requesting indexing does not guarantee it. Google still decides based on the page's value, but these steps remove avoidable delays.
Sitemaps that help rather than confuse
An XML sitemap is a list of URLs you want search engines to know about. A good sitemap includes only indexable, canonical pages that return a 200 status. Sitemaps that list redirected URLs, noindexed pages or error pages send mixed signals. We clean the sitemap so it matches exactly the set of pages you want indexed, and we keep it updated as pages are added or removed.
Indexing on small local business sites
Large sites can run into crawl budget limits, where Google does not crawl every page as often as the owner would like. Google's own guidance says this mainly concerns very large sites, such as those with a million or more pages, or sites with many thousands of pages that change daily. A typical Phoenix business site with a few dozen or a few hundred pages does not have a crawl budget problem. When pages on a small site are not indexed, the cause is almost always one of the technical or quality issues above.
Index bloat: when too many pages are indexed
The opposite problem also exists. Some sites have far more pages indexed than they intended, such as tag archives, author pages, image attachment pages, filtered views, or old test pages. This is often called index bloat.
Index bloat matters because low-value pages can dilute how Google judges the site overall and can compete with the pages you want to rank. We find these pages, decide whether each should be improved, redirected or kept out of the index with noindex, and remove them from the sitemap. Website builders and blog platforms often create such pages automatically, so we also adjust settings to stop new ones appearing.
Our indexing process
- Collect the data: Page indexing report, sitemaps, crawl of the site, URL inspections of key pages.
- Sort excluded pages into intended exclusions and problems.
- Fix technical causes first: noindex, robots.txt, canonicals, redirects and rendering.
- Fix quality causes: improve, merge or remove thin and duplicate pages, and add internal links.
- Clean the sitemap and resubmit it.
- Request indexing for key pages and monitor the report until they are indexed.
How we report progress
We track the number of valuable pages indexed, the number of unintended exclusions and the impressions those pages receive in search. Most technical fixes show up in the report after Google recrawls the affected pages, which can take from a few days to several weeks. Quality improvements can take longer, because Google re-evaluates the page's value over time.
Indexing during site changes
Redesigns, platform changes and domain moves are when indexing problems appear most often. A new site may launch with noindex left on from staging, URLs may change without redirects, or a new template may drop important content. We check indexing before and after every major change, and as part of every complete SEO audit.