Home › SEO Audit › 2026 Data Study
We took the top 300 domains from the Tranco ranking and crawled each homepage. 181 served a normal HTML page. Even the most-visited sites in the world miss cheap, obvious things – and AI-readiness is the new gap.
The headline numbers
Out of 181 homepages that rendered for our crawler.
| Check | Sites failing | Share |
|---|---|---|
| No JSON-LD structured data | 61 | 34% |
| No llms.txt (AI-readiness) | 61 | 34% |
| No Open Graph image | 51 | 28% |
| No sitemap listed in robots.txt | 49 | 27% |
| Not exactly one H1 | 47 | 26% |
| No canonical tag | 42 | 23% |
| No meta description | 35 | 19% |
| Meta description the wrong length | 35 | 19% |
| Image without alt text | 23 | 13% |
| Title tag over 60 characters | 21 | 12% |
| No mobile viewport tag | 18 | 10% |
| Blocks GPTBot | 8 | 4% |
| Not on HTTPS | 0 | 0% |
What this means for your site
How we ran this study
A reproducible crawl, not a survey.
Sample
The top 300 domains from the Tranco ranking (list of 2026-09-20), which combines Crux, Farsight, Majestic, Radar and Umbrella data.
Fetch
Each homepage over HTTPS, plus /robots.txt and /llms.txt, with a standard crawler user-agent and a 15-second timeout.
Filter
181 of the 300 served a normal HTML homepage. The other 119 were apps, APIs or blocked automated visits, so they are excluded from the percentages.
Measure
13 signals a crawler can see: title, meta description, H1 count, canonical, viewport, Open Graph image, JSON-LD, image alt text, HTTPS, sitemap in robots.txt, llms.txt, and a GPTBot block.
Aggregate
Findings are reported as the share of the 181 rendered homepages that fail each check.
AI-readiness is the new gap
The standout result in 2026 is not title tags or HTTPS – it is being legible to AI. 34% of the homepages had no llms.txt file and 4% actively blocked GPTBot, the crawler behind ChatGPT. Structured data, which feeds both rich results and AI answers, was missing on another 34%.
The basics are mostly solved
Not one homepage failed HTTPS. Titles were mostly present and well-formed. This tells you the traditional hygiene checks are no longer where the edge is – the sites that pull ahead are the ones handling the newer signals: structured data, social previews and AI access.
What each finding means
34% missing
Not a ranking factor on its own, but it powers rich results, clarifies entities and feeds the AI layer that decides who to cite.
34% no llms.txt, 4% block GPTBot
The cheapest, most overlooked lever in 2026 – a five-minute fix that most competitors have not done.
28% no OG image
Shared links show a blank card. You paid for the traffic; the preview is where the click is won or lost.
26% not one H1, 23% no canonical
Basic hygiene, still mishandled as sites change. Cheap to fix, quick to verify.
How to check your own site
Five minutes, no signup.
Run the free audit
The free website SEO audit checks llms.txt and AI-crawler access alongside titles, headings, links, images, indexability and speed.
Check AI access
Confirm GPTBot, PerplexityBot, ClaudeBot and Google-Extended are not blocked, and publish an llms.txt file.
Add structured data
Start with Organization and WebSite on the homepage, and FAQPage where you answer questions. See the on-page SEO audit.
Fix the hygiene
Work the 40-point checklist for H1s, canonicals and meta descriptions.
How to cite this study
For journalists, bloggers and newsletters.
Key stats you can quote:
- 34% of the 181 homepages had no structured data.
- 34% had no llms.txt file; 4% actively blocked GPTBot.
- 28% had no Open Graph image; 27% listed no sitemap in robots.txt.
- 26% did not have exactly one H1; 23% had no canonical tag.
- 0% failed HTTPS – the traditional basics are solved.
Raw data and method available on request. Suggested in-text citation: “GSA Growth’s 2026 study of the 300 most-visited websites found that 34% had no llms.txt and 4% blocked GPTBot.”
Limitations
- Homepage-only, static-HTML snapshot; we did not measure Core Web Vitals, content quality or backlinks.
- The sample is the most-visited domains, which are large, well-resourced sites – small sites may score differently.
- Percentages are of the 181 domains that rendered for our crawler, not all 300.
- Checks reflect a single crawl in September 2026; the web changes quickly.
Frequently asked questions
How many websites did you audit?
What checks did you run?
Does this mean big sites are badly optimised?
What is an llms.txt file?
Can I reproduce this study?
Is this a formal research paper?
Why does AI-readiness matter now?
How do I check my own site?
Related reading
Run a free SEO audit
No signup. Up to 5 pages, technical and on-page, prioritised in under a minute.