01
Two views
The browser with JavaScript and MetozBot without JavaScript are compared using the same main-content extraction.
AI crawler view
How AI reads
your website.
Metoz shows the difference between what people see and the text a crawler receives without JavaScript.
No sign-in. The request respects the target website’s robots.txt.
What the test answers
Accessible does not automatically mean understandable.
A website can allow a bot and still provide hardly any usable text. Metoz therefore separates access, readability, and answerability—with visible evidence instead of a single check mark.
02
Gatekeeper
robots.txt is evaluated separately for search, user-directed agent requests, and training.
03
Citation test
Six fixed questions show which key claims are supported by the crawler-visible text alone.
From finding to fix
Concrete work briefs instead of another checklist.
After a scan, the Metoz workshop creates an evidence-based brief for developers or coding agents. Separate tools help with robots.txt rules and missing website claims. The workshop is currently available in German.
01
AI work brief from the scan
Recommendations, measurements, guardrails, and acceptance criteria in Markdown.
02
robots.txt for AI bots
Configure search, agent requests, and training separately.
03
Content brief
Collect confirmed key facts without inventing missing information.
Metoz Observatory
353 German websites, measured the same way every month.
The observatory turns individual observations into an open time series covering public authorities, DAX and MDAX companies, news sites, and online shops. Raw data, CSV, and Parquet are published under CC BY 4.0.
View the observatory (DE)353
domains in the corpus
4
segments
monthly
fixed cycle
CC BY
open licence
Understand AI crawler visibility
Technical foundations and practical fixes—neutral, open, and without ranking promises.
Frequently asked questions
What does Metoz test?
Metoz compares a website’s rendered main text with the text received by an identified crawler without JavaScript. It also checks robots.txt rules for AI bots and asks six fixed questions about citability.
Is Metoz a complete SEO audit?
No. The scan specifically answers what an AI crawler may access and actually receives as main text. Rankings, backlinks, internal search architecture, and many traditional SEO factors are outside its scope.
Are scan pages indexed by search engines?
New scan permalinks are noindex by default. Indexing is enabled only after multiple independent requests. Corpus measurements are never released; their canonical page lives in the observatory.
Does Metoz respect robots.txt?
Yes. MetozBot identifies itself, checks robots.txt, and stops the crawler request when the target site blocks it.
The measurement method, limits, and correction process are documented under Methodology (DE). Contact: info@metoz.de.