Methodology & Web-to-Token Metrics
How Sitecube measures HTML token bloat, /llms.txt compliance, AI crawler policies, and edge response timings.
| Metric / Dimension | Weight | Verification Method |
|---|---|---|
| AI Manifests & Token Efficiency | 28% | Measures raw HTML token weight vs. stripped readable text tokens (Signal-to-Noise ratio), validates /llms.txt (# H1, > summary, ## sections, markdown links), probes /.well-known/mcp.json, and parses /robots.txt across 10 AI crawlers. |
| Network TTFB & Payload | 20% | cURL socket timing measurements for DNS lookup, TCP handshake, TLS negotiation, and Time to First Byte (TTFB). |
| TLS & HTTP Security | 18% | Port 443 OpenSSL peer certificate inspection and presence of HSTS, CSP, X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Permissions-Policy, SPF, and DMARC. |
| Document Metadata | 14% | HTML <title>, meta[name=description], canonical link, heading structure, OpenGraph tags, and XML sitemap verification. |
| DNS & Edge Stack | 20% | Authoritative DNS queries (A, AAAA, NS, MX, TXT) and reverse-proxy/CDN signature detection. |