SITECUBE /llms.txt Validator · AI Crawler Registry · Web-to-Token Observatory

sitecube.com

A+ (98/100) Open to Search & Training

Sitecube — /llms.txt Validator, AI Crawler Registry & Web-to-Token Observatory

Raw HTML Tokens: ~11,832 tok
Clean Text Tokens: ~1,132 tok (9.6% signal)
/llms.txt Manifest: 1,600 tok (Spec Compliant)
Edge / TTFB: Hostinger Edge CDN · 99 ms
AI & /llms.txt Readiness Signal-to-Noise Ratio Token Savings via /llms.txt Network TTFB TLS & HTTP Security DNS & Edge Stack
96/100 9.6% (1,132 / 11,832 tok) -86.5% fewer tokens 99 ms (98/100) 98/100 Hostinger Edge CDN (Custom SSR / Static)

Web-to-Token Efficiency & Agent Ingestion Cost

~4 bytes/token UTF-8 heuristic
Ingestion Source Payload Est. Tokens Cost / 1k Reads (Flash $0.35) Cost / 1k Reads (Pro $1.25)
Raw HTML DOM (GET /) 46.2 KB 11,832 $4.1412 $14.7900
Stripped Readable Text 4.4 KB 1,132 $0.3962 $1.4150
/llms.txt Manifest 6.2 KB 1,600 $0.5600 $2.0000
HTML Signal-to-Noise Ratio: 9.6% readable text vs. 90.4% markup/script overhead.

/llms.txt Spec Linter & Agent Endpoint Probe

Status: Spec Compliant
GET /llms.txt HTTP 200 · 6,398 bytes (~1,600 tokens · 40 lines)
Spec Check: # H1 Title Pass ("Sitecube.com — /llms.txt Validator, AI Crawler Registry & Web-to-Token Observatory")
Spec Check: > Summary Pass ("Sitecube.com is a technical observatory and specification linter that measures any domain’s HTML-to-LLM token footprint (`rawHtmlTokens` vs. `cleanTextTokens`), signal-to-noise rat")
## H2 Sections & Links 3 sections · 31 canonical markdown links
GET /llms-full.txt HTTP 200 · 6,398 bytes (~1,600 tokens)
GET /.well-known/mcp.json Active Model Context Protocol Manifest
JSON-LD Schema Entities Organization, ImageObject, WebSite, SearchAction, WebApplication
Live Payload from https://sitecube.com/llms.txt
# Sitecube.com — /llms.txt Validator, AI Crawler Registry & Web-to-Token Observatory

> Sitecube.com is a technical observatory and specification linter that measures any domain’s HTML-to-LLM token footprint (`rawHtmlTokens` vs. `cleanTextTokens`), signal-to-noise ratio, `/llms.txt` compliance, `/.well-known/mcp.json` discovery, AI crawler `robots.txt` directives across 10 bots (`OAI-SearchBot`, `PerplexityBot`, `ChatGPT-User`, `Claude-SearchBot`, `GPTBot`, `ClaudeBot`, `Google-Extended`, `Apple

AI Crawler /robots.txt Access Matrix

Open to Search & Training
User-Agent Token Purpose / Operator Effective Directive
OAI-SearchBot ChatGPT Search Citations Allow: /
PerplexityBot Perplexity Answer Citations Allow: /
ChatGPT-User Live User Agent Browsing Allow: /
Claude-SearchBot Anthropic Search Grounding Allow: /
GPTBot Foundation Model Training Allow: /
ClaudeBot Foundation Model Training Allow: /
Google-Extended Gemini Training Control Allow: /
Applebot-Extended Apple Intelligence Training Allow: /
Meta-ExternalAgent Llama Training & Indexing Allow: /
CCBot Open Dataset Training Corpus Allow: /

HTTP Response & Handshake Timing

GET https://sitecube.com/
DNS Lookup 62 ms HTML Document 46.2 KB
TCP Connect 7 ms Script Tags 4
TLS Handshake 14 ms Stylesheets 1
Time to First Byte (TTFB) 99 ms Images (with alt) 0 / 0
Total Transfer 100 ms HTTP/3 (Alt-Svc) Yes (h3)

Edge Network & Application Stack

Header & DOM Signatures
Edge / Reverse Proxy Hostinger Edge CDN
Application Framework Custom SSR / Static
Server Header hcdn
Mail Provider (MX) Custom MX
Telemetry Signatures Cloq
TXT Verifications Google Search Console

TLS Certificate & HTTP Security Policy

Port 443 X.509
Certificate Issuer Let's Encrypt (70 days remaining)
Strict-Transport-Security Present
Content-Security-Policy Present
X-Content-Type-Options Present
X-Frame-Options Present
Referrer-Policy Present
Permissions-Policy Present
SPF / DMARC Policy SPF: Yes · DMARC: Yes

Authoritative DNS & Document Head

A / AAAA / NS / Canonical
A (IPv4) 145.223.124.229, 147.79.79.172
AAAA (IPv6) 2a02:4780:20:d643:4f4b:f41c:151d:bdcd, 2a02:4780:1f:6fbc:fbd9:5f9:7706:7854
NS (Nameservers) ns1.dns-parking.com, ns2.dns-parking.com
HTML Title (78c) Sitecube — /llms.txt Validator, AI Crawler Registry & Web-to-Token Observatory
/sitemap.xml Valid XML Sitemap

Hosted /llms.txt Mirror & Status Badge

Platform Setup Guide →
1-Line <head> Tag (for Shopify, Webflow, Framer, Wix, Squarespace & WordPress):
Sitecube status for sitecube.com
GitHub / Docs Markdown Badge:
HTML Badge:

Technical Synthesis & Agentic Ingestion Analysis: sitecube.com

Automated Engineering Assessment

1. HTML-to-Token Efficiency & LLM Context Window Cost

An unauthenticated HTTPS GET request to https://sitecube.com/ returns an initial HTML payload of 46.2 KB, corresponding to approximately 11,832 UTF-8 tokens when ingested raw by an AI coding agent, retrieval pipeline, or headless browser. After stripping inline <script>, <style>, <svg>, and DOM markup tags, the human-readable text footprint measures 1,132 tokens (4.4 KB)—yielding an HTML Signal-to-Noise Ratio of 9.6%. Because sitecube.com publishes a valid /llms.txt manifest (measuring 1,600 tokens across 40 lines), AI retrieval agents that read /llms.txt instead of scraping the raw DOM reduce token consumption by 86.5%—cutting 1,000-read inference input costs from $14.7900 to $2.0000 at frontier model rates ($1.25/1M tokens), or $0.5600 on Gemini 4 Hydrogen ($0.35/1M).

2. Indexed Structure Inside sitecube.com/llms.txt

Sitecube’s specification linter verified 3 H2 section headings and 31 Markdown links inside https://sitecube.com/llms.txt under the primary heading "Sitecube.com — /llms.txt Validator, AI Crawler Registry & Web-to-Token Observatory". Documented sections include: Primary Research & Benchmarks</code>, <code>Interactive Engineering & FinOps Utilities</code>, <code>Verified Host Dossiers (24 Production Domains). Key canonical endpoints referenced in the manifest include The 2026 Web-to-Token Benchmark Study (https://sitecube.com/research/llms-txt-token-efficiency-2026), Global Host Registry & Token Leaderboard (https://sitecube.com/leaderboard), Methodology & Scoring Specification (https://sitecube.com/about), Live Host & Token Inspector (https://sitecube.com/), /llms.txt Specification Generator (https://sitecube.com/tools/llms-txt-generator).

3. Edge Delivery, TLS Handshake & Security Header Assessment

Traffic for sitecube.com resolves to 145.223.124.229 via Hostinger Edge CDN running Custom SSR / Static, achieving a measured Time to First Byte (TTFB) of 99 ms (DNS lookup: 62 ms, TCP connect: 7 ms, TLS handshake: 14 ms). The port 443 TLS certificate is issued by Let's Encrypt with 70 days remaining before expiration (valid through 2027-01-24). Outbound email authentication via SPF is present and DMARC (_dmarc.sitecube.com) is enforced.

Comparable Hosts in Web-to-Token & /llms.txt Registry

Full Index (26 hosts) →
Host Rating /llms.txt HTML Tokens Text Tokens Signal % Crawler Posture TTFB
sitecube.com A+ 98 1,600 tok 11,832 1,132 9.6% Open to Search & Training 99 ms
cloudflare.com A+ 96 4,294 tok 333,631 471 0.1% Open to Search & Training 291 ms
trendyflash.com A+ 95 2,987 tok 12,123 1,660 13.7% Open to Search & Training 112 ms
multifactor.com A+ 94 656 tok 98,275 673 0.7% Open to Search & Training 97 ms
github.com A+ 92 7,165 tok 131,504 1,159 0.9% Open to Search & Training 35 ms
vercel.com A+ 92 1,385 tok 149,859 889 0.6% Open to Search & Training 53 ms
aiimpacted.org A+ 92 971 tok 17,598 3,568 20.3% Open to Search & Training 55 ms
tierones.io A+ 92 481 tok 3,744 693 18.5% Open to Search & Training 95 ms
cloq.com A 91 1,169 tok 2,109 690 32.7% Open to Search & Training 51 ms
supabase.com A 89 879 tok 337,513 1,905 0.6% Open to Search & Training 48 ms
dosefacts.org A 89 10,936 tok 29,008 7,291 25.1% Open to Search & Training 335 ms
postman.com A 89 5,375 tok 118,892 11,040 9.3% Open to Search & Training 947 ms