Posts tagged "dataset"

23 posts found.

July 23, 2026 22 min read

Who Moves the World's DNS: 15 Months of Delegation Flux, a One-Month Million-Domain Migration, and the Nameservers That Outlive Their Owners

We followed the NS delegation of every apex domain across nine full crawls, April 2025 - July 2026 (404 million apexes). 61% of the web kept the exact same nameservers for fifteen months; operator switching runs at 2.3% a month; and the biggest moves were corporate re-plumbs - GoDaddy consolidating 2.1 million NameFind domains onto Afternic, Namecheap shifting 1.1 million parked names to an undocumented platform in one month. Meanwhile 339,000 domains still delegate to the retired Dan.com. For DNS operators, security teams, and Internet measurement researchers.

July 18, 2026 21 min read

The SaaS Ledger: 15 Months of TXT Records, a 625x AI Adoption Curve, and Five Million Tokens Moved by One Corporate Decision

We re-classified 18 billion DNS TXT queries across ten full crawls (April 2025 - July 2026) and turned the SaaS verification-token census into a time series. Anthropic tokens grew 625x to 131,812 apexes. The Dan.com shutdown moved five million marketplace tokens. Microsoft 365's domain share is flat while Google's climbs. And across every major vendor, only 19-29% of token disappearances are deliberate cleanup - the rest is domain death. For SaaS analysts, security teams, and Internet measurement researchers.

July 11, 2026 17 min read

The Restless Aliases: the CNAME Layer re-crawled after 32 Days — 92.5% Held Still, 12.6 Million Rewired or Vanished, and the Biggest Moves Were Made by Platforms, Not People

Two CNAME-typed crawls of 1.9 billion hostnames, 32 days apart: 169 million cross-domain aliases followed by name. 7.5% of the alias layer rewires or disappears in one month, parking lost 1.54M aliases, one Chinese registrar generated 13% of all new aliases — and the three biggest 'migrations' were platforms re-pointing their own chains, dragging half a million customer domains along. For DNS operators, measurement researchers, and anyone who builds on someone else's CNAME.

July 10, 2026 18 min read

The Parking Lot: We Measured the Parked Web Every Month for 38 Months — It Grew to One Domain in Four, Peaked Five Days After Its Business Model Died, and Has Been Shrinking Since

26 monthly DNS snapshots, April 2023 to June 2026: domains pointing at parking, for-sale, expiry and registrar-default landers grew from 16.0% to a peak of 25.5% of all website records — 73.6 million domains — then began the first sustained decline on record, in the exact quarter Google switched off the parking ad economy and Bodis shut down. GoDaddy front doors hold three-quarters of the lot. For measurement researchers, registrars, security teams, and anyone who counts 'websites'.

July 06, 2026 22 min read

Where the Web Moves: We Followed 110 Million Domains for 38 Months — Two-Thirds Never Left Their Host, and One in Four Domains Now Points at a Parking Lot

We compared the website address of every domain that resolved in both April 2023 and June 2026 — 110.6 million survivors — and classified both endpoints by BGP network. 46.8% answer from the exact same IP three years later; 67.7% never left their provider. The biggest migration wasn't chosen by any webmaster: GoDaddy moved its 10-million-domain lander estate from Google Cloud to AWS anycast, and the parked web doubled to 24.8% of all site records. For hosting strategists, registrars, measurement researchers, and anyone who thinks the cloud wars are moving the web.

July 03, 2026 20 min read

The AI Crawler Census: 14 Bots, 97,454 Requests, and More Knocks Than Every Search Engine Combined

We logged 1,348,706 requests across two production web properties for 174 days and classified every user-agent. AI crawlers now generate 7.2% of all observed traffic — more than Googlebot, Bingbot, DuckDuckBot, Baiduspider, and PetalBot combined. OpenAI alone accounts for 38% of the AI crawl, 69% of AI requests still feed training pipelines, and a robots.txt honeypot shows 99% of AI-crawler violations trace to a single bot. For publishers, infrastructure teams, and anyone deciding what to do about the new tenants of the crawl economy.

July 02, 2026 18 min read

The Half-Life of a Domain: We Tracked 181 Million Resolving Websites for 38 Months — 63.5% Are Still Alive

We followed every one of the 181.1 million domains that resolved in April 2023 across 26 DNS snapshots to June 2026, and did the same for a 246.8-million-domain 2025 cohort. The resolving web loses 13-15% of itself every year — a half-life of roughly five years — but the deaths are wildly unequal: Freenom's five TLDs went 99% extinct in the largest die-off ever recorded in the DNS, the cheap-promo gTLDs lose up to 95% of their web in three years, and Germany's .de keeps 84% alive. For registrars, investors, security teams, and anyone who links to anything.

June 27, 2026 23 min read

Who Controls the Internet in 2026? Four Core Layers, Four Different Owners, and a 92% Single Point of Failure

The first edition of the Internet Concentration Index: a synthesis of four full-corpus, typed-DNS censuses of the same ~3-billion-name dataset, measuring who controls the Internet's core layers one domain at a time. DNS, email, cloud hosting and the CDN/proxy edge each have a different owner — GoDaddy, Google, Amazon and Cloudflare — no single company controls all four, the enterprise giant everyone names (Microsoft) is nearly invisible by domain count, and 92.5% of delegated domains sit on a single point of failure. For policymakers, security teams, and anyone who has watched one outage take down half the web.

June 26, 2026 21 min read

Where the Web Lives, 2023–2026: The Map That Refused to Move

We replayed 26 monthly snapshots of where the world's domains resolve — 38 months of A-record and geoIP data, Apr 2023 to Jun 2026 — counted one domain one vote across tens of millions of apexes per month. The expected story was de-Americanization and cloud consolidation. The data tells a stranger one: the geographic map barely moved. The United States hosted 45.9% of resolving domains in 2023 and 45.2% in 2026. The only line that climbed is the anycast bucket that hides where a domain truly lives, and the only provider that genuinely surged is Amazon, which tripled its footprint while its share of cloud spending fell. This is the first longitudinal post in the series.

Page 1 of 3 Older →