{"@context":"https://schema.org","@type":"Dataset","name":"Crawled — which machines read a small website","description":"First-party record of AI and search crawler fetches against projectnothing.ai, at page granularity, with time from publication to first machine read, and a fixed holdout of pages published without a machine-readable brief.","url":"https://www.projectnothing.ai/experiments/crawled","license":"https://creativecommons.org/licenses/by/4.0/","creator":{"@type":"Organization","name":"Project Nothing","url":"https://www.projectnothing.ai"},"isPartOf":{"@type":"WebSite","name":"Project Nothing","url":"https://www.projectnothing.ai"},"experiment":"https://www.projectnothing.ai/e/crawled","method":{"collection":"Every request to a tracked public path passes through edge middleware, which matches the user-agent against a fixed table and records one row. Requests are counted before any cache, which is the only place these fetches are visible at all.","identification":"A user-agent is a claim, not a proof. Nothing here is verified by reverse DNS or published IP range, so an agent that lies is recorded as whatever it said.","excluded":"Agents not in the table are not counted, and neither are paths outside the tracked set. An unknown-bot bucket would be mostly scrapers and uptime checks, and putting that beside GPTBot would flatter the result.","arms":"Half of the experiment story pages publish a machine-readable brief and are listed in /llms.txt; half deliberately do not. Assignment is the parity of the experiment number — even numbers publish a brief, odd numbers do not — fixed before measurement began. Experiment numbers run in time order, so the arms are balanced and stratified by age at once. The two arms are not compared until each has at least 3 pages read by the answer layer.","absence":"A path no machine has asked for reports null, never zero. \"Nobody came\" and \"we have not watched long enough\" are different facts.","cannotSee":"Whether any of this reaches an answer. Nothing here observes a citation, a recommendation, or a purchase made by an agent — those require a channel that does not exist, and no number on this page should be read as measuring them."},"taxonomy":[{"id":"chatgpt-user","label":"ChatGPT-User","operator":"OpenAI","purpose":"answer","purposeLabel":"Fetching live to answer somebody","userAgentContains":"ChatGPT-User"},{"id":"oai-searchbot","label":"OAI-SearchBot","operator":"OpenAI","purpose":"ai-search","purposeLabel":"Indexing for AI answers","userAgentContains":"OAI-SearchBot"},{"id":"gptbot","label":"GPTBot","operator":"OpenAI","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"GPTBot"},{"id":"claude-user","label":"Claude-User","operator":"Anthropic","purpose":"answer","purposeLabel":"Fetching live to answer somebody","userAgentContains":"Claude-User"},{"id":"claude-searchbot","label":"Claude-SearchBot","operator":"Anthropic","purpose":"ai-search","purposeLabel":"Indexing for AI answers","userAgentContains":"Claude-SearchBot"},{"id":"claudebot","label":"ClaudeBot","operator":"Anthropic","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"ClaudeBot"},{"id":"anthropic-ai","label":"anthropic-ai","operator":"Anthropic","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"anthropic-ai"},{"id":"perplexity-user","label":"Perplexity-User","operator":"Perplexity","purpose":"answer","purposeLabel":"Fetching live to answer somebody","userAgentContains":"Perplexity-User"},{"id":"perplexitybot","label":"PerplexityBot","operator":"Perplexity","purpose":"ai-search","purposeLabel":"Indexing for AI answers","userAgentContains":"PerplexityBot"},{"id":"google-extended","label":"Google-Extended","operator":"Google","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"Google-Extended"},{"id":"googleother","label":"GoogleOther","operator":"Google","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"GoogleOther"},{"id":"google-inspectiontool","label":"Google-InspectionTool","operator":"Google","purpose":"search","purposeLabel":"Indexing for a search engine","userAgentContains":"Google-InspectionTool"},{"id":"googlebot","label":"Googlebot","operator":"Google","purpose":"search","purposeLabel":"Indexing for a search engine","userAgentContains":"Googlebot"},{"id":"bingbot","label":"bingbot","operator":"Microsoft","purpose":"search","purposeLabel":"Indexing for a search engine","userAgentContains":"bingbot"},{"id":"applebot-extended","label":"Applebot-Extended","operator":"Apple","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"Applebot-Extended"},{"id":"applebot","label":"Applebot","operator":"Apple","purpose":"search","purposeLabel":"Indexing for a search engine","userAgentContains":"Applebot"},{"id":"meta-externalagent","label":"meta-externalagent","operator":"Meta","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"meta-externalagent"},{"id":"facebookbot","label":"FacebookBot","operator":"Meta","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"FacebookBot"},{"id":"bytespider","label":"Bytespider","operator":"ByteDance","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"Bytespider"},{"id":"amazonbot","label":"Amazonbot","operator":"Amazon","purpose":"ai-search","purposeLabel":"Indexing for AI answers","userAgentContains":"Amazonbot"},{"id":"mistralai-user","label":"MistralAI-User","operator":"Mistral","purpose":"answer","purposeLabel":"Fetching live to answer somebody","userAgentContains":"MistralAI-User"},{"id":"duckassistbot","label":"DuckAssistBot","operator":"DuckDuckGo","purpose":"ai-search","purposeLabel":"Indexing for AI answers","userAgentContains":"DuckAssistBot"},{"id":"duckduckbot","label":"DuckDuckBot","operator":"DuckDuckGo","purpose":"search","purposeLabel":"Indexing for a search engine","userAgentContains":"DuckDuckBot"},{"id":"youbot","label":"YouBot","operator":"You.com","purpose":"ai-search","purposeLabel":"Indexing for AI answers","userAgentContains":"YouBot"},{"id":"cohere-ai","label":"cohere-ai","operator":"Cohere","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"cohere-ai"},{"id":"diffbot","label":"Diffbot","operator":"Diffbot","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"Diffbot"},{"id":"ai2bot","label":"AI2Bot","operator":"Allen Institute","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"AI2Bot"},{"id":"timpibot","label":"Timpibot","operator":"Timpi","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"Timpibot"},{"id":"omgili","label":"omgili","operator":"Webz.io","purpose":"training","purposeLabel":"Building a training corpus","userAgentContains":"omgili"},{"id":"ccbot","label":"CCBot","operator":"Common Crawl","purpose":"archive","purposeLabel":"Archiving the web","userAgentContains":"CCBot"},{"id":"ia_archiver","label":"ia_archiver","operator":"Internet Archive","purpose":"archive","purposeLabel":"Archiving the web","userAgentContains":"ia_archiver"},{"id":"yandexbot","label":"YandexBot","operator":"Yandex","purpose":"search","purposeLabel":"Indexing for a search engine","userAgentContains":"YandexBot"},{"id":"petalbot","label":"PetalBot","operator":"Huawei","purpose":"search","purposeLabel":"Indexing for a search engine","userAgentContains":"PetalBot"}],"ledger":{"generatedAt":"2026-09-01T08:26:07.489Z","configured":true,"measuring":true,"startedAt":"2026-08-30T11:45:16.041Z","totals":{"fetches":225,"agents":11,"answerFetches":149,"searchFetches":76,"pagesTracked":27},"agents":[{"id":"bytespider","label":"Bytespider","operator":"ByteDance","purpose":"training","purposeLabel":"Building a training corpus","layer":"answer","fetches":58,"lastSeenAt":"2026-09-01T08:14:47.372Z"},{"id":"claudebot","label":"ClaudeBot","operator":"Anthropic","purpose":"training","purposeLabel":"Building a training corpus","layer":"answer","fetches":53,"lastSeenAt":"2026-09-01T08:26:07.055Z"},{"id":"bingbot","label":"bingbot","operator":"Microsoft","purpose":"search","purposeLabel":"Indexing for a search engine","layer":"search","fetches":48,"lastSeenAt":"2026-09-01T06:41:00.151Z"},{"id":"gptbot","label":"GPTBot","operator":"OpenAI","purpose":"training","purposeLabel":"Building a training corpus","layer":"answer","fetches":15,"lastSeenAt":"2026-09-01T00:49:39.505Z"},{"id":"oai-searchbot","label":"OAI-SearchBot","operator":"OpenAI","purpose":"ai-search","purposeLabel":"Indexing for AI answers","layer":"answer","fetches":14,"lastSeenAt":"2026-09-01T07:19:59.332Z"},{"id":"petalbot","label":"PetalBot","operator":"Huawei","purpose":"search","purposeLabel":"Indexing for a search engine","layer":"search","fetches":11,"lastSeenAt":"2026-09-01T07:34:51.236Z"},{"id":"duckduckbot","label":"DuckDuckBot","operator":"DuckDuckGo","purpose":"search","purposeLabel":"Indexing for a search engine","layer":"search","fetches":9,"lastSeenAt":"2026-09-01T07:08:49.403Z"},{"id":"amazonbot","label":"Amazonbot","operator":"Amazon","purpose":"ai-search","purposeLabel":"Indexing for AI answers","layer":"answer","fetches":5,"lastSeenAt":"2026-09-01T07:00:59.722Z"},{"id":"applebot","label":"Applebot","operator":"Apple","purpose":"search","purposeLabel":"Indexing for a search engine","layer":"search","fetches":4,"lastSeenAt":"2026-09-01T07:57:26.210Z"},{"id":"chatgpt-user","label":"ChatGPT-User","operator":"OpenAI","purpose":"answer","purposeLabel":"Fetching live to answer somebody","layer":"answer","fetches":4,"lastSeenAt":"2026-08-31T16:51:11.662Z"},{"id":"googlebot","label":"Googlebot","operator":"Google","purpose":"search","purposeLabel":"Indexing for a search engine","layer":"search","fetches":4,"lastSeenAt":"2026-08-31T15:37:08.114Z"}],"pages":[{"slug":"needed","name":"Needed","arm":"brief","armLabel":"Machine-readable brief","path":"/e/needed","briefPath":"/e/needed/brief.json","publishedAt":"2026-08-31T15:53:00.000Z","fetches":2,"agents":["bingbot","ClaudeBot"],"firstAnswerLayer":{"at":"2026-09-01T07:24:35.832Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":15.5},"firstSearchLayer":{"at":"2026-09-01T02:46:49.880Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":10.9}},{"slug":"slop","name":"Slop","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/slop","briefPath":null,"publishedAt":"2026-08-30T17:40:00.000Z","fetches":3,"agents":["bingbot","ClaudeBot","GPTBot"],"firstAnswerLayer":{"at":"2026-08-31T02:09:04.316Z","agent":"GPTBot","operator":"OpenAI","hoursAfterPublish":8.5},"firstSearchLayer":{"at":"2026-08-30T21:40:51.025Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":4}},{"slug":"done","name":"Done","arm":"brief","armLabel":"Machine-readable brief","path":"/e/done","briefPath":"/e/done/brief.json","publishedAt":"2026-08-30T17:00:00.000Z","fetches":3,"agents":["bingbot","ClaudeBot","GPTBot"],"firstAnswerLayer":{"at":"2026-08-31T02:09:13.520Z","agent":"GPTBot","operator":"OpenAI","hoursAfterPublish":9.2},"firstSearchLayer":{"at":"2026-08-30T21:40:58.006Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":4.7}},{"slug":"up-peak","name":"Up Peak","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/up-peak","briefPath":null,"publishedAt":"2026-08-30T16:30:00.000Z","fetches":4,"agents":["bingbot","ClaudeBot","GPTBot","PetalBot"],"firstAnswerLayer":{"at":"2026-08-31T02:09:11.388Z","agent":"GPTBot","operator":"OpenAI","hoursAfterPublish":9.7},"firstSearchLayer":{"at":"2026-08-30T21:41:02.312Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":5.2}},{"slug":"crawled","name":"Crawled","arm":"brief","armLabel":"Machine-readable brief","path":"/e/crawled","briefPath":"/e/crawled/brief.json","publishedAt":"2026-08-30T14:00:00.000Z","fetches":4,"agents":["bingbot","ClaudeBot","GPTBot","PetalBot"],"firstAnswerLayer":{"at":"2026-08-31T02:09:12.432Z","agent":"GPTBot","operator":"OpenAI","hoursAfterPublish":12.2},"firstSearchLayer":{"at":"2026-08-30T21:41:01.176Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":7.7}},{"slug":"out-of-ten","name":"Out of Ten","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/out-of-ten","briefPath":null,"publishedAt":"2026-08-30T09:00:00.000Z","fetches":3,"agents":["bingbot","ClaudeBot","GPTBot"],"firstAnswerLayer":{"at":"2026-08-31T02:09:10.168Z","agent":"GPTBot","operator":"OpenAI","hoursAfterPublish":17.2},"firstSearchLayer":{"at":"2026-08-30T21:41:00.375Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":12.7}},{"slug":"perfect-score","name":"Perfect Score","arm":"brief","armLabel":"Machine-readable brief","path":"/e/perfect-score","briefPath":"/e/perfect-score/brief.json","publishedAt":"2026-08-29T10:00:00.000Z","fetches":2,"agents":["ClaudeBot","GPTBot"],"firstAnswerLayer":{"at":"2026-08-31T02:09:15.437Z","agent":"GPTBot","operator":"OpenAI","hoursAfterPublish":40.2},"firstSearchLayer":null},{"slug":"insula","name":"Insula","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/insula","briefPath":null,"publishedAt":"2026-08-28T09:00:00.000Z","fetches":2,"agents":["ClaudeBot","PetalBot"],"firstAnswerLayer":{"at":"2026-09-01T07:24:36.770Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":94.4},"firstSearchLayer":{"at":"2026-08-31T12:44:43.790Z","agent":"PetalBot","operator":"Huawei","hoursAfterPublish":75.7}},{"slug":"exactly","name":"Exactly","arm":"brief","armLabel":"Machine-readable brief","path":"/e/exactly","briefPath":"/e/exactly/brief.json","publishedAt":"2026-08-26T10:00:00.000Z","fetches":2,"agents":["ClaudeBot","PetalBot"],"firstAnswerLayer":{"at":"2026-09-01T07:24:50.512Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":141.4},"firstSearchLayer":{"at":"2026-08-31T07:28:55.195Z","agent":"PetalBot","operator":"Huawei","hoursAfterPublish":117.5}},{"slug":"spent","name":"Spent","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/spent","briefPath":null,"publishedAt":"2026-08-25T09:00:00.000Z","fetches":2,"agents":["ClaudeBot","PetalBot"],"firstAnswerLayer":{"at":"2026-09-01T07:25:06.664Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":166.4},"firstSearchLayer":{"at":"2026-08-31T05:17:00.565Z","agent":"PetalBot","operator":"Huawei","hoursAfterPublish":140.3}},{"slug":"sieve","name":"Sieve","arm":"brief","armLabel":"Machine-readable brief","path":"/e/sieve","briefPath":"/e/sieve/brief.json","publishedAt":"2026-08-24T10:00:00.000Z","fetches":5,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-30T12:16:41.296Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":146.3},"firstSearchLayer":{"at":"2026-08-30T15:35:02.992Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":149.6}},{"slug":"hold-the-note","name":"Hold the Note","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/hold-the-note","briefPath":null,"publishedAt":"2026-08-22T09:00:00.000Z","fetches":2,"agents":["Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-30T17:26:28.573Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":200.4},"firstSearchLayer":null},{"slug":"signal-weather","name":"Signal Weather","arm":"brief","armLabel":"Machine-readable brief","path":"/e/signal-weather","briefPath":"/e/signal-weather/brief.json","publishedAt":"2026-05-20T09:00:00.000Z","fetches":3,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-31T10:45:54.363Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2473.8},"firstSearchLayer":{"at":"2026-08-31T08:34:23.673Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2471.6}},{"slug":"ai-deas","name":"AI-deas","arm":"brief","armLabel":"Machine-readable brief","path":"/e/ai-deas","briefPath":"/e/ai-deas/brief.json","publishedAt":"2026-05-11T09:00:00.000Z","fetches":2,"agents":["Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-31T07:10:37.407Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2686.2},"firstSearchLayer":null},{"slug":"built-from-a-phone","name":"Built From a Phone","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/built-from-a-phone","briefPath":null,"publishedAt":"2026-05-04T17:00:00.000Z","fetches":3,"agents":["bingbot","ClaudeBot"],"firstAnswerLayer":{"at":"2026-09-01T07:26:02.634Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":2870.4},"firstSearchLayer":{"at":"2026-08-30T19:07:45.572Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2834.1}},{"slug":"micro-break","name":"Micro-Break","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/micro-break","briefPath":null,"publishedAt":"2026-05-04T16:00:00.000Z","fetches":4,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-31T08:31:11.837Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2848.5},"firstSearchLayer":{"at":"2026-08-30T18:02:49.484Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2834}},{"slug":"code-of-laws","name":"Code of Laws","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/code-of-laws","briefPath":null,"publishedAt":"2026-05-04T15:00:00.000Z","fetches":3,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-31T01:00:29.261Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2842},"firstSearchLayer":{"at":"2026-09-01T04:46:57.937Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2869.8}},{"slug":"llm-price-volatility","name":"LLM Price Volatility","arm":"brief","armLabel":"Machine-readable brief","path":"/e/llm-price-volatility","briefPath":"/e/llm-price-volatility/brief.json","publishedAt":"2026-05-04T14:00:00.000Z","fetches":4,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-30T12:34:30.470Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2830.6},"firstSearchLayer":{"at":"2026-09-01T04:34:09.722Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2870.6}},{"slug":"voidle","name":"Voidle","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/voidle","briefPath":null,"publishedAt":"2026-05-04T13:00:00.000Z","fetches":4,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-30T18:54:10.763Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2837.9},"firstSearchLayer":{"at":"2026-08-30T17:02:35.052Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2836}},{"slug":"velocity-cards","name":"Velocity Cards","arm":"brief","armLabel":"Machine-readable brief","path":"/e/velocity-cards","briefPath":"/e/velocity-cards/brief.json","publishedAt":"2026-05-04T12:00:00.000Z","fetches":2,"agents":["Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-30T18:13:49.574Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2838.2},"firstSearchLayer":null},{"slug":"instruction-elasticity","name":"Instruction Elasticity Index","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/instruction-elasticity","briefPath":null,"publishedAt":"2026-05-04T11:00:00.000Z","fetches":1,"agents":["ClaudeBot"],"firstAnswerLayer":{"at":"2026-09-01T07:25:16.240Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":2876.4},"firstSearchLayer":null},{"slug":"the-replacement-index","name":"The Replacement Index","arm":"brief","armLabel":"Machine-readable brief","path":"/e/the-replacement-index","briefPath":"/e/the-replacement-index/brief.json","publishedAt":"2026-05-04T10:00:00.000Z","fetches":2,"agents":["Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-30T22:19:29.378Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2844.3},"firstSearchLayer":null},{"slug":"signal-from-the-void","name":"Signal from the Void","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/signal-from-the-void","briefPath":null,"publishedAt":"2026-05-04T09:00:00.000Z","fetches":2,"agents":["Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-30T14:28:24.527Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2837.5},"firstSearchLayer":null},{"slug":"subscribe-to-nothing","name":"Subscribe to Nothing","arm":"brief","armLabel":"Machine-readable brief","path":"/e/subscribe-to-nothing","briefPath":"/e/subscribe-to-nothing/brief.json","publishedAt":"2026-05-04T08:00:00.000Z","fetches":3,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-31T04:51:42.874Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2852.9},"firstSearchLayer":{"at":"2026-08-31T04:16:11.960Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2852.3}},{"slug":"make-interesting","name":"Make Interesting","arm":"holdout","armLabel":"HTML only (holdout)","path":"/e/make-interesting","briefPath":null,"publishedAt":"2026-05-04T18:00:00.000Z","fetches":5,"agents":["bingbot","Bytespider","ClaudeBot"],"firstAnswerLayer":{"at":"2026-08-31T00:49:27.353Z","agent":"Bytespider","operator":"ByteDance","hoursAfterPublish":2838.8},"firstSearchLayer":{"at":"2026-08-30T11:45:16.041Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":2825.8}},{"slug":"on-me","name":"On Me","arm":"brief","armLabel":"Machine-readable brief","path":"/e/on-me","briefPath":"/e/on-me/brief.json","publishedAt":"2026-08-30T19:00:00.000Z","fetches":2,"agents":["bingbot","ClaudeBot"],"firstAnswerLayer":{"at":"2026-09-01T07:59:25.492Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":37},"firstSearchLayer":{"at":"2026-09-01T02:46:50.768Z","agent":"bingbot","operator":"Microsoft","hoursAfterPublish":31.8}},{"slug":"mutation-observer","name":"Mutation Observer","arm":"brief","armLabel":"Machine-readable brief","path":"/e/mutation-observer","briefPath":"/e/mutation-observer/brief.json","publishedAt":"2026-05-04T19:00:00.000Z","fetches":1,"agents":["ClaudeBot"],"firstAnswerLayer":{"at":"2026-09-01T07:24:51.642Z","agent":"ClaudeBot","operator":"Anthropic","hoursAfterPublish":2868.4},"firstSearchLayer":null}],"arms":{"brief":{"arm":"brief","label":"Machine-readable brief","pages":14,"pagesRead":14,"medianHoursToFirstRead":1310.1,"answerFetches":37},"holdout":{"arm":"holdout","label":"HTML only (holdout)","pages":13,"pagesRead":13,"medianHoursToFirstRead":2837.5,"answerFetches":38},"comparison":"The answer layer has reached 100% of the pages that publish a brief and 100% of the pages that do not. Median time from publishing to first machine read: 1310.1h with a brief, 2837.5h without. This is a difference between two small groups on one small site, not a finding about the web.","comparable":true},"machineSurfaces":[{"path":"/llms.txt","fetches":1,"agents":["GPTBot"]}],"recent":[{"at":"2026-09-01T08:26:07.055Z","agent":"ClaudeBot","operator":"Anthropic","path":"/robots.txt"},{"at":"2026-09-01T08:18:10.008Z","agent":"ClaudeBot","operator":"Anthropic","path":"/sitemap.xml"},{"at":"2026-09-01T08:14:47.372Z","agent":"Bytespider","operator":"ByteDance","path":"/robots.txt"},{"at":"2026-09-01T08:02:47.472Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/out-of-ten"},{"at":"2026-09-01T08:01:22.853Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/i-built-a-game-from-my-phone"},{"at":"2026-09-01T08:01:22.580Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/signal-from-the-void"},{"at":"2026-09-01T08:00:59.122Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/llm-price-volatility"},{"at":"2026-09-01T08:00:58.748Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/velocity-cards"},{"at":"2026-09-01T08:00:56.493Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/code-of-laws"},{"at":"2026-09-01T08:00:56.120Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/the-replacement-index"},{"at":"2026-09-01T08:00:27.212Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/instruction-elasticity"},{"at":"2026-09-01T08:00:26.816Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/voidle"},{"at":"2026-09-01T07:59:25.492Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/on-me"},{"at":"2026-09-01T07:59:25.031Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/up-peak"},{"at":"2026-09-01T07:59:18.003Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/hold-the-note"},{"at":"2026-09-01T07:59:17.513Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/perfect-score"},{"at":"2026-09-01T07:59:17.072Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/slop"},{"at":"2026-09-01T07:59:07.768Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/sieve"},{"at":"2026-09-01T07:59:07.260Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/spent"},{"at":"2026-09-01T07:59:06.829Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/crawled"},{"at":"2026-09-01T07:59:01.215Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/done"},{"at":"2026-09-01T07:59:00.703Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/insula"},{"at":"2026-09-01T07:59:00.488Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/needed"},{"at":"2026-09-01T07:57:26.210Z","agent":"Applebot","operator":"Apple","path":"/robots.txt"},{"at":"2026-09-01T07:34:51.236Z","agent":"PetalBot","operator":"Huawei","path":"/robots.txt"},{"at":"2026-09-01T07:26:32.646Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/out-of-ten"},{"at":"2026-09-01T07:26:32.152Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/signal-weather"},{"at":"2026-09-01T07:26:31.768Z","agent":"ClaudeBot","operator":"Anthropic","path":"/"},{"at":"2026-09-01T07:26:25.932Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/signal-from-the-void"},{"at":"2026-09-01T07:26:25.577Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/llm-price-volatility"},{"at":"2026-09-01T07:26:02.634Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/built-from-a-phone"},{"at":"2026-09-01T07:25:55.589Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/velocity-cards"},{"at":"2026-09-01T07:25:54.944Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/the-replacement-index"},{"at":"2026-09-01T07:25:54.885Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/code-of-laws"},{"at":"2026-09-01T07:25:37.723Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/subscribe-to-nothing"},{"at":"2026-09-01T07:25:36.933Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/ai-deas"},{"at":"2026-09-01T07:25:16.240Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/instruction-elasticity"},{"at":"2026-09-01T07:25:15.725Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/up-peak"},{"at":"2026-09-01T07:25:15.257Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/hold-the-note"},{"at":"2026-09-01T07:25:15.005Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/perfect-score"},{"at":"2026-09-01T07:25:14.159Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/slop"},{"at":"2026-09-01T07:25:07.171Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/sieve"},{"at":"2026-09-01T07:25:06.664Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/spent"},{"at":"2026-09-01T07:25:06.519Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/micro-break"},{"at":"2026-09-01T07:24:51.642Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/mutation-observer"},{"at":"2026-09-01T07:24:50.672Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/make-interesting"},{"at":"2026-09-01T07:24:50.512Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/exactly"},{"at":"2026-09-01T07:24:42.498Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/crawled"},{"at":"2026-09-01T07:24:41.995Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/done"},{"at":"2026-09-01T07:24:41.564Z","agent":"ClaudeBot","operator":"Anthropic","path":"/experiments/exactly"},{"at":"2026-09-01T07:24:36.770Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/insula"},{"at":"2026-09-01T07:24:36.331Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/voidle"},{"at":"2026-09-01T07:24:35.832Z","agent":"ClaudeBot","operator":"Anthropic","path":"/e/needed"},{"at":"2026-09-01T07:24:34.860Z","agent":"ClaudeBot","operator":"Anthropic","path":"/robots.txt"},{"at":"2026-09-01T07:19:59.332Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/robots.txt"},{"at":"2026-09-01T07:08:49.403Z","agent":"DuckDuckBot","operator":"DuckDuckGo","path":"/"},{"at":"2026-09-01T07:00:59.722Z","agent":"Amazonbot","operator":"Amazon","path":"/experiments/needed"},{"at":"2026-09-01T06:49:46.097Z","agent":"ClaudeBot","operator":"Anthropic","path":"/"},{"at":"2026-09-01T06:49:45.815Z","agent":"ClaudeBot","operator":"Anthropic","path":"/robots.txt"},{"at":"2026-09-01T06:41:00.151Z","agent":"bingbot","operator":"Microsoft","path":"/"},{"at":"2026-09-01T06:06:34.254Z","agent":"Bytespider","operator":"ByteDance","path":"/robots.txt"},{"at":"2026-09-01T06:06:02.464Z","agent":"Bytespider","operator":"ByteDance","path":"/robots.txt"},{"at":"2026-09-01T04:46:57.937Z","agent":"bingbot","operator":"Microsoft","path":"/e/code-of-laws"},{"at":"2026-09-01T04:45:38.544Z","agent":"PetalBot","operator":"Huawei","path":"/sitemap.xml"},{"at":"2026-09-01T04:34:09.722Z","agent":"bingbot","operator":"Microsoft","path":"/e/llm-price-volatility"},{"at":"2026-09-01T04:21:28.047Z","agent":"bingbot","operator":"Microsoft","path":"/sitemap.xml"},{"at":"2026-09-01T04:21:02.123Z","agent":"bingbot","operator":"Microsoft","path":"/robots.txt"},{"at":"2026-09-01T04:21:01.661Z","agent":"bingbot","operator":"Microsoft","path":"/robots.txt"},{"at":"2026-09-01T03:07:22.495Z","agent":"bingbot","operator":"Microsoft","path":"/e/sieve"},{"at":"2026-09-01T03:03:42.098Z","agent":"Bytespider","operator":"ByteDance","path":"/experiments/velocity-cards"},{"at":"2026-09-01T02:53:29.678Z","agent":"PetalBot","operator":"Huawei","path":"/e/crawled"},{"at":"2026-09-01T02:52:06.858Z","agent":"PetalBot","operator":"Huawei","path":"/experiments/insula"},{"at":"2026-09-01T02:46:50.768Z","agent":"bingbot","operator":"Microsoft","path":"/e/on-me"},{"at":"2026-09-01T02:46:49.880Z","agent":"bingbot","operator":"Microsoft","path":"/e/needed"},{"at":"2026-09-01T02:44:29.688Z","agent":"bingbot","operator":"Microsoft","path":"/sitemap.xml"},{"at":"2026-09-01T02:30:42.919Z","agent":"Bytespider","operator":"ByteDance","path":"/robots.txt"},{"at":"2026-09-01T01:58:06.675Z","agent":"PetalBot","operator":"Huawei","path":"/experiments/exactly"},{"at":"2026-09-01T01:45:01.404Z","agent":"Bytespider","operator":"ByteDance","path":"/robots.txt"},{"at":"2026-09-01T00:49:39.505Z","agent":"GPTBot","operator":"OpenAI","path":"/sitemap.xml"},{"at":"2026-09-01T00:49:39.371Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/robots.txt"},{"at":"2026-09-01T00:38:42.464Z","agent":"bingbot","operator":"Microsoft","path":"/experiments/sieve"},{"at":"2026-08-31T23:55:13.746Z","agent":"bingbot","operator":"Microsoft","path":"/e/built-from-a-phone"},{"at":"2026-08-31T23:51:41.179Z","agent":"PetalBot","operator":"Huawei","path":"/robots.txt"},{"at":"2026-08-31T23:16:35.630Z","agent":"bingbot","operator":"Microsoft","path":"/e/make-interesting"},{"at":"2026-08-31T22:56:38.695Z","agent":"bingbot","operator":"Microsoft","path":"/experiments/signal-from-the-void"},{"at":"2026-08-31T22:46:13.934Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/"},{"at":"2026-08-31T22:14:25.156Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/"},{"at":"2026-08-31T22:14:22.163Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/experiments/signal-from-the-void"},{"at":"2026-08-31T22:14:22.128Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/robots.txt"},{"at":"2026-08-31T22:12:28.421Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/robots.txt"},{"at":"2026-08-31T22:09:55.909Z","agent":"bingbot","operator":"Microsoft","path":"/experiments/code-of-laws"},{"at":"2026-08-31T21:30:09.925Z","agent":"Amazonbot","operator":"Amazon","path":"/"},{"at":"2026-08-31T21:07:05.980Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/robots.txt"},{"at":"2026-08-31T20:52:09.323Z","agent":"bingbot","operator":"Microsoft","path":"/e/micro-break"},{"at":"2026-08-31T19:53:18.312Z","agent":"bingbot","operator":"Microsoft","path":"/e/voidle"},{"at":"2026-08-31T19:48:01.297Z","agent":"OAI-SearchBot","operator":"OpenAI","path":"/robots.txt"},{"at":"2026-08-31T19:01:54.522Z","agent":"Bytespider","operator":"ByteDance","path":"/robots.txt"},{"at":"2026-08-31T19:01:23.194Z","agent":"Applebot","operator":"Apple","path":"/robots.txt"},{"at":"2026-08-31T18:31:50.506Z","agent":"Bytespider","operator":"ByteDance","path":"/robots.txt"},{"at":"2026-08-31T16:51:11.662Z","agent":"ChatGPT-User","operator":"OpenAI","path":"/"},{"at":"2026-08-31T16:51:09.893Z","agent":"ChatGPT-User","operator":"OpenAI","path":"/"},{"at":"2026-08-31T16:40:59.296Z","agent":"Applebot","operator":"Apple","path":"/robots.txt"},{"at":"2026-08-31T15:37:08.114Z","agent":"Googlebot","operator":"Google","path":"/robots.txt"},{"at":"2026-08-31T15:29:38.234Z","agent":"DuckDuckBot","operator":"DuckDuckGo","path":"/"},{"at":"2026-08-31T13:39:37.103Z","agent":"PetalBot","operator":"Huawei","path":"/e/up-peak"},{"at":"2026-08-31T12:44:43.790Z","agent":"PetalBot","operator":"Huawei","path":"/e/insula"},{"at":"2026-08-31T11:46:26.873Z","agent":"DuckDuckBot","operator":"DuckDuckGo","path":"/"},{"at":"2026-08-31T11:25:23.260Z","agent":"Amazonbot","operator":"Amazon","path":"/experiments/i-built-a-game-from-my-phone"},{"at":"2026-08-31T11:00:05.615Z","agent":"DuckDuckBot","operator":"DuckDuckGo","path":"/"},{"at":"2026-08-31T10:45:54.363Z","agent":"Bytespider","operator":"ByteDance","path":"/e/signal-weather"},{"at":"2026-08-31T09:57:35.231Z","agent":"DuckDuckBot","operator":"DuckDuckGo","path":"/"},{"at":"2026-08-31T09:43:14.988Z","agent":"ChatGPT-User","operator":"OpenAI","path":"/"},{"at":"2026-08-31T09:21:50.385Z","agent":"bingbot","operator":"Microsoft","path":"/e/sieve"},{"at":"2026-08-31T09:02:48.531Z","agent":"Googlebot","operator":"Google","path":"/robots.txt"},{"at":"2026-08-31T08:34:23.673Z","agent":"bingbot","operator":"Microsoft","path":"/e/signal-weather"},{"at":"2026-08-31T08:31:11.837Z","agent":"Bytespider","operator":"ByteDance","path":"/e/micro-break"},{"at":"2026-08-31T07:28:55.195Z","agent":"PetalBot","operator":"Huawei","path":"/e/exactly"},{"at":"2026-08-31T07:10:37.407Z","agent":"Bytespider","operator":"ByteDance","path":"/e/ai-deas"},{"at":"2026-08-31T06:54:56.602Z","agent":"bingbot","operator":"Microsoft","path":"/robots.txt"},{"at":"2026-08-31T06:52:02.932Z","agent":"bingbot","operator":"Microsoft","path":"/experiments/sieve"}]}}