When Search Eats the Web: A Model of Corpus Erosion
A model examining the impact of generative search engines on web content sustainability.
Generative search engines (GSEs) provide direct answers from crawled web content, impacting traffic that finances content production. This paper models the crawlable corpus as a common-pool resource, showing how publisher responses lead to degradation in volume, quality, and lifetime of content. It explores the consequences of extraction and discusses survival mechanisms for publishers.