As AI bots flood their sites, publishers are no longer just trying to keep them out – they’re starting to re-engineer content for them.
In doing so, publishers are preparing for an agentic web – a future in which AI agents make decisions and perform tasks on behalf of users. To stay visible in AI search, Time, The Economist and another major news publishers are already experimenting with parallel, agent-readable versions of their sites, stricter controls on which bots can crawl them and, in some cases, entirely new web standards.
Time, for example, is converting all its webpages from HTML into markdown versions, a simplified format that AI systems and agents can process more efficiently. Delivering content to AI agents in HTML is inefficient because it bundles layout, style and navigation information designed for humans and browsers. Markdown strips that away, giving agents the content and metadata, not the surrounding page design.
Continue reading this article on digiday.com. Sign up for Digiday newsletters to get the latest on media, marketing and the future of TV.