How AI Engines Parse Video: Transcripts, Chapters, and the T

Last updated: 2026-10-13

TL;DR

Transcripts are near-human. OpenAI’s Whisper team reports a speech-recognition model approaching human accuracy and robustness after training on 680,000 hours of audio paired with transcripts [1].

The engines mostly read text, not frames. Of the major vendors, only Google’s Gemini API documents native YouTube URL input with timestamp Q&A [3]. Anthropic’s Vision docs cover image inputs; no video format is named [4].

Research direction. Video-RAG (NeurIPS 2025) builds answers from a video’s audio, on-screen text, and detected objects, routed to the model as text alongside the frames [2].

Chapters become retrievable segments. YouTube requires the first timestamp at 00:00, at least three in ascending order, each at least ten seconds long [6]. In OtterlyAI’s data, 78% of timestamped videos were cited more than once, often across two to five chapters, on Google’s AI surfaces [9].

Limit. Nothing here guarantees a citation; schema alone did not shift AI citations in Ahrefs’s 1,885-page test [15], and answer lists shift run to run.

OpenAI trained Whisper on 680,000 hours of multilingual audio paired with transcripts and reports results approaching human-level speech recognition and robustness [1]. When you upload a long-form YouTube interview, something like that model reads what you said and attaches a text track to your video. That text track, plus the chapters and the description, is what an AI engine retrieves. The frames are mostly along for the ride.

This spoke sits under the pillar on why video is winning AI search. The pillar makes the case for video as a format. This one answers the mechanics question underneath it: what text actually reaches the model from a YouTube video? If you are a founder weighing the recording hours, the question matters, because the parts you shape are the parts the engines quote.

What text actually reaches a model when you give it a YouTube video?

A long-form video is not one artifact to an engine. It is five.

Surface	What is in it	Who writes it	Who tends to read it
The transcript	Spoken words, auto-captioned	ASR (Whisper or similar)	Any engine that retrieves video text
Chapters	Timestamp plus label	You, in the description	Google AI Overviews, Google AI Mode [9]
The description	Summary, named entities, links	You	Peec AI documented AI Mode answering from a description directly [11]
The video page URL and title	Your words at the top	You	All search surfaces
The watch page you host	Chapters plus full transcript as text	You	Any crawler that reaches your domain

I call the retrievable set a Citable Surface, a term I use across the Pillar 2 series. The reason the term matters: popularity is not the lever, retrievable text is. In OtterlyAI’s analysis of more than 100 million AI citations, views, likes, and subscribers each correlated with citation at about zero [9]. Description length led the metadata correlates at r = 0.31 [9]. That is the ratio to carry around. A long description beat every other knob the study measured.

Can an AI engine watch a video, or just read about it?

The honest answer: it depends on the vendor, and most do not watch.

Google. The Gemini API accepts public YouTube URLs, processes the audio and visual streams, and takes questions about a moment by MM:SS timestamp [3]. This is native video input.
Anthropic. The Vision guide covers image inputs. No video format is listed [4]. In practice, Claude reaches a video through what a web page or a search tool can quote back as text.
OpenAI. ChatGPT search returns answers with links to web sources [17]. OpenAI does not document native YouTube video reading at retrieval time.
Perplexity. Each answer carries numbered citations; the help page describes a system that fetches web results and cites them [16].

My reading, and it is a reading: outside Google’s Gemini, a video enters an AI answer because text about the video entered a retrieval pipeline. The video did not play. OtterlyAI’s data lines up with that: of all YouTube citations in the study, 38.7% came from Perplexity and 36.6% from Google AI Overviews, while ChatGPT supplied 4.4% [9]. The engines that read the most text about a video cite it the most. This is also why the gap I covered in indexed vs cited by AI opens up: being indexed is a web-search property; being cited is a retrieval-plus-generation property that your video’s text has to earn separately.

How accurate is the transcript the engine reads?

Whisper reports results approaching human accuracy and robustness after large-scale weak supervision training on 680,000 hours of multilingual audio with transcripts [1]. That is strong across accents, dialects, and background noise.

YouTube’s own help page is more cautious: automatic captions can misrepresent speech because of mispronunciations, accents, dialects, or background noise, and the service tells creators to review and edit them [5].

The weak spot for a founder is specialized vocabulary. Model names, product names, acronyms, and non-English surnames are where I see the most slips in my own transcripts. The fix is a human pass, then publishing the corrected text on a page you own. The corrected transcript becomes a crawlable artifact separate from the YouTube caption file, which is what Peec AI documented AI Mode drawing on when it built an answer directly from a video’s attached text [11]. The deeper point for a founder, which I cover in what AI engines hear when a founder speaks, is that the text that reaches the model is a sentence-by-sentence copy of what you said. The specifics you use in the room are the specifics an engine can quote.

How do YouTube chapters become retrievable segments?

Chapters are not a cosmetic nicety. They convert one 50-minute video into a sequence of named, timestamped segments an engine can quote at the chapter grain.

YouTube’s rules are specific: the first timestamp must be 00:00, there must be at least three in ascending order, and each chapter must run at least ten seconds [6]. Google says it prioritizes the key moments you set in a YouTube description and documents a Clip markup for marking those moments on a watch page you host [7].

The retrieval mechanics follow a pattern Stockebrand describes for text in his O’Reilly book on AEO. Google’s Vertex AI RAG defaults chunk long documents at about 1,024 tokens with 256 tokens of overlap (Chapter 2, semantic chunking), and coherent paragraphs that stand alone perform better than long sections that only make sense in context [10]. A chapter title in a video description does the same job for a video: it labels the chunk so the chunk can carry its own meaning when a retriever lifts it out.

OtterlyAI’s data showed only 31% of cited videos had timestamp signals at all, but 78% of timestamped videos were cited more than once, often across two to five chapters, and the chapter-level citations appeared on Google’s AI Overviews and AI Mode specifically [9]. Not everyone uses chapters. The ones who do get repeat-cited on Google’s surfaces.

I title each chapter as a question a viewer would type, like “How did you find your first customer?” rather than “First customer.” The engines score chunks as text, and a question-shaped chunk reads like an answer target. That is a choice, not a controlled finding.

What does the description earn that the title does not?

Metadata is where the correlations get interesting. In the OtterlyAI data, title length correlated with citation at r = 0.02; description length correlated at r = 0.31; the average cited description ran 334 words [9]. Fifteen times the correlation for the long paragraph nobody reads.

A good description holds: a summary paragraph, the named people and products, the chapter list with timestamps, and the links. Each is a retrievable string. Peec AI documented AI answers drawing on both the description and the auto-generated transcript, including a case of Google AI Mode building a response straight from a description [11]. The description is the chunk an engine can lift without any audio processing at all. Treat it as the first page of your paper, not the back cover.

What is Video-RAG, and does it change what you ship?

Video-RAG, a NeurIPS 2025 paper, points at where multimodal retrieval is going. The system improves long-video question-answering by extracting text from a video’s audio, on-screen text (via OCR), and detected objects, and giving all three to the model alongside the frames [2]. The research route is to turn pixels into retrievable text and let a language model reason over the combined stream.

Stockebrand’s O’Reilly chapters on retrieval and the near future match this direction. Chapter 6 flags YouTube leading Google AI Overviews citations at roughly 23% of a 36-million-AIO sample from Surfer SEO [10]. Chapter 8 notes that multimodal retrieval is coming online faster than many expect, with images, video, and audio becoming first-class retrievable content types [10]. Chapter 8 also summarizes Anthropic’s Contextual Retrieval work, which enriches each chunk with document-level context during indexing and reports up to a 69% retrieval-error reduction when combined with hybrid retrieval [10]. Different system, same arrow: the context around a chunk is what makes the chunk citable.

My read: the engines you shop against today still mostly read the text track. Video-RAG and its cousins will raise what counts as retrievable to include visual and object cues. The ship-today move does not change: give the engines a complete, correct text track. A system that someday also reads your slides and your face will still read your transcript first.

The Citable Surface stack: what each piece earns

Pulling this together, here is what each surface earns in the stack we build around one episode.

Surface	What the engine reads	Primary engines that read it
Transcript on YouTube	ASR-generated spoken words	Perplexity, Google AI Mode (quote level) [9]
Transcript on your watch page	Human-corrected text, indexed by any crawler	ChatGPT via Bing, Perplexity, Google AI Mode [9][10]
Chapters	Chunk labels with timestamps	Google AI Overviews, Google AI Mode (chapter level) [9]
Description	300-plus-word summary, entities, links	Google AI Mode direct answers [11]; strongest metadata correlate at r = 0.31 [9]
Watch page	VideoObject plus Clip markup, chapter list, full transcript	Google surfaces [7][8]
Written companion post	A quote-led article that cites the video	ChatGPT lanes where video alone is thin [9][14]

A concrete grounding. In our October 2026 Hector Hernandez episode, the salesperson-and-AI-assistant story runs about 90 seconds inside a 53-minute conversation and sits in a chapter titled with the question a viewer would type [13]. The watch page carries the full corrected transcript as text and a VideoObject block [13]. In our August 2026 audit, Perplexity and Google AI Mode leaned on the video and the watch page, each pulling from the transcript or the chapters. When ChatGPT did not cite a YouTube video, it reached guests through third-party authoritative posts (founder profiles, twosetai.com pages) [14]. The written companion tends to be the ChatGPT lane; the video and the watch page tend to be the Perplexity and Google lanes. Both exist for the program episode.

What should a founder do this week?

Three things, in order, before the next recording.

Open your last long-form video. Confirm the description runs about 300 words and names the people, products, and links. If title and description are blank, start there [9][11].
Add at least three chapters starting at 00:00, each labeled as a question a viewer would type [6]. Repeat citation on Google’s AI surfaces lives in that chapter grain [9].
Publish a human-corrected transcript on a watch page on your own site, with VideoObject markup. The page is cheap, and it is the artifact engines can index outside YouTube [7][8].

A fourth, if you can budget the time: write one quote-led companion article that cites the video and lives on your domain. ChatGPT in our data read the companion before it read the video [9][14]. The video does the retrieval work on Perplexity and Google’s AI surfaces; the companion opens the ChatGPT lane. See how AI cites YouTube videos for the engine-by-engine breakdown.

There is no silver bullet. Schema alone did not shift AI citations in Ahrefs’s 1,885-page study [15], and Google says AI Overviews needs only an indexed, snippet-eligible page with no special optimizations [12]. What every one of these surfaces does is clear: it turns a conversation you had once into text an engine can quote whenever a buyer asks.

Next in the pillar: how AI cites YouTube videos and one episode to thirty content assets.

Who is TwoSetAI?

TwoSetAI is one of the top AI podcast and video studios in Silicon Valley, and the team behind this article, hosted by Angelina Yang. The show has featured 100+ AI founders and leaders on how real AI products get built, sold, and trusted. Alongside the podcast, TwoSetAI runs the Founder Visibility Program that turns each conversation into the articles, clips, and pages that make its guests easy to find. Watch full episodes on YouTube at youtube.com/@TwoSetAI, and find every guest and episode at twosetai.com.

Glossary

Citable Surface (also called: retrievable video text, video citation surface). Any text attached to a video that an AI engine can retrieve and quote: the transcript, chapter titles, description, and watch page.
Whisper (also called: OpenAI ASR, automatic speech recognition). OpenAI’s open-source speech-recognition model, trained on 680,000 hours of audio paired with transcripts [1].
Chapters (also called: timestamps, key moments, video chapters). Timestamped sections in a YouTube description that split a video into named parts. First timestamp must be 00:00; minimum three in ascending order; each at least ten seconds [6].
Semantic chunk (also called: retrieval chunk, RAG chunk). A coherent passage sized for a retrieval system, roughly 1,024 tokens with 256 tokens of overlap in a common default [10].
Video-RAG (also called: multimodal RAG, visually aligned retrieval). A retrieval approach that extracts text from a video’s audio, on-screen text, and detected objects, then feeds the text to the LLM alongside the frames [2].
Native video input (also called: direct video understanding). An API that accepts a video URL and answers questions about the content without a text-only transcript step, as in Google’s Gemini API [3].

About Angelina Yang

Angelina Yang is VP of AI, a fast.ai fellow, and the host of the TwoSetAI podcast, where she has interviewed 100+ AI founders and leaders about how real AI products get built, sold, and trusted. Find her on LinkedIn, talk to her AI clone on Delphi, watch the podcast on YouTube, or browse twosetai.com.

Frequently Asked Questions

How do AI engines read a YouTube video? Mostly as text. Of the major vendors, only Google’s Gemini API documents native YouTube URL input with timestamp Q&A [3]. Elsewhere, an engine reaches a video through the auto-generated transcript, chapter titles, description, and any watch page that hosts the same material as crawlable text [4][5][11].

Does ChatGPT watch YouTube videos? Not natively, based on public documentation. OpenAI describes ChatGPT search as answers with links to web sources [17]. In OtterlyAI’s 100-million-plus citation study, ChatGPT supplied 4.4% of all YouTube citations, far below Perplexity’s 38.7% and Google AI Overviews’ 36.6% [9].

What does the Whisper speech-recognition model do for AI search? Whisper and similar models produce the transcript the engines retrieve. OpenAI reports Whisper approaching human-level speech recognition after training on 680,000 hours of multilingual audio paired with transcripts [1]. YouTube’s auto-captions use similar technology and should be reviewed by the creator for product and vocabulary errors [5].

Are YouTube chapters a ranking signal for AI citations? They are a retrieval signal. YouTube requires the first timestamp at 00:00, at least three in ascending order, each at least ten seconds [6]. In OtterlyAI’s data, 78% of timestamped videos were cited more than once, often across two to five chapters, on Google AI Overviews and AI Mode specifically [9]. Google says it prioritizes the key moments you set in a description [7].

Does the YouTube description matter for AI search? A lot. Description length led the metadata correlates in OtterlyAI’s data at r = 0.31, versus r = 0.02 for title length, with the average cited description running 334 words [9]. Peec AI documented Google AI Mode building an answer straight from a description [11].

What is Video-RAG, and should I optimize for it? Video-RAG, a NeurIPS 2025 paper, routes a video’s audio, on-screen text, and detected objects into text an LLM reads alongside the frames [2]. The research direction raises what counts as retrievable in the next generation of engines. The ship-today move does not change: a clean transcript, chapters, and a long description still carry the load.

Does my watch page need VideoObject schema? Google documents the Video and Clip structured-data types for marking a video on a page [7] and recommends a dedicated page for each video where the video is the main content [8]. Treat the markup as hygiene, not a lever: Ahrefs tracked 1,885 pages that added JSON-LD and found no major citation uplift on any AI platform [15]. For the deeper mechanism of what schema likely does and does not do in the retrieval pipeline, including controlled schema-only extraction experiments across ChatGPT, Claude, Gemini, Perplexity, and Google AI Mode, see How Schema Markup Might Actually Work in AI Search [18]. I add it because it is cheap.

Why does Claude not cite YouTube videos the same way Google does? Anthropic’s API Vision guide documents image inputs and does not list a video format [4]. Where the engines read mostly text, a video reaches Claude through a web page that carries the transcript or a quote from it, so the text layer on your own domain is doing the work there.

Can AI engines read my podcast if it is audio only? Yes, if the transcript is published as text on a page a crawler can reach. Whisper-class ASR turns the audio into a transcript [1]. The engines then treat the transcript as any other retrievable text. The audio file itself is rarely what is cited.

References

[1] Radford, A. et al. (2022). “Robust Speech Recognition via Large-Scale Weak Supervision.” OpenAI; arXiv:2212.04356, December 6, 2022. https://arxiv.org/abs/2212.04356

[2] Luo, Y. et al. (2025). “Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension.” NeurIPS 2025; arXiv:2411.13093. https://arxiv.org/abs/2411.13093

[3] Google (2026). “Video understanding.” Gemini API documentation, Google AI for Developers, last updated October 9, 2026. https://ai.google.dev/gemini-api/docs/video-understanding

[4] Anthropic (2026). “Vision.” Claude API documentation, accessed October 13, 2026. https://platform.claude.com/docs/en/build-with-claude/vision

[5] YouTube (2026). “Use automatic captioning.” YouTube Help, accessed October 13, 2026. https://support.google.com/youtube/answer/6373554

[6] YouTube (2026). “Video Chapters.” YouTube Help, accessed October 13, 2026. https://support.google.com/youtube/answer/9884579

[7] Google (2026). “Video (VideoObject, Clip, BroadcastEvent) structured data.” Google Search Central, last updated September 24, 2026. https://developers.google.com/search/docs/appearance/structured-data/video

[8] Google (2025). “Video SEO best practices.” Google Search Central, last updated December 18, 2025. https://developers.google.com/search/docs/appearance/video

[9] Tousseyn, R. (2026). “The YouTube Citation Study 2026.” OtterlyAI, updated March 2, 2026. https://otterly.ai/blog/youtube-ai-citation-study-2026/

[10] Stockebrand, R. (2026). Answer Engine Optimization. O’Reilly Media, July 2026 (Ch 2, semantic chunking defaults; Ch 6, YouTube at roughly 23% of Surfer SEO’s 36-million-AIO sample; Ch 8, multimodal retrieval and Anthropic’s Contextual Retrieval). https://learning.oreilly.com/library/view/answer-engine-optimization/0642572275549/

[11] Rudzki, T. (2026). “YouTube in AI search: #2 most-cited domain, transcripts included.” Peec AI documentation, March 2026. https://docs.peec.ai/research/youtube-in-ai-search

[12] Google (2025). “AI features and your website.” Google Search Central, last updated December 10, 2025. https://developers.google.com/search/docs/appearance/ai-features

[13] TwoSetAI (2026). Hector Hernandez (Co-founder and COO, Speakeasy) episode, 52:59, published October 5, 2026; watch page https://www.twosetai.com/watch/hector-hernandez/. https://www.youtube.com/watch?v=yhOLsoqSSEY

[14] TwoSetAI (2026). Guest search audit, checked August 10 to 25, 2026, incognito, screenshots on file. Internal data; see Methodology note.

[15] Linehan, L. (2026). “We Tracked 1,885 Pages Adding Schema. AI Citations Barely Moved.” Ahrefs, May 11, 2026. https://ahrefs.com/blog/schema-ai-citations/

[16] Perplexity (2026). “How does Perplexity work?” Perplexity Help Center, accessed October 13, 2026. https://www.perplexity.ai/help-center/en/articles/10352895-how-does-perplexity-work

[17] OpenAI (2024). “Introducing ChatGPT search.” October 31, 2024. https://openai.com/index/introducing-chatgpt-search/

[18] Yang, A. (2025). “How Schema Markup Might Actually Work in AI Search.” Medium, December 17, 2025 (republished at twosetai.com/blogs/how-schema-markup-might-actually-work-in-ai-search). https://angelina-yang.medium.com/how-schema-markup-might-actually-work-in-ai-search-d017b1d2ef83

Methodology note. Last verified 2026-10-13. Engine behavior is drawn from each vendor’s own documentation, dated above. The YouTube chapter requirements, transcript cautions, and schema guidance come from YouTube Help and Google Search Central at the dates cited. Whisper and Video-RAG results are self-reported by the authors; treat them as directional for any specific production system. OtterlyAI’s YouTube Citation Study measures videos already cited, so its correlations describe repeat citation, not what makes a video eligible [9]. The TwoSetAI audit [14] covered past guests across August 10 to 25, 2026, in incognito windows, with about seven queries per guest; the recent-cohort figures used here are from that audit, and the recent-versus-older comparison is directional, not controlled. “Citable Surface” and the Pillar coined terms (Citation Gap, Buyer Bucket, Founder-Voice Moat, Editorial Layer, 5-Category Architecture) are TwoSetAI’s own terms, not industry standards. Nothing here guarantees a citation.