Video for AI · Apify actor
Let your AI model “watch” a video file
GPT, Claude, Gemini and most AI agents cannot watch a video: they read images and text. Give this tool a video link and get back a scene list, one keyframe image per scene, the speech transcript lined up with each scene, one grid image of all keyframes and a ready-to-paste llmContext text block. No LLM inside and no API key to bring: you keep your own model and prompt.
What it does
Video to LLM Context, in short
- Finds the scene cuts and keeps at most N scenes (default 12): the strongest cuts win, long shots are split at pauses in the speech
- One keyframe image per scene with a public link, stepping past black frames
- Transcribes the speech with OpenAI Whisper (99 languages, auto-detect) and assigns every word to the scene it was spoken in
- One labelled grid image of all keyframes (free): send one picture to a vision model instead of 12
llmContext: plain text with one block per scene (time range, image link, speech) to paste straight into a prompt- Direct video file links only; video platform pages are refused on purpose
Who it's for
Built for
Example
Input and output
One dataset row per result, downloadable as JSON, CSV or Excel, or read through the Apify API.
{
"videoUrls": ["https://your-bucket.s3.amazonaws.com/webinars/demo.mp4"],
"maxKeyframes": 12,
"language": "auto"
}From a real cloud run on the actor's 21-second public-domain sample clip, trimmed to one of its 3 scenes.
{
"fileName": "sample-vertical-talk.mp4",
"durationSeconds": 21.55,
"width": 720,
"height": 1280,
"hasAudio": true,
"status": "ok",
"sceneCount": 3,
"keyframeCount": 3,
"gridImageUrl": "https://api.apify.com/v2/key-value-stores/.../records/...keyframe-grid.jpg",
"transcriptLanguage": "en",
"billedKeyframes": 3,
"billedAudioMinutes": 1,
"scenes": [
{ "index": 2, "start": 2.936, "end": 13.246, "boundary": "cut",
"keyframeUrl": "https://api.apify.com/v2/key-value-stores/.../records/...scene002-000008090ms.jpg",
"transcript": "Inside that disc is essential clearing ..." }
],
"llmContext": "VIDEO: sample-vertical-talk.mp4\nduration 0:21 | 720x1280 ...",
"error": null
}Pricing
Pay per result. Failures are free.
No subscription, no monthly minimum. Apify platform compute is included in the price; Apify also adds a tiny start fee per run ($0.00005 per GB of run memory).
| You run | You pay |
|---|---|
| A 30-second ad, 8 scenes | $0.024 |
| A 10-minute webinar, 12 scenes | $0.104 |
| A 5-minute silent screen recording, 12 keyframes | $0.024 |
2 failed downloads = $0.00
Not charged:
- The grid image, metadata, scene list and
llmContexttext - Refused links, failed downloads, unreadable files and over-limit videos
- Audio minutes for videos with no audio track or no speech
Audio is charged per started minute, and only when speech was found and transcribed. Blank keyframes are skipped and not charged. You can set Maximum cost per run in the run options and the actor stops cleanly when it is reached.
FAQ
Questions
Does it call ChatGPT, Claude or another LLM?
No. It prepares the video for your model. Nothing is sent to a third-party AI service, and you choose the model and the prompt.
Which videos work?
Any file ffmpeg can decode (MP4, MOV, WEBM, MKV, AVI, MPEG-TS and more), up to 300 minutes and 4 GB. The link must point to the file itself; Dropbox and Google Drive share links are converted to direct downloads automatically.
Can I use YouTube or TikTok links?
No. Those platforms forbid downloading in their terms, so the actor refuses them (free row).
How accurate is the transcript?
It uses Whisper base: good on clear speech, weaker on names, heavy accents and noisy audio. Set language when you know it; that helps short clips most.
How many keyframes should I ask for?
8-16 works for most prompts. The grid image lets you send them all as one picture to save vision tokens.
Try Video to LLM Context on Apify
Apify's free plan includes monthly platform credit you can use to try it. Also callable from AI assistants through the Kanto Labs MCP server.
Other Kanto Labs tools
Video Frame Extractor
Scene-change keyframes, thumbnails or one frame every N seconds, plus a contact sheet.
Video to Slides
Every unique slide from a lecture or webinar video, as images, text and one PDF.
Video OCR
Burned-in subtitles, captions, hook text and lower-thirds as timed text, a transcript and SRT/VTT.