How I Built an AI Demo Video Tool on Cloudflare (2026)

Nook takes a screen recording you already have and turns it into a narrated demo video plus a written step-by-step guide. I built it alone over the last few weeks, almost entirely on Cloudflare: one Worker, a SQLite database, object storage, a durable workflow engine, and a container that runs ffmpeg and Remotion. This post walks through how the pieces fit, with real code from the repo, and the parts that took longer than I expected.

#Short version

• One Worker serves the API (Hono) and the React app (Vite build, served as static assets). D1 holds the data, R2 holds every video, still and audio file.
• One Cloudflare Workflow (VideoPipeline) runs each video through transcode, analysis, transcription, step detection, screenshots, privacy scan, script, voice and render. Every stage is a durable step with its own retries.
• Cloudflare Containers run a small Node server with ffmpeg, Tesseract and Remotion 4. The Worker submits jobs over HTTP and the workflow polls them. The container never holds storage credentials; it reads and writes R2 through an internal route on the Worker.
• Open models do the AI work. A text LLM and a vision model run on DeepInfra through its OpenAI-compatible API. Speech-to-text and English voice run on Workers AI. Non-English voices use Kokoro on DeepInfra.
• The same Remotion composition renders the browser preview (Remotion Player) and the final MP4 in the container, so what you see is what you get.
• Billing is Dodo Payments as merchant of record, wired up with webhooks into D1.
• The hardest parts were not the infrastructure. They were timing: keeping voice, captions and picture in sync after cutting dead time out of a recording.

#What Nook does, briefly

You upload an MP4, MOV or WebM (up to 15 minutes). Nook splits it into steps, from your narration if you talked or from what changes on screen if you did not. It writes one script line per step, voices it (or keeps your own voice with the dead time cut), adds captions, blurs things like emails and API keys, and renders a video with your brand. It also produces a guide with a heading, a note and a screenshot per step. If you want the product side, the launch post covers it. This post is about how it works underneath.

#The architecture

Here is the whole system in one table.

And the flow of one video:

browser ──multipart upload──▶ Worker ──▶ R2 (raw file)
                               │
                               └─▶ PIPELINE.create({ projectId, mode: "full" })
                                        │
     ┌──────────────────────────────────┘
     ▼
 VideoPipeline (Workflow)
   transcode ─▶ analyse ─▶ transcribe ─▶ see actions ─▶ build steps
   ─▶ frames ─▶ privacy scan ─▶ write script ─▶ voice ─▶ render ─▶ finalize
     │  container jobs: POST /jobs, poll GET /jobs/:id
     ▼
 RenderContainer (ffmpeg, tesseract, remotion)
     │  reads and writes media via Worker /internal/r2/*
     ▼
   R2 (source.mp4, shots/*.jpg, final-*.mp4)

The Worker config is short. The interesting bindings are these, from wrangler.jsonc (ids and keys left out):

"d1_databases": [{ "binding": "DB", "database_name": "demoreel-db" }],
"r2_buckets": [{ "binding": "MEDIA", "bucket_name": "demoreel-media" }],
"ai": { "binding": "AI" },
"workflows": [
  { "name": "demoreel-pipeline", "binding": "PIPELINE", "class_name": "VideoPipeline" }
],
"containers": [
  { "class_name": "RenderContainer", "image": "../render/Dockerfile",
    "max_instances": 4, "instance_type": "standard-3" }
]

(The internal names still say "demoreel". That was the working name before Nook. Renaming bindings is not worth a migration.)

#Why Workers, D1 and R2

I wanted one deploy and one bill. A video tool has a lot of moving parts, and as a solo founder the thing I can least afford is operating five services across three providers.

Workers covers the API and the frontend in one deploy. The SPA is a Vite build in apps/web/dist, attached as static assets with run_worker_first: true, so the Worker decides per request whether it is the marketing host (usenook.live), the app host, or an API call. The marketing host serves a prerendered landing page and the blog; any other path redirects to the app.

D1 is SQLite, which is what I would pick for this anyway. The data is relational and small: projects, steps, cuts. Most writes happen inside workflow steps, and db.batch() makes it cheap to update twenty step rows at once. There are 22 migration files so far, each a plain .sql file. Nothing about the schema needed Postgres.

R2 holds everything heavy. Uploads go straight from the browser into an R2 multipart upload through the Worker in 10 MB parts, so a 600 MB recording never sits in Worker memory. There are no egress fees, which matters for a product whose output is video people stream from a share page.

#Workflows: the pipeline as durable steps

The core of Nook is VideoPipeline, a class extending WorkflowEntrypoint. Before Workflows, this kind of pipeline meant a queue, a job table, a worker process, and a lot of code for "where did this job stop and can I resume it". With Workflows, each stage is a step.do() call. The result of a finished step is stored, so if anything later fails and the instance resumes, completed steps are not run again.

A typical step looks like this, from transcription:

const segs = await step.do(
  `transcribe ${i}`,
  { retries: { limit: 3, delay: "15 seconds", backoff: "exponential" }, timeout: "5 minutes" },
  async () => {
    const obj = await env.MEDIA.get(chunk.key);
    if (!obj) throw new NonRetryableError(`audio chunk missing: ${chunk.key}`);
    const audio = b64(await obj.arrayBuffer());
    const r = await env.AI.run("@cf/openai/whisper-large-v3-turbo", {
      audio, task: "transcribe", vad_filter: true,
    });
    // ...shift segment and word times by chunk.offsetS
  },
);

Two things I like here. Retries are declared per step, so a flaky model call retries three times while a database write does not need to. And NonRetryableError lets me say "this will never work, stop now", for cases like a missing file or a recording longer than the plan allows.

On top of per-step retries there is one whole-pipeline retry. If the workflow fails and the error does not match a list of permanent errors (no video stream, no sound, waiting for payment), it schedules a fresh instance with attempt + 1. The comment in the code says why: "most failures are a container restart or a model hiccup". After one retry it gives up, marks the project failed and emails the owner.

The same workflow class handles four modes: full (new upload), rerender (after you edit the script or settings), translate, and cut (the shorter social versions). A rerender skips everything up to voicing, because steps, screenshots and script are already in D1.

#Containers: ffmpeg and Remotion next to the Worker

Workers cannot run ffmpeg or a headless Chromium, and Remotion needs both. Cloudflare Containers let me ship a normal Docker image (node:22-bookworm-slim plus ffmpeg, Tesseract, Noto fonts including CJK, and Chromium dependencies) and address it from the Worker like a Durable Object.

The container side is a tiny HTTP server with an in-memory job queue that runs one job at a time:

if (req.method === "POST" && url.pathname === "/jobs") {
  const body = JSON.parse(raw);
  const id = randomUUID();
  jobs.set(id, { status: "running", progress: 0 });
  queue.push({ id, body });
  runNext();
  return json(res, 200, { jobId: id });
}

Job types are transcode, analyse, frames, pii, render and a couple of others. Each lives in its own file under apps/render/src/jobs/.

#How the Worker talks to the container

The workflow submits a job, then sleeps and polls. Each poll is its own durable step, so a long render does not hold a single step open for twenty minutes:

for (let i = 0; i * intervalS < maxWaitSeconds; i++) {
  await step.sleep(`${tag}: wait ${i}`, `${intervalS} seconds`);
  const st = await step.do(`${tag}: poll ${i}`, { retries: { limit: 6, delay: "10 seconds" } },
    async () => {
      const res = await send(`/jobs/${jobId}`);
      if (res.status === 404) return { status: "lost" };
      // ...
    });
  if (st.status === "done") return JSON.parse(st.resultJson ?? "null");
  if (st.status === "failed") throw new NonRetryableError(`${type} failed: ${st.error}`);
  if (st.status === "lost") { lost = true; break; }
}

The lost status is the lesson from the first week. The job queue lives in container memory. When a deploy rolls out, or the box is evicted after being idle, or it runs out of memory, the job simply vanishes and the poll gets a 404. So a lost job is resubmitted, up to three times, with a short backoff. Because transcode and frame jobs write deterministic R2 keys, a resubmit just redoes the work.

Routing is a hash. Each project maps to one of three named instances (render-0 to render-2) so its jobs land on the same box, and Studio plan videos go to a separate render-priority instance so they never queue behind other renders.

#The container has no storage credentials

The container does not get R2 keys. Instead it calls back into the Worker at /internal/r2/<key> with a shared internal header, and the Worker reads or writes R2 using its binding. Large files use R2 multipart through the same route in 50 MB parts. This keeps credentials in one place and means the container image is the same in dev and production; in dev I point the Worker at a local container with an env var.

#The models: what runs where

All AI calls go through two places: the Workers AI binding and DeepInfra's OpenAI-compatible endpoint. DeepInfra was useful because it hosts many open models behind one API and one key, so switching the script model is an env var change, not a new integration.

The LLM wrapper is about 30 lines. It asks for JSON mode, parses the reply with a zod schema from the shared package, and if parsing fails it tells the model what was wrong and tries once more:

for (let attempt = 0; attempt < 2; attempt++) {
  const msgs = attempt === 0 ? messages
    : [...messages, { role: "user", content: `Your previous answer was invalid: ${lastErr}. Return corrected JSON only.` }];
  const res = await fetch(url, { method: "POST", headers, body: JSON.stringify({
    model: opts.model ?? env.LLM_MODEL, messages: msgs,
    temperature: opts.temperature ?? 0.4, response_format: { type: "json_object" },
  }) });
  const content = (await res.json()).choices?.[0]?.message?.content ?? "";
  try { return parse(JSON.parse(content)); }
  catch (e) { lastErr = String(e).slice(0, 500); }
}

Validating with zod and feeding the error back fixed most malformed output. The schemas (ScriptOutput, TranslationOutput, GuideMeta) live in packages/shared, so the Worker, the frontend and the prompts agree on shapes.

#Finding steps in a silent recording

If you narrate, Whisper gives me segments and words, and those already suggest where steps start. A silent recording is harder. Most uploads come from QuickTime or OBS, which record pixels only. There is no click log.

So the container reads the video itself. The analyse job decodes it at 640 px wide, 10 fps, greyscale, and looks at what changed between frames: blobs of changed pixels, the cursor (two small blobs where the old frame at one spot looks like the new frame at the other), scrolls (a frame that matches the previous one shifted up or down), typing (small changes along one row while the cursor stays put). Anything bigger than the cursor after the cursor stops becomes a candidate, with a still from just before and just after. The comment at the top of that file sets the policy: tuned to catch everything, because false alarms are cheap and misses are not.

The Worker then sends each before/after pair to the vision model in batches of eight, with a prompt that asks for one of click, type, select, dismiss or none, plus the control's label and box. Then it checks the model against the pixels. If the box the model names does not contain the pointer, or does not overlap the area that actually changed, the box is dropped. If the model claims a click on what was really a loading screen, the candidate is dropped. This cross-check mattered more than prompt tuning did.

It is still a guess. Fast or very visual recordings can get misjudged, which is why every step is editable before you render. Zoom is also only applied to clicks from the recorder, not to actions read from video, because the position from pixels is not precise enough to zoom on.

#Audio and caption timing

Each step has a source range in the recording and, with AI voice, a narration clip of some length. The timeline code decides how long each step is shown and how fast the footage plays to fit. It lives in packages/composition/src/timeline.js, plain JavaScript on purpose, so the container (Node) and the browser share one implementation.

The rules in short:

• With AI voice or captions, a still stretch of screen can be removed. With your own voice, only stretches that are both still and silent can go, otherwise I would cut you off mid-sentence.
• The first 1.2 seconds of each still stretch are kept, because that is usually the result of your last click and people need a moment to see it. A quarter second at the edges is kept so cuts do not feel abrupt.
• No step shrinks below 1.5 seconds.
• Footage speeds up or slows down to match the voice, within limits that depend on the pace setting (relaxed, normal, fast).
• With your own voice, footage plays at real speed and nothing is stretched.

Captions for your own voice come from Whisper's word timings. Words are grouped into lines of up to seven, breaking at sentence ends or pauses over 0.7 seconds, and each word's source time is mapped through the kept segments into output frames. If a word falls inside a cut, it is dropped from the captions too. That mapping is the piece I rewrote most often. Once you remove dead time, a source timestamp and an output frame are no longer the same thing, and every off-by-a-segment bug shows up as captions drifting ahead of the voice.

The other cost-relevant detail: TTS audio is cached by a hash of voice plus text. If you edit one line of a 20-step script and re-render, only that line is voiced again.

#Privacy blur

After screenshots are taken, the container runs Tesseract on each one in TSV mode, which gives a bounding box per word. A list of detectors runs on single words and on groups of two to four words on the same line (phone and card numbers get split by spaces). The detectors cover emails, card numbers (with a Luhn check), IBANs, US social security numbers, phone numbers, IP addresses, common API key prefixes, long mixed letter-and-digit tokens, and URLs with token= or key= style parameters.

Regexes cannot find a person's name, so the OCR text also goes to the LLM, which is asked for names and street addresses and told to skip product names and placeholder text. Matches become blur rectangles on the step. In the video they are drawn as blurred boxes over the footage; in the guide they are applied to the screenshot. You can dismiss or add blurs by hand.

The privacy step is wrapped so that if it fails, the video still renders without it. An OCR failure should not block a video, but it also means I tell people to check the blurs before sharing anything sensitive.

#One composition for preview and render

This is the decision I am happiest with. The Remotion compositions (Demo.tsx for the main video, Story.tsx for the animated story, Backdrop.tsx for backgrounds) live in packages/composition. The web app imports them into @remotion/player for the live preview in the editor. The container imports the same package and calls renderMedia() with the same props.

So when you change the accent color, switch backgrounds, or edit a script line, the preview you see in the browser is the actual video, just played in a browser instead of encoded. There is no second "preview approximation" to keep in sync. The Dockerfile copies the package into node_modules and bundles it at build time.

After render, ffmpeg adds one MP4 chapter per step (titled from the guide headings), so players that read chapters pick them up without re-encoding.

#The animated story, and a sync bug

The animated story is a short cut (up to 90 seconds) that retells the recording on your brand background. It has two looks. "Drawn" rebuilds each screen as a clean card from the real labels: fields with what was typed, dropdowns with what was picked, buttons, and an animated cursor working through them. "Screens" shows the real screenshot in a browser-style window that eases toward where the click landed.

The first version of "screens" used one still per step, taken from the middle of the step. That looked fine for short steps and wrong for long ones: the voice would describe three things while the picture showed one frozen moment. You noticed it as the narration and the picture slipping apart.

The fix was to show several moments of the recording per step. storyShot() in the shared package picks one, two or three times spread across the step (more for longer steps), with the last one just before the click. The composition crossfades between them as the narration moves on. Instead of extracting more stills, it pulls each moment straight from the source video:

const Moment = ({ shot, t, videoUrl, w, h }) => {
  const { fps } = useVideoConfig();
  if (videoUrl && t !== null)
    return <Freeze frame={Math.max(0, Math.round(t * fps))}>
      <OffthreadVideo src={videoUrl} muted style={{ width: w, height: h, objectFit: "cover" }} />
    </Freeze>;
  return shot.url ? <Img src={shot.url} style={{ width: w, height: h, objectFit: "cover" }} /> : null;
};

Freeze holds a single frame of OffthreadVideo at the chosen source time. Because both the Player and the container read frames from the same source.mp4, the preview and the render match exactly. If no video is available, it falls back to the step's still.

#Billing with Dodo

Dodo Payments is the merchant of record, so they deal with sales tax and invoicing, not me. Checkout is a POST to their API with the product, the customer email and metadata: user id, purchase kind, and project id for a single-video purchase. The webhook does the rest.

Webhooks follow the Standard Webhooks spec, so verification is an HMAC-SHA256 over the id, timestamp and body, with timestamps older than five minutes rejected. Every webhook id is stored in a webhook_events table and duplicates are ignored, because webhooks can arrive more than once. The handler also checks the product id and ignores anything that is not a Nook product. Paying for a video starts a new workflow run. Free videos are cut to the free length, but the full recording is kept, so a truncated video goes through the full pipeline again while an untruncated one only needs a rerender to drop the watermark. Nobody has to upload again.

The "upgrade within 7 days and the $9 comes off" offer is a one-use, one-cycle percentage discount code created through Dodo's API at checkout time, restricted to that product and expiring in a day. Purchases and plan changes also write a row to a credit_ledger table, which makes support questions answerable. If you want the pricing reasoning rather than the plumbing, see per-video pricing.

#How I thought about cost

I do not have usage numbers to share yet; Nook just launched. But the design choices were made with cost in mind, and some of them are visible in the code:

• Container idle time. sleepAfter is three minutes. The comment next to it is the reason: jobs in one pipeline arrive within about a minute of each other, and a ten-minute idle tail cost more than the render itself.
• Small, fixed fleet. Three hashed instances plus one priority box, capped at four. Each runs one job at a time. Rendering is slower under load, but the bill is predictable.
• Cheap work first. Pixel diffing on the container is free compared to a vision call, so it narrows a recording down to at most 60 candidate moments before any model sees a frame.
• Cache voice by content. Re-renders only pay for lines that changed.
• Downscale early. Transcode caps the source at 1080p and 30 fps once. Every later job reads that smaller file instead of the raw upload.
• No egress fees on R2, which matters most for the share pages.

#What was hard, and what I'd do differently

Timing was the hard part, not infrastructure. Workflows, D1 and Containers mostly did what the docs said. The bugs that took days were all about mapping source time to output time after cuts, and about voice, captions and picture drifting apart.

The in-memory job queue. It was the fastest thing to build and it works, but losing jobs on restart forced the lost-and-resubmit logic. Next time I would have the container write its result to R2 and notify the workflow, probably with step.waitForEvent, instead of polling every ten seconds. The polling loop also creates a lot of workflow steps for a long render.

Two ways of waiting. The privacy scan polls the container inline inside one step, while everything else polls with durable sleeps. It works because OCR is short, but it is an inconsistency I would remove.

Constants copied by hand. A few timing constants in story.mjs carry comments like "keep in step with Story.tsx". The render side needs them before Remotion runs, to compute durations. They should live in one shared module, the way timeline.js already does.

Vision is a guess. Step detection from pixels plus a vision model is good on ordinary click-through recordings and weaker on fast or very visual ones. A recorder that logs real clicks solves that, which is why there is a Chrome extension in the works. It is not in the Chrome Web Store yet.

What I would keep. Workflows for anything multi-step. Sharing the composition between preview and render. zod schemas in a shared package. Validating model output against the pixels instead of trusting it.

If you are writing a narrated demo yourself rather than building a tool for it, how to make a product demo video and turning a recording into a step-by-step guide cover the human side.

#FAQ

#Why not run everything inside the Worker?

Workers cannot run ffmpeg or a headless browser, and Remotion needs both. The Worker does orchestration, storage and model calls. The container does anything that needs a real filesystem and native binaries.

#Why Cloudflare Workflows instead of a queue?

A video goes through about ten stages, some of which wait minutes on a container. Workflows store each step's result, retry steps on their own terms, and resume where they stopped. With a queue I would have had to build that state tracking myself.

#Which AI models does Nook use?

Whisper large v3 turbo on Workers AI for transcription, Deepgram Aura on Workers AI for English voice, and on DeepInfra a Qwen3-VL vision model for reading frames, a DeepSeek model for scripts and translation, and Kokoro-82M for voices in other languages. The text and vision models are set by env vars, so they can change.

#How does the preview match the final video?

The browser preview and the container render use the same Remotion composition package with the same props. The preview plays it in Remotion Player; the container encodes it with renderMedia().

#How does the container access storage without credentials?

It calls an internal route on the Worker with a shared header. The Worker reads and writes R2 with its own binding, including multipart uploads for large files.

#What happens if a container restarts mid-render?

The poll returns 404, the workflow marks the job lost, waits 20 seconds and resubmits, up to three times. If the whole pipeline still fails for a reason that is not permanent, it runs once more from a fresh workflow instance.

#Is the privacy blur guaranteed to catch everything?

No. It uses OCR with pattern detectors plus an LLM pass for names and addresses. It catches common cases, but OCR can miss small or low-contrast text, so check the blurs before sharing anything sensitive.

If you have a screen recording sitting in a folder, you can see what this pipeline does with it. Make your first video free. The free tier gives you one video up to a minute, and you can edit every step and line before anything gets shared.