Local film production on one Mac

Offline copy of the web monograph at https://nyimbi.com/research/ai-film-production/. Archived sources are in the skills package under film-production/references/sources/.

TikTok is full of six-part series that claim every frame was generated. The faces hold, the voices hold, the cuts land every two seconds. Then you count the shots and notice the same two rooms, the same two faces, the same three camera setups. The format is doing as much work as the model.

This report is about the part that comes after the demo: a local pipeline that takes an outline through story, character design, storyboard, shot generation, edit, sound and delivery on one Apple Silicon Mac. It runs. It does not make the model better than it is.

Contents

1. The claim
2. What the reference work does
3. The pipeline
4. The local stack
5. The story problem
6. The design problem
7. The generation problem
8. The quality ceiling
9. The arithmetic
10. Setup
11. What this does not solve
12. Method and sources
13. Appendix A: core model manifest
14. Appendix B: pilot score sheet

The claim

The short answer is that the workflow is solved and the look is not. A local machine can now do every step. It cannot yet match the best photoreal social series by default, and the gap is not evenly distributed. It is narrow in close-ups with simple blocking. It widens with crowds, hands, fast speech and sustained dialogue.

The reference series usually rely on cloud models with native dialogue and lip-sync, locked character references or LoRAs, cloned voices, a dedicated lip-sync pass, heavy post, and many takes per shot. Some of them are hybrid: stock, real plates, or a real actor for a hero shot. "Fully AI" is a claim, not a specification.

The local stack in this report is built around rapid-mlx, which reads the Hugging Face cache and serves text, image, video, TTS and STT through one OpenAI-compatible API. Unsloth Studio is the second runtime, with a UI, an audio engine for speech and music, and video models for performance, avatar and lip-sync work. ComfyUI is there for GGUF models, control nets and upscaling. No cloud key is on the critical path.

That is a real capability. It is not a promise of parity.

What the reference work does

The pattern is boring once you see it.

• Two characters, maybe three. Two rooms. A corridor.
• Close-ups and singles. Very few wides, because wides are where faces turn to mush.
• Cuts every one to three seconds, which hides the fact that nothing moves for long.
• A locked reference image, or a small LoRA, for each face.
• A cloned voice per character, reused across episodes.
• Native dialogue on the model, or a dedicated lip-sync pass afterwards.
• A grade, grain, sound design, music. The picture is not the finished thing.
• Forty takes for the one that works.

None of that is a model. It is production. The generator, the part everyone talks about, is one line in the budget.

The pipeline

Ten skills, nine stages, five approvals. The approvals are the useful part. A bad premise is cheap to fix before design and expensive to fix after a thousand generations.

The pipeline does not generate. It refuses to let you skip the cheap decisions. Each stage writes an artifact the next stage reads, and a gate decides whether the next stage may start. The state lives on disk, not in a chat window.

Every project is one directory with 01_story, 02_design, 03_plan, 04_gen, 05_post and 06_marketing. filmctl.py scaffolds it, validates the artifacts each stage owes, compiles a canonical shot record into each model's prompt dialect, and assembles approved takes into a rough cut. It never generates and never spends money.

Key art is a parallel track, not a stage. Concept starts at design lock, when the style and hero frames exist. Final type, credits, rating and release happen after picture lock. film-poster owns the poster and thumbnail package. film-postproduction checks the files at delivery.

The local stack

The default runtime is rapid-mlx. It reads the Hugging Face cache directly and serves text, image, video, TTS and STT through one OpenAI-compatible API. It wraps the same MLX kernels that mlx-lm, mlx-video, mlx-audio and mflux use, with batching, model profiles and speculative decode.

Unsloth Studio is the second runtime. unsloth studio runs a UI and API on port 8888, and the bundled audio.cpp engine on 8080 handles TTS, voice cloning, ASR, alignment, music, MiniMax-H3 video, speech-to-video, avatars and lip-sync. It also reads LLM and GGUF models from the HF cache and trains LoRAs. Some of its media packages install into their own GGUF store, which is the one documented exception to the single-cache rule.

ComfyUI covers the node graphs: GGUF video models, control nets, upscalers and interpolation. Its model directories should be symlinks into the HF cache, not copies.

The core set is about 91 GB:

Four-bit and q4 are the fast ones. q8 and bf16 are for hero shots with nothing else loaded. Every weight lives in ~/.cache/huggingface/hub. The pull script skips what is already there, resumes partial downloads, and refuses to start if the disk cannot hold the set plus a reserve.

The quality tier adds qwen-image for legible text in key art, wan2.2-i2v-a14b-q8 for photoreal image-to-video, ltx-2.5-mlx-q8 for the best local LTX output, qwen-image-edit for edits and hidream-o1-dev for a different look. At most two of those fit on a 150 GB disk alongside the core set and the film's outputs. The tools tier adds MuseTalk and LatentSync for lip-sync, ACE-Step for music and MusicGen for beds.

omlx may already own port 8000. Run rapid-mlx on 8001 and up. One model per process. Do not co-load the LLM, the image model and a 14B video model; 96 GB is enough for one heavy tenant and a small audio process, not a studio's worth at once.

The story problem

The story skill runs a rubric with eight dimensions: premise, causal structure, character, agency, scene craft, dialogue, theme and emotional arc, originality and feasibility. It requires scene and line evidence for anything below a three. Lock needs an average of at least three and no dimension below two.

Then it asks three readers and a hostile producer to attack the draft in separate contexts. A structural editor looks for causality, turns, the midpoint and pacing. A character auditor looks for want, need, wound, agency, contradiction and voice. An audience proxy marks where attention drops and what confuses. A hostile producer asks why anyone should fund it.

A model asked whether its own draft is good will say yes. A model asked to find the scene that could be cut will find it.

The draft also runs through storycheck.py, which reports runtime, scene lengths, dialogue share, voice similarity, an agency proxy, unfilmable interior states, on-the-nose dialogue, passive constructions, repeated phrases, card completeness and setup/payoff matches. It is a defect finder, not a verdict. A clean run means those mechanical defects were not found.

Character quality is not a design problem. A character can have a beautiful sheet and no agency. The tests are simple. Does the protagonist's choice cause the major turns? Can a reader identify each principal from dialogue alone? Does the arc show up in behavior rather than in a speech? Does the antagonist want something a reasonable person could want?

The last gate is a person. The model drafts and diagnoses. It does not decide whether the film is worth making.

The design problem

Each principal becomes a contract: a 60 to 120 word identity block, a five-view sheet, expression and wardrobe variants, a hero frame, a voice, and a drift audit. Each recurring location gets an establishing frame, a reverse and a floor plan.

The floor plan matters more than it sounds. A video model does not know where the last shot put the camera. It will flip the room, change the light direction and put the window on the wrong wall. The plan and the fixed reference order are how you tell it.

The continuity system records the canonical blocks, the order the references are passed, the seeds, the frozen model versions, and the drift audits. A character in more than twenty shots gets a LoRA or a dedicated reference set. Training is cheaper than re-rolling when a lead appears in dozens of shots.

Design is where a film becomes cheap or expensive. A look without a lock turns generation into a slot machine. A locked one turns it into execution.

The generation problem

The keyframe decides the shot. The video model re-animates it. If the keyframe is wrong, the video take is not a performance problem. It is a photograph problem. Fix the photograph.

Prompts are compiled from one canonical shot record into each model's dialect. The dialects are not interchangeable. Kling image-to-video wants fifteen to forty words of motion. Seedance drops beats past roughly 250 words. Runway has no negative field, so constraints must be restated positively. Veo orders style before audio. Wan binds positional reference tokens. A prompt that works on one model is under-specified or noisy on another.

Every take is logged with provider, model version, seed, references, duration, resolution, cost and verdict. Continuity and artifact QA run before approval. Two identity failures change the reference or the keyframe. A third failure redesigns the shot. A shot that cannot be generated is usually a writing problem, not a rendering problem.

The local video models are real but tiered. LTX-2.3 q4 is the fast default and carries audio. Wan 2.2 TI2V-5B q8 is the practical quality step. Wan 2.2 I2V-A14B q8 is the best local option for photoreal image-to-video, and it is about 40 GB and slow. LTX-2.5 q8 is better again, at 63 GB. MiniMax H3 is reference-driven and slow. HunyuanVideo 1.5 is a quality alternative that is too slow for coverage on this machine.

Dialogue is the weakest link. Native dialogue models are better than a generated face with a separate lip-sync pass. MuseTalk MLX is light and realtime. LatentSync-MLX is slower and cleaner. Both fail on profiles, teeth, occlusion and fast speech. The pipeline includes the step; the local tools still need a pilot before anyone promises a talking performance.

The quality ceiling

The direct answer to the TikTok question is that the pipeline fully enables the workflow, not equal quality. The ceiling is set by the model tier, the hardware, the iteration budget and the craft.

Where the local route is behind:

1. Photoreal human motion. Wan 2.2 I2V-A14B q8 is the local best, and it is still below the cloud leaders.
2. Lip-sync on generated faces. The step exists. Nothing is installed or tested yet.
3. Voice performance. Consistency is solvable with voice IDs and reference audio. Emotional range and naturalness are not yet at the best commercial level.
4. Resolution and finish. Local output is typically 512 to 1080p. Upscaling adds sharpness, not detail.
5. Iteration. Reference series are curated from many takes. A local sixty-second pilot with twenty shots and three to five variants each is already hours. A ten-minute episode is a multi-day campaign.

Three routes:

The pilot is sixty to ninety seconds, two principals, one location, fifteen to twenty shots, four to six dialogue lines. Hold the script, keyframes, references, voices, aspect and edit constant. Run LTX-2.3 q4, Wan 2.2 I2V-A14B q8 and, only if permitted, one cloud model. Score blind on identity, lip-sync, realism, voice consistency, emotional read and artifacts, against one reference clip.

Pass for a parity claim: identity at least 4, lip-sync at least 3.5, realism at least 3.5, voice consistency at least 4, emotional read at least 3, no hard artifact. Below that, do not promise parity. Choose the hybrid, restrict the format, or accept the lower tier.

The arithmetic

At roughly four seconds per generated shot:

Three variants per shot triples the generation time. Retries add another twenty to fifty percent. A feature is not a long render. It is a campaign, and most of it should be animatic, stock, 2D inserts or selective generation.

The core pull is about 91 GB. A quality tier adds 29 to 63 GB per model. A feature's outputs are another 20 to 100 GB. A 150 GB disk is the minimum, and 200 GB is comfortable. If the cache is on an external disk, set HF_HUB_CACHE before pulling.

Make a five-minute film first. Not a feature.

Setup

The full runbook is written for an agent to execute on a fresh Mac. The short version:

brew install ffmpeg rapid-mlx
python3 -m pip install --user -U "huggingface_hub[cli]"

mkdir -p ~/src/ml/skills && cd ~/src/ml/skills
curl -fL -o film-production-skills-2026-10-30.tar.gz \
  https://nyimbi.com/downloads/film-production-skills-2026-10-30.tar.gz
tar -xzf film-production-skills-2026-10-30.tar.gz
cd film-production-skills
python3 sync-skills.py

python3 skills/film-production/scripts/pull_models.py --tier core --check
python3 skills/film-production/scripts/pull_models.py --tier core --yes
./smoke_test.sh

Then start the servers on 8001 and up, because omlx may already own 8000:

rapid-mlx serve qwen3.6-35b-ud --port 8001
rapid-mlx serve flux2-klein-4b --port 8002
rapid-mlx serve ltx-2.3-mlx-q4 --port 8003 --video-output-dir ~/films/_gen
rapid-mlx serve qwen3-tts-4bit --port 8004
rapid-mlx serve whisper --port 8005

The runbook covers preflight, disk and memory, optional Unsloth and ComfyUI installs, model tiers, health checks, the smoke test, the first project, the pilot and the failure modes. It stops on conflicts and failed disk checks. It does not use cloud keys, duplicate model stores, or rm -rf.

What this does not solve

Two model families were retired in the five weeks before this was written. Sora 2 and the OpenAI Videos API went on 24 September. Veo 3.1 Preview left the Gemini API on 22 October. A pipeline that hardcodes a provider has an expiry date.

The rest is craft. Photoreal human motion is the hardest local target. Lip-sync still fails on profiles, teeth, occlusion and fast speech. Local voices are consistent but flatter than the best commercial work. Upscaling adds sharpness, not detail. Four bits is fast because it has given something up.

A local-only pipeline is a choice with a look. Make it on purpose.

Method and sources

The survey started with the skills already on the machine, then moved outward: GitHub searches for filmmaking, AI film, storyboard, screenplay and film production skills; then the practitioner guides and vendor documentation behind them. Candidate metadata came from the GitHub API on 30 October 2026.

The adopted skills cover directing, prompt craft and character consistency. The new skills cover the missing connective tissue: story development, production design, generation execution, postproduction, key art and the state machine that holds them together. The model-specific skill per provider was rejected because model churn is faster than skill churn; one dated dialect table is easier to maintain.

The current models were verified against vendor documentation on 30 October 2026. Gemini Omni Flash 1.1, Kling 4.0, Seedance 2.5, WAN 3.0, MiniMax H3, Veo 3.1, Gen-4.5 and the Runway catalog are the current cloud families. Sora 2 and Veo 3.1 Preview are retired. The local stack was probed on the machine: ComfyUI installed but not running, LTX 0.9.8, MiniMax H3 and Wan 2.2 GGUF files present, rapid-mlx 0.14.3, Unsloth 2026.10.10, 96 GB unified memory and 146 GiB free at the time of the survey.

The archived source documents and the extracted claims are in the skills package under film-production/references/sources/. The full research notes, including the rejected options and the corrections made during the work, are in the downloadable report.

Appendix A: core model manifest

Quality additions: qwen-image at q6 (28.9 GB) for text in key art, wan2.2-i2v-a14b-q8 (39.7 GB) for photoreal image-to-video, ltx-2.5-mlx-q8 (63.0 GB) for the best local LTX, qwen-image-edit at q8 (34.9 GB) for edits, and hidream-o1-dev at bf16 (16.4 GB). Tools: MuseTalk 1.5 MLX, LatentSync 1.5, ACE-Step 1.5, MusicGen medium.

Appendix B: pilot score sheet

Score each dimension from 1 to 5, blind, against one reference clip.

Pass for a parity claim: identity at least 4, lip-sync at least 3.5, realism at least 3.5, voice consistency at least 4, emotional read at least 3, no hard artifact. Record seconds per accepted shot, takes per accepted shot, GPU hours and cloud spend.

The pilot is the measurement. Model cards, demos and "fully AI" claims are not.

Downloads

• Skills package
• Agent setup runbook
• Report as Markdown
• Report as HTML
• The short version