# Agent setup — local AI film production on Apple Silicon

This is a runbook for an AI agent with shell access. It installs the film-production
skills, the local runtimes, and the model set on a fresh Apple Silicon Mac, then proves the
install with a smoke test. It does not generate a finished film; it gets the machine to the
point where the film pipeline can run.

**Scope:** macOS 14+, Apple Silicon, 64 GB unified memory minimum, 96 GB recommended,
150 GB free disk minimum, 200 GB recommended. Linux/CUDA is out of scope.

**Result:** 10 skills installed in the agent's skill roots, `rapid-mlx` serving local
models from the Hugging Face cache, the core model set present, `ffmpeg` working, and a
two-shot project that scaffolds, compiles prompts, validates and assembles.

---

## 0. Rules

1. **Local only.** No cloud model, no API key, no remote generation on the critical path.
2. **One model store.** Every weight goes in `~/.cache/huggingface/hub` (or `HF_HUB_CACHE`).
   Do not create a second store.
3. **Do not delete.** No `rm -rf`, no cache purge, no “clean up” of model directories.
   Partial downloads resume.
4. **One heavy model at a time.** Do not co-load the LLM, the image model and a 14B video
   model. Stop a server before loading a heavier one.
5. **Ports:** `omlx` may own `:8000`; use `8001+` for rapid-mlx. Bind to `127.0.0.1` only.
6. **Gates are real.** If `filmctl validate` fails, read the missing artifact and fix it.
   Do not bypass a gate to “keep moving”.
7. **Stop and report** if a preflight check fails, a required download cannot finish, or
   the smoke test fails twice. Do not improvise a different stack.

---

## 1. Preflight

Run each command; stop if the result is not what is expected.

```bash
uname -m                          # expect arm64
sysctl -n hw.memsize              # expect >= 68719476736 (64 GB); 96 GB ideal
df -h "$HOME" | tail -1           # expect >= 150 GiB available
xcode-select -p                   # expect a path; if not: xcode-select --install
brew --version                    # if missing, install Homebrew first
python3 --version                 # expect >= 3.11; rapid-mlx prefers 3.12+
ffmpeg -version | head -1         # if missing, install in step 3
curl -sI https://huggingface.co | head -1   # expect 200/302
```

If disk is below 150 GB, either free space or set an external cache:

```bash
export HF_HUB_CACHE=/Volumes/<disk>/huggingface/hub
mkdir -p "$HF_HUB_CACHE"
```

Record the machine, RAM, free disk and macOS version for the final report.

---

## 2. Install the skills package

```bash
mkdir -p ~/src/ml/skills
cd ~/src/ml/skills
curl -fL -o film-production-skills-2026-10-30.tar.gz \
  https://nyimbi.com/downloads/film-production-skills-2026-10-30.tar.gz
curl -fL -o film-production-skills-2026-10-30.tar.gz.sha256 \
  https://nyimbi.com/downloads/film-production-skills-2026-10-30.tar.gz.sha256
shasum -a 256 -c film-production-skills-2026-10-30.tar.gz.sha256
tar -xzf film-production-skills-2026-10-30.tar.gz
cd film-production-skills
```

Expected tree:

```text
film-production-skills/
  skills/
    film-production/          orchestrator, filmctl.py, pull_models.py, manifest, sources
    film-story-development/   screenplay, structure, series bible, storycheck.py
    film-design/              style lock, world bible, continuity
    character-design/         character pack, identity lock, voice, drift audit
    film-directing/           adopted DirectorSKILL (MIT)
    ai-video-directing/       adopted Visual Skills video (CC BY 4.0)
    ai-image-art-direction/   adopted Visual Skills image (CC BY 4.0)
    film-generation/          canonical shot, dialects, local stack, QA
    film-postproduction/      edit, sound, QC, delivery
    film-poster/              key art, poster, template
  sync-skills.py
  smoke_test.sh
  AGENT_SETUP.md
  RESEARCH.md
  RESEARCH.html
  research.css
  NOTICE.md
```

Sync into the agent skill roots. The script links by default into `~/.claude/skills`,
`~/.config/opencode/skill`, `~/.agents/skills`, `~/.pi/agent/skills` and `~/.codex/skills`;
missing roots are created.

```bash
python3 sync-skills.py --dry-run
python3 sync-skills.py
```

If a root reports a **conflict** (a real directory or a foreign symlink with the same
name), stop and report it. Do not pass `--force`; resolve the collision manually.

If the machine already has these skills from an earlier install, `sync-skills.py` may
re-link managed entries to this package. Stop and reconcile with the user before running it.

Verify:

```bash
for d in film-production film-story-development film-design character-design \
         film-directing ai-video-directing ai-image-art-direction \
         film-generation film-postproduction film-poster; do
  test -f ~/.claude/skills/$d/SKILL.md || echo "MISSING $d"
done
```

Adjust the root if the target agent does not read `~/.claude/skills`; the important thing is
that all ten `SKILL.md` files are reachable by that agent.

---

## 3. Install the runtimes

```bash
brew install ffmpeg rapid-mlx
```

If `rapid-mlx` is not available in Homebrew on that machine:

```bash
curl -fsSL https://rapidmlx.com/install.sh | bash
export PATH="$HOME/.local/bin:$PATH"
```

Install the Hugging Face CLI (the pull script uses it):

```bash
python3 -m pip install --user -U "huggingface_hub[cli]"
export PATH="$HOME/.local/bin:$PATH"
hf --version
```

Optional, only if the user wants the Studio, training, avatar or media pages:

```bash
brew install --cask unsloth
unsloth doctor          # expect Metal/MPS detected; CUDA is not required
```

Optional, only for GGUF video models, control nets, upscale and interpolation:

```bash
git clone https://github.com/comfyanonymous/ComfyUI ~/src/ComfyUI
cd ~/src/ComfyUI
python3 -m venv venv
./venv/bin/pip install -r requirements.txt
```

ComfyUI is **not** required for the core rapid-mlx path. Do not install it just to satisfy a
checklist.

Verify:

```bash
ffmpeg -version | head -1
rapid-mlx --version
hf --version
```

---

## 4. Pull the models into the HF cache

```bash
cd ~/src/ml/skills/film-production-skills
S=skills/film-production/scripts/pull_models.py

python3 "$S" --list
python3 "$S" --tier core --check
python3 "$S" --tier core --yes
```

The core set is ~91 GB (about 66 GB if some models are already present). It is sequential;
expect 1–3 hours on a domestic connection. Run it in the background so the session is not
blocked:

```bash
nohup python3 "$S" --tier core --yes \
  > ~/Library/Logs/film-model-pull.log 2>&1 &
echo $!
```

Monitor without polling too often:

```bash
tail -20 ~/Library/Logs/film-model-pull.log
python3 "$S" --tier core --check      # exit 0 when all present
du -sh ~/.cache/huggingface/hub
```

Do not start the next step until `--check` exits 0.

Optional quality tier, only if disk allows (choose at most two of the 29–63 GB models):

```bash
python3 "$S" --tier quality --check
python3 "$S" --tier quality --yes
```

Optional tools (lip-sync, music):

```bash
python3 "$S" --tier tools --check
python3 "$S" --tier tools --yes
```

If the disk check fails, stop and report the numbers. Do not pass `--allow-tight` without
the user's explicit agreement.

---

## 5. Start the local servers

Run each in its own terminal, tmux session or background process. One model per process.
Use `8001+`; `omlx` may already own `8000`.

```bash
# LLM — story, screenplay, prompt compilation
rapid-mlx serve qwen3.6-35b-ud --port 8001

# Image — keyframes, character sheets, storyboard panels
rapid-mlx serve flux2-klein-4b --port 8002

# Video — default shot engine, q4 with audio
rapid-mlx serve ltx-2.3-mlx-q4 --port 8003 \
  --video-output-dir "$HOME/films/_gen"

# TTS — dialogue and scratch voice
rapid-mlx serve qwen3-tts-4bit --port 8004

# STT and alignment — captions
rapid-mlx serve whisper --port 8005
```

Health checks:

```bash
curl -s localhost:8001/v1/models
curl -s localhost:8002/v1/models
curl -s localhost:8003/v1/videos/capabilities
curl -s localhost:8004/v1/audio/voices
curl -s localhost:8005/v1/models
```

`rapid-mlx ps` lists running servers. Stop a server with Ctrl-C or `kill` its PID before
loading a heavier model. Do not bind to `0.0.0.0`.

---

## 6. Smoke test

```bash
cd ~/src/ml/skills/film-production-skills
./smoke_test.sh
```

Expected:

```text
filmctl: PASS
smoke: ok
```

The test scaffolds a temporary project, writes a one-shot `shots.json`, generates a
synthetic clip with `ffmpeg`, compiles a prompt, validates the generation gate, assembles a
rough cut, and checks the output duration. It does not call a model. A failure here is an
install problem; fix it before going further.

---

## 7. First project

```bash
FILMCTL=skills/film-production/scripts/filmctl.py
python3 "$FILMCTL" doctor
python3 "$FILMCTL" new first-film --root "$HOME/films" \
  --format short --runtime 480 --aspect 16:9 --tier local
```

Then load the **`film-production`** skill in the agent and follow the pipeline. Do not
reimplement it. The order is:

1. `film-story-development` → `01_story/` (screenplay, scene cards, quality report).
2. `film-design` + `character-design` → `02_design/` (style lock, character packs, world bible).
3. `film-directing` → `03_plan/` (shot list, boards, animatic).
4. `film-generation` → `04_gen/` (keyframes, compiled prompts, takes, QA).
5. `film-postproduction` → `05_post/` (cut, sound, grade, captions, delivery).
6. `film-poster` → `06_marketing/poster/` (key art, one-sheet, thumbnail).

The script compiles prompts but does not call the video endpoint. The generation step must
send the compiled prompt to the chosen provider and save the returned clip under
`04_gen/shots/<shot>/`, then record it in `generation-log.csv`. The endpoint map is in
`skills/film-generation/references/local-stack.md` and
`skills/film-production/references/local-production-manifest.md`; the model matrix is in
`skills/film-generation/references/model-dialects.md`.

---

## 8. Quality gate before a series

Do not promise TikTok-grade quality until a pilot passes. The protocol is in
`skills/film-production/references/quality-ceiling.md`:

- 60–90 seconds, two principals, one location, 15–20 shots, four to six dialogue lines.
- Hold script, keyframes, references, voices, aspect and edit constant.
- Run LTX-2.3 q4, Wan 2.2 I2V-A14B q8, and one cloud model only if the user permits it.
- Score blind on identity, lip-sync, realism, voice consistency, emotional read and artifacts.
- Pass: identity ≥ 4, lip-sync ≥ 3.5, realism ≥ 3.5, voice consistency ≥ 4, emotional read ≥ 3,
  no hard artifact.

Record the scores, seconds per accepted shot, takes per accepted shot and spend. The
decision (local, hybrid, or a narrower format) goes in `decision-log.md`.

---

## 9. Acceptance checklist

- [ ] `uname -m` is `arm64`; RAM and disk recorded.
- [ ] All ten `SKILL.md` files are reachable by the agent.
- [ ] `ffmpeg`, `rapid-mlx` and `hf` are on `PATH`.
- [ ] `python3 pull_models.py --tier core --check` exits 0.
- [ ] `curl` health checks answer on 8001–8005.
- [ ] `./smoke_test.sh` prints `smoke: ok`.
- [ ] `filmctl doctor` reports ffmpeg, rapid-mlx, and the local model paths.
- [ ] `first-film` project exists with `PRODUCTION.md` and `state.json`.
- [ ] The user has been told the core set is local and no cloud keys are required.
- [ ] The pilot plan is agreed before a series commitment.

Report back: machine, macOS, RAM, free disk, rapid-mlx version, `hf` version, core model
check result, smoke output, and any conflict or failure.

---

## 10. Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| `rapid-mlx: command not found` | not on PATH | `export PATH="$HOME/.local/bin:$PATH"`; reopen the shell |
| `hf: command not found` | user pip bin not on PATH | `export PATH="$HOME/.local/bin:$PATH"`; `python3 -m pip install --user -U "huggingface_hub[cli]"` |
| HF 401/403 | gated repo | `export HF_TOKEN=...` from a Hugging Face account with access |
| disk check fails | footprint + reserve > free | free space, use `HF_HUB_CACHE` on an external disk, or pull fewer models |
| download stalls | network or sleep | rerun the same command; `hf download` resumes; `caffeinate -dimsu` during the pull |
| `--check` says missing while pulling | `.incomplete` blobs | expected; wait for the pull, then recheck |
| port 8000 in use | omlx | use 8001+ |
| ComfyUI not running | optional install | skip; rapid-mlx covers the core path |
| lip-sync missing | tools tier not pulled | `--tier tools`; native lip-sync models are not part of core |
| `filmctl validate` fails | missing artifact or shot field | read the named file/field; fix it; do not bypass |
| smoke test fails twice | install or ffmpeg problem | stop and report the output |

---

## 11. What not to do

- Do not use cloud models, cloud keys, or the `:cloud` models in ollama.
- Do not duplicate weights into `ComfyUI/models/` or ollama’s blob store; symlink from the
  HF cache if ComfyUI needs a file.
- Do not delete partial downloads or “clean” the cache.
- Do not co-load the LLM, the image model and a 14B video model.
- Do not expose the servers beyond `127.0.0.1`.
- Do not claim TikTok parity without the pilot.
- Do not skip the story quality gate. A weak script cannot be rescued by a better model.

---

## 12. Sources and licences

The package includes `RESEARCH.md` and `RESEARCH.html`, plus the source archive under
`skills/film-production/references/sources/`. The web monograph is at
https://nyimbi.com/research/ai-film-production/. Upstream licences are in each skill:
`film-directing` MIT (wangzhang-wu), `ai-video-directing` and
`ai-image-art-direction` CC BY 4.0 (Serge Shima), `character-design` adapted from
Refay Skills CC BY 4.0 (Khaled Al Refay). New material © 2026 Nyimbi Odero, shared as-is.
Model weights carry their own licences; check the model card before a commercial release.