OpenGenie: an ongoing reproduction of Google's Genie API at the lowest cost possible.

Please see below for some examples, and scroll further yet for a list of wanted contributions.

We use training-free diffusion-forcing style sliding latent-token continuation, aggressive Magcaching and extensive kernel fusion to achieve realtime streaming of 576p15 audio-video on an H100, and the video can be steered with text prompts. The aim is to:

• First, build a serving stack for fast Diffusion Transformers rollout. This is where we are now.
• Second, via VLA-Adapter-style training on video-game data, produce a playable, realtime action-steerable model.
• Third, figure out persistent World Memory, possibly through a method like this.

Examples

forest_world_shortprompts_25fps_clean.mp4

A 25-second clip of a claymation character walking forward through a forest. We steer the model to ask for humans walking across at the t=10s point.

An realtime 576p15 LTX2.3-distilled runtime and serverstack, optimized for serving on an H100, aggressive caching, fused kernels, and diffusion-forcing style latent token continuation for unbounded duration videos. But importantly: prompt-steering

combined_dense_annotated.1.mp4

A 25-second clip of a hamster running on a wheel as different steering actions are presented. The overlapping frames are due to an old version where the server was streaming an extra chunk generated by the model that should have been trimmed. It has since been fixed.

Instructions to run

OpenGenie supports out-of-the-box Modal deployment. Follow the instructions below:

Modal Run Instructions

Install dependencies:

uv sync --extra dev
touch env.local

Run the 25s prompt-only world-continuation example on one Modal H100:

PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python uv run modal run \
  modal_apps/ltx23_official_warm_experiments.py \
  --mode forest_prompt_right_model \
  --bucket std_16x9_25fps_5s_overlap32_v1 \
  --attention sdpa-cudnn \
  --seed 20260611 \
  --continuation-frames 17 \
  --plan-limit 5

Run the live H100 worker:

PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python uv run modal serve \
  modal_apps/ltx23_live_h100_worker.py

Check or stop Modal apps:

PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python uv run modal app list
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python uv run modal app stop --yes <app-id>

Contributing

We primarily want contributors in writing kernels for LTX2.3-distilled.

Implemented:

• AdaZero fusion: RMSNorm + Ada scale/shift in one Triton kernel.
• Ada-from-table fusion: avoided materializing Ada shift/scale/gate tensors from scale_shift_table + timestep.
• FP8 input quantization: BF16 activation -> clamped E4M3 before FP8 scaled-mm.
• FP8Linear BF16 bias add: exact Triton broadcast bias after scaled-mm.
• Video Q/K preattention: Q/K RMSNorm + split RoPE fused before FA3 attention.
• Video text AdaLN affine/gated residual path: exact fusion around video text cross-attention.
• Output bias + gated residual: fused video attention output bias with Ada-gated residual.
• Residual gate stride fix / exact residual gate kernels: made x + y * gate exact for the safe sites.
• Audio Ada values: useful in the live 576p/720p-ish path, though not a 1080p exact win.

To be fixed | status:

• FFN GELU + FP8 quantization: parity passed, but E2E timing was neutral/regressed.
• Video FFN output bias + residual: parity passed, but full E2E regressed.
• Video Q/K bias preattention: parity passed, but timing regressed.
• Video MSA branch scaffold / direct branch / gate multiply: exact or unit-tested, but not faster.
• Per-head FP8 Q/K/V quantizer for FP8 FA3: kernel worked, but real LTX FP8 video attention failed quality parity.

To be implemented | status:

• New dense video attention kernel: not built.
• FP8 video self-attention as production path: blocked by real-tensor drift.
• Window/local attention: faster but quality-risky.
• QKV packed/grouped projection variants: slower or failed parity.
• FA3 tile/no-LSE/scheduler source overlays: microbench-neutral or real-denoiser-regressed.

Please reach out at my email or twitter to collaborate!