AI Video JSON Schema v2 - Overview

AI Video JSON Schema v2 is a structured JSON specification for AI-generated video pipelines, compatible with rendering tools like Revideo or Remotion.

Design Goals

Goal	Description
AI Friendly	Clear structure, easy for AI to generate and understand
Simple Rendering	Clean mapping to Revideo components
Timeline Logic	Intuitive time-based clip system
Multi-element Support	Video, image, text, shapes, layout containers, subtitles
Multi-track Audio	Support for background music, voice, sound effects simultaneously
Subtitle Sync	Word-level timestamps with highlight support
Extensible	Easy to add new features

Top-level Structure

{
  "meta": {},
  "video": {},
  "audio": {},
  "assets": {},
  "tracks": []
}

Field	Type	Required	Description
meta	object	Yes	Metadata (version, title, description, etc.)
video	object	Yes	Global video settings (resolution, fps, duration, etc.)
audio	object	No	Global audio resources (multi-track: bgm, voice, sfx)
assets	object	No	Asset definitions (fonts, images, videos, audio)
tracks	array	Yes	Array of timeline tracks, each containing multiple clips

Rendering Pipeline

AI generates JSON Schema
        ↓
    Schema Validator
        ↓
    Schema Parser
        ↓
    Revideo Renderer
        ↓
    Video Frame Output
        ↓
    Final Video File

Quick Example

{
  "meta": {
    "version": "2.0.0",
    "title": "Product Demo Video"
  },
  "video": {
    "width": 1920,
    "height": 1080,
    "fps": 30,
    "duration": 15
  },
  "audio": {
    "bgm": {
      "src": "https://example.com/bgm.mp3",
      "volume": 0.4
    }
  },
  "tracks": [
    {
      "id": "main",
      "type": "visual",
      "clips": [
        {
          "type": "text",
          "text": "Welcome",
          "start": 0.5,
          "duration": 5,
          "transform": {
            "x": "50%",
            "y": "30%",
            "anchor": "center"
          },
          "style": {
            "fontSize": 96,
            "fill": "#ffffff"
          },
          "animations": [
            { "type": "fadeIn", "duration": 1 }
          ]
        }
      ]
    }
  ]
}

Documentation Index

Document	Description
meta	Metadata configuration details
video	Video configuration details
audio	Audio configuration details (multi-track support)
assets	Asset definition details
tracks	Track system details
clips	Clip element details
transform	Transform property details
style	Style property details
animations	Animation system details
layout	Layout container details
gradients	Gradient fill details
filters	Filter effect details
api	API documentation
examples	Complete example collection
subtitle	Subtitle system details

API Endpoints

POST /api/json

Create a new video configuration.

Request Body: VideoSchema JSON

Response:

{
  "taskId": "uuid",
  "url": "/v/uuid",
  "configUrl": "uuid.json",
  "schema": "AI Video JSON Schema v2"
}

GET /v/:id

View the video player.

TypeScript Type Definitions

For complete TypeScript definitions, refer to:

src/revideo/schema/types.ts - Schema type definitions
src/revideo/clips/types.ts - Clip renderer types

Version History

Version	Date	Changes
2.0.0	2024	Complete redesign with nested layouts, gradients, filters, full animation system
1.0.0	2024	Initial version with simple image slideshow + subtitles

Overview

On this page