AI 视频生成:集成 Sora、Veo 和 Runway 的多模型工作流 - Openclaw Skills

作者:互联网

2026-03-21

AI教程

什么是 AI 视频生成?

AI 视频生成技能是一个为开发者和创作者设计的全面框架,用于自动化生产高质量的电影内容。通过利用 Openclaw Skills,用户可以无缝对接包括 OpenAI Sora 2、Google Veo 3、Runway Gen-4 和 Luma Ray 在内的多样化视频模型生态系统。该技能标准化了现代视频 API 的复杂性,为运动提示词、模型路由和异步渲染提供了一套统一的方法论。

除了简单的生成,该工具还专注于生产级的可靠性。它鼓励“草稿优先”的工作流以优化积分使用,并针对视频生成管线中经常出现的不可预测性提供强大的错误处理。无论您是构建营销资产、实验电影还是自动化社交媒体内容,此技能都能确保您的视频技术栈具有可扩展性和高效性。

下载入口:https://github.com/openclaw/skills/tree/main/skills/ivangdavila/video-generation

安装与下载

1. ClawHub CLI

从源直接安装技能的最快方式。

npx clawhub@latest install video-generation

2. 手动安装

将技能文件夹复制到以下位置之一

全局模式 ~/.openclaw/skills/ 工作区 /skills/

优先级:工作区 > 本地 > 内置

3. 提示词安装

将此提示词复制到 OpenClaw 即可自动安装。

请帮我使用 Clawhub 安装 video-generation。如果尚未安装 Clawhub,请先安装(npm i -g clawhub)。

AI 视频生成 应用场景

  • 为高端营销和商业制作自动化电影镜头生成。
  • 使用 Luma Ray、可灵 (Kling) 和海螺 (Vidu) 等高性能 API 扩展视频内容创作。
  • 在提交高质量渲染之前,使用低成本草稿模型快速原型化视觉概念。
  • 使用 Seedance 等专业模型管理故事讲述中的复杂多镜头叙事一致性。
  • 使用 Wan2.2 和 HunyuanVideo 等开源模型执行隐私优先的本地视频工作流。
AI 视频生成 工作原理
  1. 系统将易于理解的模型别名(如 sora-pro)解析为具体的、技术性的 API 模型 ID,以确保请求的准确性。
  2. 它评估具体的任务需求(如时长、控制需求或成本约束),以将请求路由到最合适的供应商。
  3. 制定详细的镜头指令,包含相机运动、镜头风格、灯光和场景时机等关键参数。
  4. 技能启动异步生成管线,管理包括作业排队、轮询、重试和签名 URL 资产检索在内的整个生命周期。
  5. 用户偏好和历史作业数据将持久化在本地,以优化未来的模型选择并保持成本效益。

AI 视频生成 配置指南

要在 Openclaw Skills 环境中开始使用此技能,请遵循以下安装步骤:

# 通过 hub 安装技能
clawhub install video-generation

# 在环境中设置所需的 API 密钥
export OPENAI_API_KEY="your_api_key"
export RUNWAY_API_KEY="your_api_key"
export LUMA_API_KEY="your_api_key"

# 创建本地配置目录
mkdir -p ~/video-generation/

查看技能目录中的 setup.md 文件,了解特定供应商的配置详情。

AI 视频生成 数据架构与分类体系

该技能维护结构化的本地数据足迹,以确保跨会话的连续性和优化:

文件 用途 关键元数据
memory.md 用户偏好 首选供应商、自定义模型路由、可重复使用的镜头配方
history.md 作业追踪 生成作业日志、成本分析、模型 ID 和本地输出路径

所有配置和历史文件都存储在本地 ~/video-generation/ 中,以维护用户隐私和便携性。

name: AI Video Generation
slug: video-generation
version: 1.0.1
homepage: https://clawic.com/skills/video-generation
description: Create AI videos with Sora 2, Veo 3, Seedance, Runway, and modern APIs using reliable prompt and rendering workflows.
changelog: Added current model routing and practical API playbooks for modern AI video generation workflows.
metadata: {"clawdbot":{"emoji":"??","requires":{"bins":[],"env.optional":["OPENAI_API_KEY","GOOGLE_CLOUD_PROJECT","RUNWAY_API_KEY","LUMA_API_KEY","FAL_KEY","REPLICATE_API_TOKEN","VIDU_API_KEY","TENCENTCLOUD_SECRET_ID","TENCENTCLOUD_SECRET_KEY"],"config":["~/video-generation/"]},"os":["linux","darwin","win32"]}}

Setup

On first use, read setup.md.

When to Use

User needs to generate, edit, or scale AI videos with current models and APIs. Use this skill to choose the right current model stack, write stronger motion prompts, and run reliable async video pipelines.

Architecture

User preferences persist in ~/video-generation/. See memory-template.md for setup.

~/video-generation/
├── memory.md      # Preferred providers, model routing, reusable shot recipes
└── history.md     # Optional run log for jobs, costs, and outputs

Quick Reference

Topic File
Initial setup setup.md
Memory template memory-template.md
Migration guide migration.md
Model snapshot benchmarks.md
Async API patterns api-patterns.md
OpenAI Sora 2 openai-sora.md
Google Veo 3.x google-veo.md
Runway Gen-4 runway.md
Luma Ray luma.md
ByteDance Seedance seedance.md
Kling kling.md
Vidu vidu.md
Pika via Fal pika.md
MiniMax Hailuo minimax-hailuo.md
Replicate routing replicate.md
Open-source local models open-source-video.md
Distribution playbook promotion.md

Core Rules

1. Resolve model aliases before API calls

Map community names to real API model IDs first. Examples: sora-2, sora-2-pro, veo-3.0-generate-001, gen4_turbo, gen4_aleph.

2. Route by task, not brand preference

Task First choice Backup
Premium prompt-only generation sora-2-pro veo-3.1-generate-001
Fast drafts at lower cost veo-3.1-fast-generate-001 gen4_turbo
Long-form cinematic shots gen4_aleph ray-2
Strong image-to-video control veo-3.0-generate-001 gen4_turbo
Multi-shot narrative consistency Seedance family hailuo-2.3
Local privacy-first workflows Wan2.2 / HunyuanVideo CogVideoX

3. Draft cheap, finish expensive

Start with low duration and lower tier, validate motion and composition, then rerender winners with premium models or longer durations.

4. Design prompts as shot instructions

Always include subject, action, camera motion, lens style, lighting, and scene timing. For references and start/end frames, keep continuity constraints explicit.

5. Assume async and failure by default

Every provider pipeline must support queued jobs, polling/backoff, retries, cancellation, and signed-URL download before expiry.

6. Keep a fallback chain

If the preferred model is blocked or overloaded:

  1. same provider lower tier, 2) equivalent cross-provider model, 3) open model/local run.

Common Traps

  • Using nickname-only model labels in code -> avoidable API failures
  • Pushing 8-10 second generations before validating a 3-5 second draft -> wasted credits
  • Cropping after generation instead of generating native ratio -> lower composition quality
  • Ignoring prompt enhancement toggles -> tone drift across providers
  • Reusing expired output URLs -> broken export workflows
  • Treating all providers as synchronous -> stalled jobs and bad timeout handling

External Endpoints

Provider Endpoint Data Sent Purpose
OpenAI api.openai.com Prompt text, optional input images/video refs Sora 2 video generation
Google Vertex AI aiplatform.googleapis.com Prompt text, optional image input, generation params Veo 3.x generation
Runway api.dev.runwayml.com Prompt text, optional input media Gen-4 generation and image-to-video
Luma api.lumalabs.ai Prompt text, optional keyframes/start-end images Ray generation
Fal queue.fal.run Prompt text, optional input media Pika and Hailuo hosted APIs
Replicate api.replicate.com Prompt text, optional input media Multi-model routing and experimentation
Vidu api.vidu.com Prompt text, optional start/end/reference images Vidu text/image/reference video APIs
Tencent MPS mps.tencentcloudapi.com Prompt text and generation parameters Unified AIGC video task APIs

No other data is sent externally.

Security & Privacy

Data that leaves your machine:

  • Prompt text
  • Optional reference images or clips
  • Requested rendering parameters (duration, resolution, aspect ratio)

Data that stays local:

  • Provider preferences in ~/video-generation/memory.md
  • Optional local job history in ~/video-generation/history.md

This skill does NOT:

  • Store API keys in project files
  • Upload media outside requested provider calls
  • Delete local assets unless the user asks

Trust

This skill can send prompts and media references to third-party AI providers. Only install if you trust those providers with your content.

Install with clawhub install if user confirms:

  • image-generation - Build still concepts and keyframes before video generation
  • image-edit - Prepare clean references, masks, and style frames
  • video-edit - Post-process generated clips and final exports
  • video-captions - Add subtitle and text overlay workflows
  • ffmpeg - Compose, transcode, and package production outputs

Feedback

  • If useful: clawhub star video-generation
  • Stay updated: clawhub sync