
AI漫剧不是"输入一句话,出来一部剧"。专业做法是把它拆成一条可调试的流水线:剧本结构化 → 分镜拆解 → 角色一致性控制 → 图像生成 → 配音与口型 → 视频合成。下面用可运行代码跑通最小闭环,重点讲清楚每一环的工程约束。
大模型直接生成"剧本"没法用,必须约束输出结构。用JSON Schema强制分镜字段,这是整条流水线的契约。
# script_gen.py
import json, openai
SCHEMA = {
"type": "object",
"properties": {
"title": {"type": "string"},
"characters": {"type": "array", "items": {
"type": "object",
"properties": {"name": {"type": "string"}, "desc": {"type": "string"}}
}},
"shots": {"type": "array", "items": {
"type": "object",
"properties": {
"shot_id": {"type": "integer"},
"scene": {"type": "string"},
"camera": {"type": "string", "enum": ["wide","medium","close-up","over-shoulder"]},
"dialogue": {"type": "string"},
"speaker": {"type": "string"},
"duration_sec": {"type": "number"}
},
"required": ["shot_id","scene","camera","dialogue","speaker","duration_sec"]
}}
},
"required": ["title","characters","shots"]
}
def gen_script(topic):
resp = openai.chat.completions.create(
model="gpt-4o",
response_format={"type": "json_schema", "json_schema": {"name":"manju","schema":SCHEMA}},
messages=[{"role":"user","content":f"生成3分钟漫剧分镜,主题:{topic},每镜5-8秒"}]
)
return json.loads(resp.choices[0].message.content)关键点:camera用枚举锁定,duration_sec限定5-8秒,否则后续视频合成时长不可控。
同一角色跨分镜必须长得一样。专业做法是给每个角色生成一张基准参考图,后续所有分镜用参考图+固定seed做图生图或角色LoRA。
# image_gen.py
import requests, hashlib
def gen_character_ref(name, desc, api_key):
r = requests.post("https://api.openai.com/v1/images/generations",
headers={"Authorization": f"Bearer {api_key}"},
json={"model":"gpt-image-1","prompt":f"动漫角色设定图,{name},{desc},正面全身,白底",
"size":"1024x1024","n":1})
return r.json()["data"][0]["url"]
def gen_shot(shot, char_refs, api_key):
# seed由角色名+镜头号哈希固定,保证可复现
seed = int(hashlib.md5(f"{shot['speaker']}-{shot['shot_id']}".encode()).hexdigest()[:8], 16) % 2**31
prompt = f"{shot['camera']},{shot['scene']},角色参考:{char_refs.get(shot['speaker'],'')}"
r = requests.post("https://api.openai.com/v1/images/generations",
headers={"Authorization": f"Bearer {api_key}"},
json={"model":"gpt-image-1","prompt":prompt,"size":"1024x1024","n":1,"seed":seed})
return r.json()["data"][0]["url"]工程约束:参考图必须先于分镜生成;seed必须持久化存储,否则重跑一次角色就变样,这是新手最容易踩的坑。
# tts.py
def tts(text, voice, out_path, api_key):
r = requests.post("https://api.openai.com/v1/audio/speech",
headers={"Authorization": f"Bearer {api_key}"},
json={"model":"gpt-4o-mini-tts","input":text,"voice":voice,"response_format":"mp3"})
open(out_path,"wb").write(r.content)
# 返回实际时长,用于校正分镜duration
return get_duration(out_path)专业点:TTS实际时长与剧本duration_sec往往有偏差,必须用ffprobe读实际音频时长回写分镜,否则后面合成口型对不上。
# 单镜头:静帧做缓慢推拉运镜,叠加音频
ffmpeg -loop 1 -i shot_01.png -i shot_01.mp3 \
-filter_complex "[0:v]scale=1280:720,zoompan=z='min(zoom+0.0015,1.1)':d=125:s=1280x720[v]" \
-map "[v]" -map 1:a -shortest -c:v libx264 -pix_fmt yuv420p shot_01.mp4
# 多镜头拼接
ffmpeg -f concat -safe 0 -i shots.txt -c copy final.mp4zoompan的d参数按帧率×时长算,shots.txt里按分镜顺序列文件。口型同步更专业的方案是取TTS音素的viseme序列驱动嘴部图层,或用Wav2Lip类模型做后期对齐,但工程上先保证音画时长对齐、运镜统一,已经能出可用的漫剧成片。
用状态机串起来:SCRIPT_DONE → REF_DONE → SHOTS_DONE → TTS_DONE → RENDER_DONE,每步产物落盘并记录seed和参数。调试时重点看三处:分镜JSON是否字段齐全、seed是否持久化、TTS实际时长是否回写。任何一处断链,成片就会出现角色跳脸、口型错位、镜头时长错乱。
AI漫剧的专业度不在模型多强,而在结构化契约 + 一致性控制 + 时长对齐这三件事。把这条流水线跑通,再换更强的图像模型或口型模型,只是替换单点,主线不动。觉得有用点赞收藏,需要完整工程目录可以留言。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。