Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/docs/features/proofreading.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ keywords: [字幕校对, 字幕编辑, 校对台, AI 润色, 字幕修改, 时

常用快捷键:`↑↓` 上一条 / 下一条,`Tab` 在原文与译文间切换,`⌘S` 保存,`?` 查看全部快捷键。

## 校正角色归属

启用[角色分离](./subtitle-generation#角色分离)的任务会在每条字幕旁显示角色名和颜色。点击角色标签即可重新分配当前字幕、补充多人重叠角色或指定主要角色;没有可靠识别结果的字幕会显示为「未分配」。

角色工具栏支持按角色、未分配和多人重叠筛选,也可以统一重命名、换色、移动全部字幕或合并误拆角色。角色名称和归属独立保存在校对数据中,相关修改都可撤销 / 重做,不会改动字幕正文或时间轴。

保存时默认不把角色名写入字幕文件;需要交付带角色标记的字幕时,可勾选「在字幕中写入角色名」,导出内容会使用当前角色名称。

## AI 帮你改

- **AI 优化**:选中单条让 AI 润色措辞
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/features/subtitle-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ keywords:

标准的「转写」和「转写 + 翻译」任务可在高级设置中开启**角色分离**。它会在转写、精修与翻译完成后,本地分析整段音频,把不同角色按时间轴对齐到每条字幕;处理耗时与音频长度大致成正比。

首次使用前需在「资源中心 → 本地多模型引擎」确认运行库可用并下载角色分离模型。角色信息默认只保存在应用内校对数据中,不会改变导出的字幕;需要交付可读标签时,可同时开启「将角色标签写入字幕文件」,输出 `[Speaker 1]` 等前缀。
首次使用前需在「资源中心 → 本地多模型引擎」确认运行库可用并下载角色分离模型。角色信息默认只保存在应用内校对数据中,不会改变导出的字幕;需要交付可读标签时,可同时开启「将角色标签写入字幕文件」,初始输出使用 `[Speaker 1]` 等前缀。进入[校对台](./proofreading#校正角色归属)后可以把编号改为真实角色名、纠正归属,并决定保存时是否写入当前角色名称

当前版本暂不在一条龙向导、自定义配方及含配音 / 合成的流程中开放角色分离;这些流程会在角色元数据与配音音色映射完成后再接入。

Expand Down
63 changes: 53 additions & 10 deletions main/helpers/ipcHandlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ import {
readProofreadDataFile,
updateProofreadDataFromSubtitles,
} from './proofreadData';
import {
prefixTextWithSpeakerNames,
type SpeakerInfo,
} from '../../types/proofreadData';
import {
MANUSCRIPT_EXTENSIONS,
ManuscriptFileError,
Expand Down Expand Up @@ -164,18 +168,33 @@ function buildSubtitleFileContent(
filePath: string,
subtitles: any[],
contentType = 'source',
speakerOptions?: {
speakers?: SpeakerInfo[];
embedSpeakerNames?: boolean;
},
): string {
const format = detectSubtitleFormat(filePath);
const withSpeakerPrefix = (subtitle: any, text: string): string => {
if (!speakerOptions?.embedSpeakerNames) return text;
return prefixTextWithSpeakerNames(
text,
subtitle,
speakerOptions.speakers || [],
);
};
const buildText = (subtitle): string => {
let text: string;
if (contentType === 'source') {
return subtitle.sourceContent ?? '';
text = subtitle.sourceContent ?? '';
} else {
const template =
CONTENT_TEMPLATES[contentType] || CONTENT_TEMPLATES.onlyTranslate;
text = renderTemplate(template, {
sourceContent: subtitle.sourceContent ?? '',
targetContent: subtitle.targetContent ?? '',
}).replace(/\n+$/, '');
}
const template =
CONTENT_TEMPLATES[contentType] || CONTENT_TEMPLATES.onlyTranslate;
return renderTemplate(template, {
sourceContent: subtitle.sourceContent ?? '',
targetContent: subtitle.targetContent ?? '',
}).replace(/\n+$/, '');
return withSpeakerPrefix(subtitle, text);
};

return (
Expand Down Expand Up @@ -218,8 +237,17 @@ async function writeSubtitleFile(
filePath: string,
subtitles: any[],
contentType = 'source',
speakerOptions?: {
speakers?: SpeakerInfo[];
embedSpeakerNames?: boolean;
},
): Promise<void> {
const content = buildSubtitleFileContent(filePath, subtitles, contentType);
const content = buildSubtitleFileContent(
filePath,
subtitles,
contentType,
speakerOptions,
);
await backupSubtitleFile(filePath);
await fs.promises.writeFile(filePath, content, 'utf-8');
logMessage(`保存字幕文件成功: ${filePath}`, 'info');
Expand Down Expand Up @@ -386,7 +414,10 @@ export function setupIpcHandlers(mainWindow: BrowserWindow) {
return [];
}
const proofreadData = await readProofreadDataFile(filePath);
return proofreadDataToSubtitleRows(proofreadData);
return {
subtitles: proofreadDataToSubtitleRows(proofreadData),
speakers: proofreadData.speakers,
};
} catch (error) {
logMessage(`读取校对中间态错误: ${error.message}`, 'error');
return [];
Expand Down Expand Up @@ -447,10 +478,14 @@ export function setupIpcHandlers(mainWindow: BrowserWindow) {
{
proofreadDataFile,
subtitles,
speakers = [],
embedSpeakerNames = false,
outputs = [],
}: {
proofreadDataFile: string;
subtitles: any[];
speakers?: SpeakerInfo[];
embedSpeakerNames?: boolean;
outputs: { filePath?: string; contentType?: string }[];
},
) => {
Expand All @@ -461,7 +496,11 @@ export function setupIpcHandlers(mainWindow: BrowserWindow) {
);
}

await updateProofreadDataFromSubtitles(proofreadDataFile, subtitles);
const updated = await updateProofreadDataFromSubtitles(
proofreadDataFile,
subtitles,
speakers,
);

const rendered = new Set<string>();
for (const output of outputs) {
Expand All @@ -474,6 +513,10 @@ export function setupIpcHandlers(mainWindow: BrowserWindow) {
filePath,
subtitles,
output.contentType || 'source',
{
speakers: updated.speakers,
embedSpeakerNames,
},
);
}

Expand Down
131 changes: 93 additions & 38 deletions main/helpers/proofreadData.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,38 +12,28 @@ import {
import { logMessage } from './storeManager';
import {
speakerIdsForCues,
stripSpeakerLabelPrefix,
type SpeakerDiarizationSegment,
} from './speakerDiarization/alignment';
import {
mergeSpeakerIds,
realignSpeakerIdsForCue,
} from '../../types/speakerDiarization';
import {
PROOFREAD_DATA_VERSION,
hasExplicitSpeakerAssignment,
normalizePrimarySpeakerId,
normalizeProofreadData,
normalizeSpeakerIds,
normalizeSpeakerRoster,
shouldRealignSpeakerAssignment,
type ProofreadDataCue,
type ProofreadDataFileV2,
type SpeakerInfo,
} from '../../types/proofreadData';

export interface ProofreadDataCue {
id: string;
startMs: number;
endMs: number;
source: string;
target: string;
/** 一基角色编号,与可选文本标签 `[Speaker N]` 的 N 一致。 */
speakerIds?: number[];
}

export interface ProofreadDataFile {
version: 1;
meta: {
createdAt: string;
updatedAt: string;
sourceLanguage?: string;
targetLanguage?: string;
translateContent?: string;
outputFormat?: string;
sourceFile?: string;
targetFile?: string;
finalTargetFile?: string;
};
cues: ProofreadDataCue[];
}
export type { ProofreadDataCue, SpeakerInfo } from '../../types/proofreadData';
export type ProofreadDataFile = ProofreadDataFileV2;

export interface ProofreadSubtitleRow {
id: string;
Expand All @@ -55,6 +45,8 @@ export interface ProofreadSubtitleRow {
endTimeInSeconds: number;
isEditing: boolean;
speakerIds?: number[];
primarySpeakerId?: number;
speakerAssignmentSource?: 'manual';
}

export type ProofreadDataWriteResult =
Expand Down Expand Up @@ -125,14 +117,23 @@ function buildCues(
targetByTime.get(sourceEntry.startEndTime) || targetEntries[index];
const { startMs, endMs } = parseStartEndTime(sourceEntry.startEndTime);

const assignedSpeakerIds = normalizeSpeakerIds(speakerIds[index]);
const cleanSpeakerPrefix = assignedSpeakerIds.length > 0;
const cue: ProofreadDataCue = {
id: sourceEntry.id || String(index + 1),
startMs,
endMs,
source: entryText(sourceEntry),
target: entryText(targetEntry),
source: cleanSpeakerPrefix
? stripSpeakerLabelPrefix(entryText(sourceEntry))
: entryText(sourceEntry),
target: cleanSpeakerPrefix
? stripSpeakerLabelPrefix(entryText(targetEntry))
: entryText(targetEntry),
};
if (speakerIds[index]?.length) cue.speakerIds = speakerIds[index];
if (assignedSpeakerIds.length) {
cue.speakerIds = assignedSpeakerIds;
cue.primarySpeakerId = assignedSpeakerIds[0];
}
return cue;
});
}
Expand Down Expand Up @@ -170,8 +171,9 @@ export async function writeProofreadDataFromFiles({

const targetEntries = await readSubtitleEntries(targetFile);
const now = new Date().toISOString();
const cues = buildCues(sourceEntries, targetEntries, speakerSegments);
const proofreadData: ProofreadDataFile = {
version: 1,
version: PROOFREAD_DATA_VERSION,
meta: {
createdAt: now,
updatedAt: now,
Expand All @@ -183,7 +185,8 @@ export async function writeProofreadDataFromFiles({
targetFile,
finalTargetFile,
},
cues: buildCues(sourceEntries, targetEntries, speakerSegments),
speakers: normalizeSpeakerRoster([], cues),
cues,
};

const proofreadDataFile = getProofreadDataPath(file);
Expand Down Expand Up @@ -211,11 +214,27 @@ export async function readProofreadDataFile(
filePath: string,
): Promise<ProofreadDataFile> {
const content = await fs.promises.readFile(filePath, 'utf-8');
const parsed = JSON.parse(content) as ProofreadDataFile;
if (parsed?.version !== 1 || !Array.isArray(parsed.cues)) {
try {
const raw = JSON.parse(content);
const normalized = normalizeProofreadData(raw);
// v1 was created while technical labels could still be embedded in the
// source/target text. Keep migration idempotent and only strip labels from
// cues that already carry structured speaker assignments.
if (raw?.version === 1) {
normalized.cues = normalized.cues.map((cue) =>
cue.speakerIds?.length
? {
...cue,
source: stripSpeakerLabelPrefix(cue.source),
target: stripSpeakerLabelPrefix(cue.target),
}
: cue,
);
}
return normalized;
} catch {
throw new Error(`Invalid proofread data file: ${filePath}`);
}
return parsed;
}

export function proofreadDataToSubtitleRows(
Expand All @@ -234,24 +253,35 @@ export function proofreadDataToSubtitleRows(
startTimeInSeconds: cue.startMs / 1000,
endTimeInSeconds: cue.endMs / 1000,
isEditing: false,
...(cue.speakerIds?.length ? { speakerIds: [...cue.speakerIds] } : {}),
...(hasExplicitSpeakerAssignment(cue)
? { speakerIds: [...(cue.speakerIds || [])] }
: {}),
...(cue.primarySpeakerId
? { primarySpeakerId: cue.primarySpeakerId }
: {}),
...(cue.speakerAssignmentSource === 'manual'
? { speakerAssignmentSource: 'manual' as const }
: {}),
};
});
}

export async function updateProofreadDataFromSubtitles(
filePath: string,
subtitles: ProofreadSubtitleRow[],
speakers?: SpeakerInfo[],
): Promise<ProofreadDataFile> {
const existing = await readProofreadDataFile(filePath);
const existingById = new Map(existing.cues.map((cue) => [cue.id, cue]));
const now = new Date().toISOString();
const updated: ProofreadDataFile = {
...existing,
version: PROOFREAD_DATA_VERSION,
meta: {
...existing.meta,
updatedAt: now,
},
speakers: normalizeSpeakerRoster(speakers || existing.speakers, subtitles),
cues: subtitles.map((subtitle, index) => {
const { startMs, endMs } = parseStartEndTime(subtitle.startEndTime);
const source =
Expand All @@ -260,21 +290,46 @@ export async function updateProofreadDataFromSubtitles(
const timingChanged = Boolean(
previous && (previous.startMs !== startMs || previous.endMs !== endMs),
);
const speakerIds = timingChanged
const hasCurrentAssignment = hasExplicitSpeakerAssignment(subtitle);
const currentSpeakerIds = hasCurrentAssignment
? subtitle.speakerIds
: previous?.speakerIds;
const speakerAssignmentSource =
subtitle.speakerAssignmentSource || previous?.speakerAssignmentSource;
// Timing-based realignment is only valid for automatic assignments.
// Once the user has corrected a cue, the current row (including an
// explicit empty array) is authoritative across timing edits and saves.
const speakerIds = shouldRealignSpeakerAssignment(
timingChanged,
speakerAssignmentSource,
)
? realignSpeakerIdsForCue(
startMs,
endMs,
existing.cues,
subtitle.speakerIds || previous?.speakerIds,
currentSpeakerIds,
)
: mergeSpeakerIds(subtitle.speakerIds || previous?.speakerIds);
: mergeSpeakerIds(currentSpeakerIds);
const normalizedSpeakerIds = normalizeSpeakerIds(speakerIds);
const primarySpeakerId = normalizePrimarySpeakerId(
subtitle.primarySpeakerId ?? previous?.primarySpeakerId,
normalizedSpeakerIds,
);
return {
id: subtitle.id || String(index + 1),
startMs,
endMs,
source,
target: subtitle.targetContent ?? '',
...(speakerIds?.length ? { speakerIds: [...speakerIds] } : {}),
...(normalizedSpeakerIds.length || speakerAssignmentSource === 'manual'
? {
speakerIds: normalizedSpeakerIds,
...(primarySpeakerId ? { primarySpeakerId } : {}),
}
: {}),
...(speakerAssignmentSource === 'manual'
? { speakerAssignmentSource: 'manual' as const }
: {}),
};
}),
};
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
"test:custom-languages": "tsc scripts/test-custom-languages.ts --outDir node_modules/.cache/custom-language-tests --module commonjs --moduleResolution node --target es2019 --esModuleInterop --skipLibCheck && node node_modules/.cache/custom-language-tests/scripts/test-custom-languages.js",
"test:refine": "tsc scripts/test-refine-units.ts --outDir node_modules/.cache/refine-tests --module commonjs --moduleResolution node --target es2019 --esModuleInterop --skipLibCheck --resolveJsonModule && node node_modules/.cache/refine-tests/scripts/test-refine-units.js",
"test:speaker-diarization": "node scripts/test-speaker-diarization-config.cjs && tsc scripts/test-speaker-diarization.ts --outDir node_modules/.cache/speaker-diarization-tests --module commonjs --moduleResolution node --target es2019 --esModuleInterop --skipLibCheck && node node_modules/.cache/speaker-diarization-tests/scripts/test-speaker-diarization.js",
"test:proofread-speakers": "tsc scripts/test-proofread-speakers.ts --outDir node_modules/.cache/proofread-speaker-tests --module commonjs --moduleResolution node --target es2019 --esModuleInterop --skipLibCheck && node node_modules/.cache/proofread-speaker-tests/scripts/test-proofread-speakers.js",
"test:manuscript": "tsc scripts/test-manuscript-matching.ts --outDir node_modules/.cache/manuscript-tests --module commonjs --moduleResolution node --target es2022 --esModuleInterop --skipLibCheck --resolveJsonModule && node node_modules/.cache/manuscript-tests/scripts/test-manuscript-matching.js",
"longgap:gen": "tsc scripts/longgap/gen-audio.ts --outDir node_modules/.cache/longgap --module commonjs --moduleResolution node --target es2019 --esModuleInterop --skipLibCheck --resolveJsonModule && node node_modules/.cache/longgap/scripts/longgap/gen-audio.js",
"test:longgap": "tsc scripts/longgap/run.ts --outDir node_modules/.cache/longgap --module commonjs --moduleResolution node --target es2019 --esModuleInterop --skipLibCheck --resolveJsonModule && node node_modules/.cache/longgap/scripts/longgap/run.js",
Expand Down
Loading
Loading