@remotion/whisper-web

warning

不稳定的 API：此软件包目前处于实验阶段。在测试过程中，我们可能会对 API 做一些更改，并在未来切换到基于 WebGPU 的后端。

类似于 @remotion/install-whisper-cpp，但用于浏览器。允许你在浏览器中本地转录音频，借助 WASM。

🌐 Similar to @remotion/install-whisper-cpp but for the browser. Allows you to transcribe audio locally in the browser, with the help of WASM.

安装

🌐 Installation

Remotion CLI
npm
bun
pnpm
yarn

npx remotion add @remotion/whisper-web

npm i --save-exact @remotion/whisper-web@4.0.431

This assumes you are currently using v4.0.431 of Remotion.
Also update remotion and all `@remotion/*` packages to the same version.
Remove all ^ character in front of the version numbers of it as it can lead to a version conflict.

pnpm i @remotion/whisper-web@4.0.431

bun i @remotion/whisper-web@4.0.431

yarn --exact add @remotion/whisper-web@4.0.431

所需配置

🌐 Required configuration

@remotion/whisper-web 使用一个需要 SharedArrayBuffer 支持的 WebAssembly (WASM) 后端。要启用此功能，你需要在应用中配置跨源隔离头。这是现代浏览器中使用 SharedArrayBuffer 的安全要求。

参见：重要注意事项

🌐 See: Important Considerations

快

🌐 Vite

要在 Vite 中使用 @remotion/whisper-web，你需要做两个重要的更改：

🌐 To use @remotion/whisper-web with Vite, you need to make two important changes:

将该包排除在 Vite 的依赖优化之外以防止已知问题
配置 SharedArrayBuffer 支持所需的安全头

vite.config.ts
import {defineConfig} from 'vite';

export default defineConfig({
  optimizeDeps: {
    // turn off dependency optimization: https://github.com/vitejs/vite/issues/11672#issuecomment-1397855641
    exclude: ['@remotion/whisper-web'],
  },
  // required by SharedArrayBuffer
  server: {
    headers: {
      'Cross-Origin-Embedder-Policy': 'require-corp',
      'Cross-Origin-Opener-Policy': 'same-origin',
    },
  },
  // ...
});

示例用法

🌐 Example usage

Transcribing with @remotion/whisper-web
import {transcribe, canUseWhisperWeb, resampleTo16Khz, downloadWhisperModel} from '@remotion/whisper-web';

const file = new File([], 'audio.wav');
const modelToUse = 'tiny.en';

const {supported, detailedReason} = await canUseWhisperWeb(modelToUse);
if (!supported) {
  throw new Error(`Whisper Web is not supported in this environment: ${detailedReason}`);
}

console.log('Downloading model...');
await downloadWhisperModel({
  model: modelToUse,
  onProgress: ({progress}) => console.log(`Downloading model (${Math.round(progress * 100)}%)...`),
});

console.log('Resampling audio...');
const channelWaveform = await resampleTo16Khz({
  file,
  onProgress: (p) => console.log(`Resampling audio (${Math.round(p * 100)}%)...`),
});

console.log('Transcribing...');
const {transcription} = await transcribe({
  channelWaveform,
  model: modelToUse,
  onProgress: (p) => console.log(`Transcribing (${Math.round(p * 100)}%)...`),
});

console.log(transcription.map((t) => t.text).join(' '));