javascript - 从 websocket 播放 OPUS 时的声音调度问题
问题描述
我正在尝试使用库https://github.com/AnthumChris/opus-stream-decoder/
我有一个来自高质量麦克风的 OPUS 编码声音流(2ch,48kHz)(但我在它上面循环播放音乐来测试这个)。我知道它有效,因为如果我使用,我可以听到它:
websocat --binary ws://third-i.local/api/sound - | mpv -
(它正在打开 websocket 并将其输出流式传输到 mpv(mplayer))。
但是当我在浏览器中播放时,我听到的只是每秒钟左右声音的一小部分。但声音本身听起来不错(我相信它只是音乐的一小部分)。
下面是我写的在浏览器中监听的 JS 代码:
let audioWorker: any;
let exampleSocket;
let opusDecoder: any;
let audioCtx: any;
let startTime = 0;
let counter = 0;
function startAudio() {
/*
const host = document.location.hostname;
const scheme = document.location.protocol.startsWith("https") ? "wss" : "ws";
const uri = `${scheme}://${host}/api/sound`;
*/
const uri = "ws://third-i.local/api/sound";
audioCtx = new AudioContext();
startTime = 100 / 1000;
exampleSocket = new WebSocket(uri);
exampleSocket.binaryType = "arraybuffer";
opusDecoder = new OpusStreamDecoder({onDecode});
exampleSocket.onmessage = (event) => opusDecoder.ready.then(
() => opusDecoder.decode(new Uint8Array(event.data))
);
exampleSocket.onclose = () => console.log("socket is closed!!");
}
function onDecode({left, right, samplesDecoded, sampleRate}: any) {
const source = audioCtx.createBufferSource();
const buffer = audioCtx.createBuffer(2, samplesDecoded, sampleRate);
buffer.copyToChannel(left, 0);
buffer.copyToChannel(right, 1);
source.buffer = buffer;
source.connect(audioCtx.destination);
source.start(startTime);
startTime += buffer.duration;
}
https://github.com/BigBoySystems/third-i-frontend/blob/play-audio/src/App.tsx#L54-L88
解决方案
调度的问题是由于您在创建 WebSocket 的同时创建了 AudioContext,从而将连接时间添加到了AudioContext
的调度中。
换句话说,当您创建AudioContext
调度时会立即开始,但由于 AudioContext 是在创建 WebSocket 时创建的(仅开始连接),因此调度会因 WebSocket 连接到上游所需的时间而关闭并接收第一个字节。
这是您修复的代码:
let audioStreamSocket;
let opusDecoder: any;
let audioCtx: AudioContext;
let startTime: number;
function startAudio() {
const host = document.location.hostname;
const scheme = document.location.protocol.startsWith("https") ? "wss" : "ws";
const uri = `${scheme}://${host}/api/sound`;
audioStreamSocket = new WebSocket(uri);
audioStreamSocket.binaryType = "arraybuffer";
opusDecoder = new OpusStreamDecoder({ onDecode });
audioStreamSocket.onmessage = (event) =>
opusDecoder.ready.then(() => opusDecoder.decode(new Uint8Array(event.data)));
}
function onDecode({ left, right, samplesDecoded, sampleRate }: any) {
if (audioCtx === undefined) {
// See how we create the AudioContext only after some data has been received
// and successfully decoded <=====================================
console.log("Audio stream connected");
audioCtx = new AudioContext();
startTime = 0.1;
}
const source = audioCtx.createBufferSource();
const buffer = audioCtx.createBuffer(2, samplesDecoded, sampleRate);
buffer.copyToChannel(left, 0);
buffer.copyToChannel(right, 1);
source.buffer = buffer;
source.connect(audioCtx.destination);
source.start(startTime);
startTime += buffer.duration;
}
推荐阅读
- javascript - 使用 Jquery 限制文本字段
- json - POST 返回 404,而 GET 在 Ajax 调用 SpringMVC 控制器时工作正常
- c# - 抽象类和具体类 - getter 和 setter
- character-encoding - ZPL 打印 € 和带有锐角的字符
- java - 本地日期时间 JPA - Java 8
- java - 返回上一个开关盒
- r - 如何通过字符串列半连接两个数据帧,其中一个以冒号分隔
- c++ - 选择功能将 Enter 键(来自用户输入)识别为客户端请求
- powerbi - PowerBI:获取每个位置和日期的区域最大值
- python - ValueError:没有足够的值来解包(预期为 3,得到 1)以打开图像