javascript - Mediastream 管道到 NodeJS socket.io 流到 Google Speech API 并流回响应
问题描述
我想使用Google Speech API实现语音到文本,但在我的前端我不太明白我应该做什么,我在后端和前端都使用Socket.io Stream。
前端(Javascript)
bindSendAudioMessage() {
let me = this;
me.sendAudioMessageButton = me.ele.find('#send-audio-message-btn');
me.sendAudioMessageButton.off('click').one('click', async function () {
let stream = await navigator.mediaDevices.getUserMedia({ audio : true});
me.recordingStarted(stream);
});
},
recordingStarted: function (inputStream) {
let serverStream = ss.createStream();
ss(chatBox.socketIO).emit('speech-to-text', serverStream);
inputStream.pipe(serverStream);
ss(chatBox.socketIO).on('speech-text', function (stream) {
console.log('receiving something');
console.log(stream);
stream.on('data', data => {
console.log(data);
})
})
},
后端(NodeJS)
// Imports the Google Cloud client library
const speech = require('@google-cloud/speech');
// Creates a client
const client = new speech.SpeechClient();
SocketStream(socket).on('speech-to-text', function (inputStream) {
console.log(inputStream);
const request = {
config: {
encoding: 'LINEAR16',
sampleRateHertz: 16000,
languageCode: 'en-US',
},
interimResults: false, // If you want interim results, set this to true
single_utterance: true,
};
// Create a recognize stream
const recognizeStream = client
.streamingRecognize(request)
.on('error', console.error)
.on('data', data =>
process.stdout.write(
data.results[0] && data.results[0].alternatives[0]
? `Transcript: ${data.results[0].alternatives[0].transcript}\n`
: `\n\nReached transcription time limit, press Ctrl+C\n`
)
);
let outputStream = SocketStream.createStream();
SocketStream(socket).emit('speech-text', outputStream);
// Pipe inputStream to recognizeStream then to outputStream
inputStream.pipe(recognizeStream).pipe(outputStream);
})
我确信我在流 API 中缺少一些东西,我知道的一个问题是navigator.mediaDevices.getUserMedia({ audio : true})
会给我一个MediaStream
与SocketIO Stream
.
如何准备音频
MediaStream
以将其流式传输到SocketIO Stream
?当我从 Google API 获得响应时,如何流回响应?
这条线
inputStream.pipe(recognizeStream).pipe(outputStream);
有意义吗?
解决方案
推荐阅读
- ios - 核心数据和解码:在 json 中更进一步的层次结构
- jasper-reports - 条形图未显示所有类别 Jaspersoft Studio
- elasticsearch - match_phrase elasticsearch 中的可选术语
- sql - SQL Server CTE 语句中的错误
- gitlab - 将 confluence 与 gitlab 连接并更新 confluence 页面
- python - 如何在 Pycharm 中加载依赖项和子文件夹?
- python - Scrapy Python Web Scraping - 创建 XPath
- javascript - 循环 Google 分析 api 调用后的 Javascript,如何显示多个结果?
- css - CSS文本背景悬停效果在鼠标悬停时表现异常
- apache - 带有 apache 过滤器的 Fail2ban 40*-命中