时间戳

首页 > 解决方案 > 获取 20 毫秒 VoIP 应用程序的麦克风数据回调

ios - 获取 20 毫秒 VoIP 应用程序的麦克风数据回调

问题描述

我正在开发 VOIP 通话应用程序，所以现在我处于需要将语音数据传输到服务器的阶段。为此，我想通过 20 毫秒回调从麦克风获取实时音频语音数据。

我确实搜索了很多链接，但由于我是音频框架的新手，所以我找不到解决方案。

细节

我们有自己的堆栈，例如 WebRTC，它使 RTP 每 20 毫秒从远程发送数据，并从 Mic 询问数据 20 毫秒，我想要实现的是从 mic 获取 20 毫秒的数据并将其传递给堆栈。所以需要知道怎么做。音频格式为 pcmFormatInt16，采样率为 8000 Hz，数据为 20 毫秒。

我已经搜索过

AVAudioEngine、AUAudioUnit、AVCaptureSession 等。

我之前关注过链接 https://gist.github.com/hotpaw2/ba815fc23b5d642705f2b1dedfaf0107

let hwSRate = audioSession.sampleRate

try audioSession.setActive(true)

 print("native Hardware rate : \(hwSRate)")

try audioSession.setPreferredIOBufferDuration(preferredIOBufferDuration)

try audioSession.setPreferredSampleRate(8000) // at 8000.0 Hz

print("Changed native Hardware rate : \(audioSession.sampleRate) buffer duration \(audioSession.ioBufferDuration)")


try auAudioUnit = AUAudioUnit(componentDescription: self.audioComponentDescription)



auAudioUnit.outputProvider = { // AURenderPullInputBlock

                (actionFlags, timestamp, frameCount, inputBusNumber, inputData) -> AUAudioUnitStatus in



                    if let block = self.renderBlock {       // AURenderBlock?
                let err : OSStatus = block(actionFlags,
                                           timestamp,
                                           frameCount,
                                           1,
                                           inputData,
                                           .none)
                if err == noErr {
                    // save samples from current input buffer to circular buffer
                    print("inputData = \(inputData) and frameCount: \(frameCount)")
                    self.recordMicrophoneInputSamples(
                        inputDataList:  inputData,
                        frameCount: UInt32(frameCount) )
                }
            }
            let err2 : AUAudioUnitStatus = noErr
            return err2

}

日志：-

更改了本机硬件速率：8000.0 缓冲区持续时间 0.01600000075995922

标签： iosswiftavfoundationreal-timeavaudiosession

解决方案

渲染块将根据硬件为 AUAudioUnit 和 AudioSession 接受的设置给你回调。如果我们想要来自麦克风的不同大小的输入，我们必须管理缓冲区。输出到扬声器的大小应与预期的大小相同，例如 128、256、512 字节等。 

   try audioSession.setPreferredSampleRate(sampleRateProvided) // at 48000.0
   try audioSession.setPreferredIOBufferDuration(preferredIOBufferDuration)

这些值可能与我们的首选大小不同。这就是为什么我们必须使用缓冲区逻辑来获取首选输入大小。

链接：https ://gist.github.com/hotpaw2/ba815fc23b5d642705f2b1dedfaf0107

renderBlock = auAudioUnit.renderBlock
    if (   enableRecording
        && micPermissionGranted
        && audioSetupComplete
        && audioSessionActive
        && isRecording == false ) {

        auAudioUnit.inputHandler = { (actionFlags, timestamp, frameCount, inputBusNumber) in
            if let block = self.renderBlock {       // AURenderBlock?
                var bufferList = AudioBufferList(
                    mNumberBuffers: 1,
                    mBuffers: AudioBuffer(
                        mNumberChannels: audioFormat!.channelCount,
                        mDataByteSize: 0,
                        mData: nil))

                let err : OSStatus = block(actionFlags,
                                           timestamp,
                                           frameCount,
                                           inputBusNumber,
                                           &bufferList,
                                           .none)
                if err == noErr {
                    // save samples from current input buffer to circular buffer
                    print("inputData = \(bufferList.mBuffers.mDataByteSize) and frameCount: \(frameCount) and count: \(count)")
                    count += 1
                    if !self.isMuteState {
                        self.recordMicrophoneInputSamples(
                            inputDataList:  &bufferList,
                            frameCount: UInt32(frameCount) )
                    }
                }
            }
        }
        auAudioUnit.isInputEnabled  = true
        auAudioUnit.outputProvider = { (    //  AURenderPullInputBlock?
            actionFlags,
            timestamp,
            frameCount,
            inputBusNumber,
            inputDataList ) -> AUAudioUnitStatus in
            if let block = self.renderBlock {
                if let  dataReceived = self.getInputDataForConsumption() {
                    let mutabledata = NSMutableData(data: dataReceived)

                    var bufferListSpeaker = AudioBufferList(
                        mNumberBuffers: 1,
                        mBuffers: AudioBuffer(
                            mNumberChannels: 1,
                            mDataByteSize: 0,
                            mData: nil))
                    let err : OSStatus = block(actionFlags,
                                               timestamp,
                                               frameCount,
                                               1,
                                               &bufferListSpeaker,
                                               .none)
                    if err == noErr {
                        bufferListSpeaker.mBuffers.mDataByteSize = UInt32(mutabledata.length)
                        bufferListSpeaker.mBuffers.mData = mutabledata.mutableBytes
                        inputDataList[0] = bufferListSpeaker
                        print("Output Provider mDataByteSize: \(inputDataList[0].mBuffers.mDataByteSize) output FrameCount: \(frameCount)")
                        return err
                    } else {
                        print("Output Provider \(err)")
                        return err
                    }
                }
            }
            return 0
        }
        auAudioUnit.isOutputEnabled  = true

        do {
            circInIdx   =   0                       // initialize circular buffer pointers
            circOutIdx  =   0
            circoutSpkIdx = 0
            circInSpkIdx = 0
            try auAudioUnit.allocateRenderResources()
            try auAudioUnit.startHardware()         // equivalent to AudioOutputUnitStart ???
            isRecording = true

        } catch let e {
            print(e)
        }

推荐阅读