ios - 如何在渲染回调中交错非交错的 AudioBufferList?
问题描述
我正在从事一个项目,该项目涉及将音频从AVPlayer
视频播放器对象流式传输到libpd
使用MTAudioProcessingTap
. 对于tap的进程循环,我使用PdAudioUnit
了srender回调代码作为指导;但我最近意识到,预期的音频格式libpd
与来自 tap 的音频不同——也就是说,tap 在传入的 AudioBufferList 中提供两个非交错音频数据的缓冲区,而 libpd 需要交错样本。我不认为我可以改变抽头本身来提供交错样本。
有谁知道我可以解决这个问题的方法?
我认为我需要以某种方式创建一个新的AudioBufferList
或浮动缓冲区并将样本交错到位;但我不太确定如何做到这一点,而且看起来会很贵。如果有人能给我一些指示,我将不胜感激!
这是我安装水龙头的代码:
- (void)installTapWithItem:(AVPlayerItem *)playerItem {
MTAudioProcessingTapCallbacks callbacks;
callbacks.version = kMTAudioProcessingTapCallbacksVersion_0;
callbacks.clientInfo = (__bridge void *)self;
callbacks.init = tap_InitCallback;
callbacks.finalize = tap_FinalizeCallback;
callbacks.prepare = tap_PrepareCallback;
callbacks.unprepare = tap_UnprepareCallback;
callbacks.process = tap_ProcessCallback;
MTAudioProcessingTapRef audioProcessingTap;
if (noErr == MTAudioProcessingTapCreate(kCFAllocatorDefault, &callbacks, kMTAudioProcessingTapCreationFlag_PreEffects, &audioProcessingTap))
{
NSLog(@"Tap created!");
AVAssetTrack *audioTrack = [playerItem.asset tracksWithMediaType:AVMediaTypeAudio].firstObject;
AVMutableAudioMixInputParameters* inputParams = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:audioTrack];
inputParams.audioTapProcessor = audioProcessingTap;
AVMutableAudioMix* audioMix = [AVMutableAudioMix audioMix];
audioMix.inputParameters = @[inputParams];
playerItem.audioMix = audioMix;
}
}
我的tap_ProcessCallback
:
static void tap_ProcessCallback(MTAudioProcessingTapRef tap, CMItemCount numberFrames, MTAudioProcessingTapFlags flags, AudioBufferList *bufferListInOut, CMItemCount *numberFramesOut, MTAudioProcessingTapFlags *flagsOut)
{
OSStatus status = MTAudioProcessingTapGetSourceAudio(tap, numberFrames, bufferListInOut, flagsOut, nil, numberFramesOut);
if (noErr != status) {
NSLog(@"Error: MTAudioProcessingTapGetSourceAudio: %d", (int)status);
return;
}
TapProcessorContext *context = (TapProcessorContext *)MTAudioProcessingTapGetStorage(tap);
// first, create the input and output ring buffers if they haven't been created yet
if (context->frameSize != numberFrames) {
NSLog(@"creating ring buffers with size: %ld", (long)numberFrames);
createRingBuffers((UInt32)numberFrames, context);
}
//adapted from PdAudioUnit.m
float *buffer = (float *)bufferListInOut->mBuffers->mData;
if (context->inputRingBuffer || context->outputRingBuffer) {
// output buffer info from ioData
UInt32 outputBufferSize = bufferListInOut->mBuffers[0].mDataByteSize;
UInt32 outputFrames = (UInt32)numberFrames;
// UInt32 outputChannels = bufferListInOut->mBuffers[0].mNumberChannels;
// input buffer info from ioData *after* rendering input samples
UInt32 inputBufferSize = outputBufferSize;
UInt32 inputFrames = (UInt32)numberFrames;
// UInt32 inputChannels = 0;
UInt32 framesAvailable = (UInt32)rb_available_to_read(context->inputRingBuffer) / context->inputFrameSize;
while (inputFrames + framesAvailable < outputFrames) {
// pad input buffer to make sure we have enough blocks to fill auBuffer,
// this should hopefully only happen when the audio unit is started
rb_write_value_to_buffer(context->inputRingBuffer, 0, context->inputBlockSize);
framesAvailable += context->blockFrames;
}
rb_write_to_buffer(context->inputRingBuffer, 1, buffer, inputBufferSize);
// input ring buffer -> context -> output ring buffer
char *copy = (char *)buffer;
while (rb_available_to_read(context->outputRingBuffer) < outputBufferSize) {
rb_read_from_buffer(context->inputRingBuffer, copy, context->inputBlockSize);
[PdBase processFloatWithInputBuffer:(float *)copy outputBuffer:(float *)copy ticks:1];
rb_write_to_buffer(context->outputRingBuffer, 1, copy, context->outputBlockSize);
}
// output ring buffer -> audio unit
rb_read_from_buffer(context->outputRingBuffer, (char *)buffer, outputBufferSize);
}
}
解决方案
回答我自己的问题...
我不确定为什么会这样,但确实如此。显然我也不需要使用环形缓冲区,这很奇怪。mNumberBuffers
当只有一个缓冲区时,我还添加了一个开关。
if (context->frameSize && outputBufferSize > 0) {
if (bufferListInOut->mNumberBuffers > 1) {
float *left = (float *)bufferListInOut->mBuffers[0].mData;
float *right = (float *)bufferListInOut->mBuffers[1].mData;
//manually interleave channels
for (int i = 0; i < outputBufferSize; i += 2) {
context->interleaved[i] = left[i / 2];
context->interleaved[i + 1] = right[i / 2];
}
[PdBase processFloatWithInputBuffer:context->interleaved outputBuffer:context->interleaved ticks:64];
//de-interleave
for (int i = 0; i < outputBufferSize; i += 2) {
left[i / 2] = context->interleaved[i];
right[i / 2] = context->interleaved[i + 1];
}
} else {
context->interleaved = (float *)bufferListInOut->mBuffers[0].mData;
[PdBase processFloatWithInputBuffer:context->interleaved outputBuffer:context->interleaved ticks:32];
}
}
推荐阅读
- swift - Swift 字符串插值产生的字符串需要很长时间才能插入字典
- python - 按下按钮后重置日期输入框并重新选择 tkinter 中的所有复选按钮
- html - 如何指定将 GET 请求的值放在 HTML 表单中的位置?
- regex - 如何拥有一个匹配 <> 内所有内容的正则表达式,除了
? - html - 我正在尝试如下连接 innerTextvalue 但不工作
- angular - Angular单个组件不接收数据
- excel - 使用 VBA 宏将文本中的数字存储在数组中并对其进行测试
- javascript - 将`<>`和`作为相邻JSX元素的封闭元素是否正常?我可以使用``和``相反?
- r - 如何在 ggplot2 中添加灰度调色板?
- javascript - 未捕获的类型错误:无法在“IntersectionObserver”上执行“观察”:参数 1 不是“元素”类型