时间戳

首页 > 解决方案 > 如何使用 YUV 颜色空间将 UIImage 转换为 CVPixelBufferRef？

ios - 如何使用 YUV 颜色空间将 UIImage 转换为 CVPixelBufferRef？

问题描述

我正在做视频录制。我需要将视图快照到 UIImage，然后将其转换为 CVPixelBufferRef。它适用于 RGBA 色彩空间。但是我需要的 CVPixelBufferRef 应该是 YUV 颜色空间。

有人有想法么？谢谢。

+ (CVPixelBufferRef) pixelBufferFromLayer:(CALayer *)layer forSize:(CGSize)size
{
    UIImage * image = [self fetchScreenShotFromLayer:layer forSize:size];

// this worked fine
//    CVPixelBufferRef rgbBuffer = [self RGBPixelBufferFromCGImage:image.CGImage];
//    return rgbBuffer;

//    NSData * imageData = UIImageJPEGRepresentation(image, 0.5);
    NSData * imageData = UIImagePNGRepresentation(image);
    CVPixelBufferRef buffer = [self yuvPixelBufferWithData:imageData width:size.width heigth:size.height];
    return buffer;
}

用 RGB 颜色空间创建 CVPixelBufferRef 很好。

// RGB color space pixel buffer
+ (CVPixelBufferRef) RGBPixelBufferFromCGImage:(CGImageRef)image
{
    NSDictionary * options = @{
                               (NSString *)kCVPixelBufferCGImageCompatibilityKey: @(YES),
                               (NSString *)kCVPixelBufferCGBitmapContextCompatibilityKey: @(YES),
                               };

    CVPixelBufferRef pxbuffer = NULL;

    CGFloat frameWidth = CGImageGetWidth(image);
    CGFloat frameHeight = CGImageGetHeight(image);

    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, frameWidth, frameHeight, kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef)options, &pxbuffer);

    NSParameterAssert(status == kCVReturnSuccess && pxbuffer != NULL);

    CVPixelBufferLockBaseAddress(pxbuffer, 0);
    void * pxdata = CVPixelBufferGetBaseAddress(pxbuffer);
    NSParameterAssert(pxdata != NULL);

    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(pxdata, frameWidth, frameHeight, 8, CVPixelBufferGetBytesPerRow(pxbuffer), rgbColorSpace, (CGBitmapInfo)kCGImageAlphaNoneSkipFirst);

    NSParameterAssert(context);
    CGContextConcatCTM(context, CGAffineTransformIdentity);
    CGContextDrawImage(context, CGRectMake(0, 0, frameWidth, frameHeight), image);
    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);

    CVPixelBufferUnlockBaseAddress(pxbuffer, 0);

    return pxbuffer;
}

// snapshoot for layer
+ (UIImage *) fetchScreenShotFromLayer:(CALayer *)layer forSize:(CGSize)size
{
    UIImage * image = nil;

    @autoreleasepool {
        NSLock * aLock = [NSLock new];
        [aLock lock];

        CGSize imageSize = size;
        UIGraphicsBeginImageContextWithOptions(imageSize, NO, 0);
        CGContextRef context = UIGraphicsGetCurrentContext();
        [layer renderInContext:context];
        image = UIGraphicsGetImageFromCurrentImageContext();
        UIGraphicsEndImageContext();

        [aLock unlock];
    }

    return image;
}

这有点不对劲。

// data to yuv buffer
+ (CVPixelBufferRef)yuvPixelBufferWithData:(NSData *)dataFrame
                                     width:(size_t)w
                                    heigth:(size_t)h
{
    unsigned char* buffer = (unsigned char*) dataFrame.bytes;
    CVPixelBufferRef getCroppedPixelBuffer = [self copyDataFromBuffer:buffer toYUVPixelBufferWithWidth:w Height:h];
    return getCroppedPixelBuffer;
}

+ (CVPixelBufferRef) copyDataFromBuffer:(const unsigned char*)buffer toYUVPixelBufferWithWidth:(size_t)w Height:(size_t)h
{
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
                             nil];

    CVPixelBufferRef pixelBuffer;
    CVPixelBufferCreate(NULL, w, h, kCVPixelFormatType_420YpCbCr8BiPlanarFullRange, (__bridge CFDictionaryRef)(options), &pixelBuffer);

    size_t count = CVPixelBufferGetPlaneCount(pixelBuffer);
    NSLog(@"PlaneCount = %zu", count);  // 2

    CVPixelBufferLockBaseAddress(pixelBuffer, 0);

    size_t d = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0);
    const unsigned char* src = buffer;
    unsigned char* dst = (unsigned char *)CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0);

    for (unsigned int rIdx = 0; rIdx < h; ++rIdx, dst += d, src += w) {
        memcpy(dst, src, w);
    }

    d = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 1);
    dst = (unsigned char *)CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 1);
    h = h >> 1;
    for (unsigned int rIdx = 0; rIdx < h; ++rIdx, dst += d, src += w) {
        memcpy(dst, src, w);
    }

    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

    return pixelBuffer;
}

这是图像。

原图

rgb 缓冲区

YUV 缓冲液

谢谢你的帮助。

标签： iosopengl-esuiimagegpuimagecvpixelbuffer

解决方案

您可以从 RGBA 格式的 UIImage 获取原始数据，然后转换为 yuv 格式，并使用 yuv 数据填充您从 CMSampleBufferRef 获得的 CVPixelBufferRef 的不同平面。这里的CMSampleBufferRef就是captureOutput的参数。请记住在初始化 AVCaptureSession 的 AVCaptureVideoDataOutput 时将视频设置设置为 kCVPixelFormatType_420YpCbCr8BiPlanarFullRange（如果这是您想要的。）。

推荐阅读