时间戳

首页 > 解决方案 > 如何将 CMSampleBufferRef/CIImage/UIImage 转换为像素，例如 uint8_t[]

c++ - 如何将 CMSampleBufferRef/CIImage/UIImage 转换为像素，例如 uint8_t[]

问题描述

我已经从捕获的相机帧中输入，CMSampleBufferRef并且我需要最好以 C 类型获取原始像素uint8_t[]。我还需要找到输入图像的配色方案。

我知道如何转换为CMSampleBufferRefpng格式，但我没有CMSampleBufferRef/CIImage`？UIImageNSDatat know how to get the raw pixels from there. Perhaps I could get it already from

此代码显示了需要和缺少的位。任何想法从哪里开始？

int convertCMSampleBufferToPixelArray (CMSampleBufferRef sampleBuffer)
{
    // inputs
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:imageBuffer];
    CIContext *imgContext = [CIContext new];
    CGImageRef cgImage = [imgContext createCGImage:ciImage fromRect:ciImage.extent];
    UIImage *uiImage = [UIImage imageWithCGImage:cgImage];
    NSData *nsData = UIImagePNGRepresentation(uiImage);
    

    // Need to fill this gap
    uint8_t* data = XXXXXXXXXXXXXXXX;
    ImageFormat format = XXXXXXXXXXXXXXXX; // one of: GRAY8, RGB_888, YV12, BGRA_8888, ARGB_8888


    // sample showing expected data values
    // this routine converts the image data to gray
    //
    int width = uiImage.size.width;
    int height = uiImage.size.height;
    const int size = width * height;
    std::unique_ptr<uint8_t[]> new_data(new uint8_t[size]);
    for (int i = 0; i < size; ++i) {
        new_data[i] = uint8_t(data[i * 3] * 0.299f + data[i * 3 + 1] * 0.587f +
                          data[i * 3 + 2] * 0.114f + 0.5f);
    }

    return 1;
}

标签： c++objective-cimagec++11avfoundation

解决方案

您可以使用一些指针来搜索更多信息。它有很好的记录，你不应该有问题。

int convertCMSampleBufferToPixelArray (CMSampleBufferRef sampleBuffer) {
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    if (imageBuffer == NULL) {
        return -1;
    }
    
    // Get address of the image buffer
    CVPixelBufferLockBaseAddress(imageBuffer, 0);
    uint8_t* data = CVPixelBufferGetBaseAddress(imageBuffer);
    
    // Get size
    size_t width = CVPixelBufferGetWidth(imageBuffer);
    size_t height = CVPixelBufferGetHeight(imageBuffer);

    // Get bytes per row
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);

    // At `data` you have a bytesPerRow * height bytes of the image data
    
    // To get pixel info you can call CVPixelBufferGetPixelFormatType, ...
    // you can call CVImageBufferGetColorSpace and inspect it, ...
    
    // When you're done, unlock the base address
    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
    
    return 0;
}

您应该注意几件事。

第一个是它可以是平面的。检查CVPixelBufferIsPlanar, CVPixelBufferGetPlaneCount,CVPixelBufferGetBytesPerRowOfPlane等。

第二个是您必须根据CVPixelBufferGetPixelFormatType. 就像是：

CVPixelBufferGetPixelFormatType(imageBuffer)

size_t pixelSize;
switch (pixelFormat) {
    case kCVPixelFormatType_32BGRA:
    case kCVPixelFormatType_32ARGB:
    case kCVPixelFormatType_32ABGR:
    case kCVPixelFormatType_32RGBA:
        pixelSize = 4;
        break;
    // + other cases
}

假设缓冲区不是平面的，并且：

  • CVPixelBufferGetWidth返回 200（像素）
  • 您的 pixelSize 为 4（每行的计算字节数为 200 * 4 = 800）
  • CVPixelBufferGetBytesPerRow可以返回任何东西>= 800

换句话说，您拥有的指针不是指向连续缓冲区的指针。如果您需要行数据，则必须执行以下操作：

uint8_t* data = CVPixelBufferGetBaseAddress(imageBuffer);

// Get size
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);

size_t pixelSize = 4; // Let's pretend it's calculated pixel size
size_t realRowSize = width * pixelSize;

size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);

for (int row = 0 ; row < height ; row++) {
    // bytesPerRow acts like an offset where the next row starts
    // bytesPerRow can be >= realRowSize
    uint8_t *rowData = data + row * bytesPerRow;
    
    // realRowSize = how many bytes are available for this row
    // copy them somewhere
}

如果您想拥有连续的缓冲区，则必须分配一个缓冲区并将这些行数据复制到那里。要分配多少字节？CVPixelBufferGetDataSize.

推荐阅读