ios - 如何为 YCbCr 图像创建 vImage

问题描述

我想创建一个库，可以将使用AVCaptureDevice捕获的任何视频帧转换为目标像素格式。因此，它必须支持kCVPixelFormatType_32BGRA,kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange和kCVPixelFormatType_420YpCbCr8BiPlanarFullRange可选kCVPixelFormatType_420YpCbCr8Planar的。

为了提高性能，我想使用 Accelerate 框架，因为它包含丰富的转换函数集。由于目标像素格式可能不同并且由库用户设置，因此使用通用vImageConvert_AnyToAny函数会很高兴：

您使用 vImage 的 vImageConvert_AnyToAny( : : : :_:) 函数在任意颜色空间和位深度的 Core Video 或 Core Graphics 图像数据之间进行转换。源图像和目标图像由一个或多个缓冲区描述。例如，Y'CbCr图像可以由一个包含亮度信息的缓冲器和一个包含色度信息的缓冲器组成。

要使用此功能，我必须创建定义图像之间转换的vImageConverter 。此类的构造函数需要vImage_CGImageFormat形式的源和目标图像格式描述：

vImage_CGImageFormat in_format = ...;
vImage_CGImageFormat out_format = ...; 
vImageConverterRef converter = vImageConverter_CreateWithCGImageFormat(&in_format, &out_format, NULL, kvImagePrintDiagnosticsToConsole, &err);

if( err == kvImageNoError )
{
    vImage_Buffer *src_planes = ...;
    vImage_Buffer *dst_planes = ...;

    err = vImageConvert_AnyToAny(converter, src_planes, dst_planes, NULL,  kvImagePrintDiagnosticsToConsole);
}

该代码基于这篇 Apple 文章：构建基本转换工作流程

对于kCVPixelFormatType_32BGRA这样的 vImage_CGImageFormat 很简单，并在vImage_Utilities.h中进行了描述：

/*!
 * @struct vImage_CGImageFormat
 * @abstract A pixel format
 * @discussion A vImage_CGImageFormat describes the ordering of the color channels, how many there are,
 * the size and type of the data in the color channels and whether the data is premultiplied by alpha or not.
 * This format mirrors the image format descriptors used by CoreGraphics to create things like CGImageRef and
 * CGBitmapContextRef.
 *
 * This vImage_CGImageFormat:
 *
 *  <pre>@textblock
 *      vImage_CGImageFormat format = {
 *          .bitsPerComponent = 8,
 *          .bitsPerPixel = 32,
 *          .colorSpace = CGColorSpaceCreateDeviceRGB(),                                    // don't forget to release this!
 *          .bitmapInfo = kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little,
 *          .version = 0,                                                                   // must be 0
 *          .decode = NULL,
 *          .renderingIntent = kCGRenderingIntentDefault
 *      };
 *  @/textblock</pre>
 *
 * codes for a little endian ARGB8888 pixel, or what is called in the rest of vImage, BGRA8888. Note: for 16-
 * and 32-bits per component formats (int16_t, uint16_t, half-float, float) most vImage image filters assume
 * the data is in host-endian format. (The APIs in this header do not.) Host-endian is little endian for Intel
 * and ARM, big endian for PowerPC. If the data is not in host-endian format, then you may use
 * vImagePermuteChannels_ARGB8888 or vImageByteSwap_Planar16U to swap the image data byte ordering.
 *
 * Some examples:
 *  <pre>@textblock
 *      ARGB8888     ->  {8, 32, NULL, alpha first, 0, NULL, kCGRenderingIntentDefault}     alpha first = { kCGImageAlphaFirst, kCGImageAlphaPremultipliedFirst, kCGImageAlphaNoneSkipFirst }
 *      RGBA8888     ->  {8, 32, NULL, alpha last,  0, NULL, kCGRenderingIntentDefault}     alpha last  = { kCGImageAlphaLast,  kCGImageAlphaPremultipliedLast,  kCGImageAlphaNoneSkipLast }
 *      BGRA8888     ->  {8, 32, NULL, alpha first | kCGBitmapByteOrder32Little, 0, NULL, kCGRenderingIntentDefault}
 *      RGB888       ->  {8, 24, NULL, kCGImageAlphaNone | kCGBitmapByteOrderDefault, 0, NULL, kCGRenderingIntentDefault}
 *      RGB565       ->  {5, 16, NULL, kCGImageAlphaNone | kCGBitmapByteOrder16Little, 0, NULL, kCGRenderingIntentDefault}
 *      ARGB1555     ->  {5, 16, NULL, alpha first | kCGBitmapByteOrder16Little, 0, NULL, kCGRenderingIntentDefault}
 *      RGBA16F      ->  {16, 64, NULL, alpha last | kCGBitmapFloatComponents | kCGBitmapByteOrder16Little, 0, NULL, kCGRenderingIntentDefault }
 *      CMYK8888     ->  {8, 32, CGColorSpaceCreateDeviceCMYK(), kCGImageAlphaNone, 0, NULL, kCGRenderingIntentDefault  }
 *      ARGBFFFF premultiplied    ->  {32, 128, NULL, kCGImageAlphaPremultipliedFirst | kCGBitmapFloatComponents | kCGBitmapByteOrder32Little, 0, NULL, kCGRenderingIntentDefault }
 *      ARGBFFFF not-premultiplied -> {32, 128, NULL, kCGImageAlphaFirst | kCGBitmapFloatComponents | kCGBitmapByteOrder32Little, 0, NULL, kCGRenderingIntentDefault }
 *      ARGBFFFF, alpha = 1 ->        {32, 128, NULL, kCGImageAlphaNoneSkipFirst | kCGBitmapFloatComponents | kCGBitmapByteOrder32Little, 0, NULL, kCGRenderingIntentDefault }
 *  @/textblock</pre>
 *
 *  Note that some of these formats, particularly RGB565 and 16F formats are supported by vImage but
 *  not necessarily CoreGraphics. They will be converted to a higher precision format as necessary by
 *  vImage in vImageCreateCGImageFromBuffer().
 *
 *  By C rules, uninitialized struct parameters are set to zero. The last three parameters are usually zero, so can usually be omitted.
 *
 *  <pre>@textblock
 *      vImage_CGImageFormat srgb888 = (vImage_CGImageFormat){
 *          .bitsPerComponent = 8,
 *          .bitsPerPixel = 24,
 *          .colorSpace = NULL,
 *          .bitmapInfo = kCGImageAlphaNone | kCGBitmapByteOrderDefault };
 *  @/textblock</pre>
 *
 * To understand how these various parameters relate to one another, we can look at the process of converting from
 * one vImage_CGImageFormat format to another:
 *
 *  1) transform endianness of src format given by bitmapInfo to host endian  (except 8 bitPerComponent content)
 *  2) remove decode array transformation, and up convert to a higher range format as necessary to preserve precision / range
 *  3) convert src colorspace to reference XYZ colorspace (may cause upconvert to preserve range / precision)
 *  4) convert XYZ to destination colorspace + rendering intent
 *  5) convert to destination precision (given by bitsPerComponent)
 *  6) deal with any alpha changes (given by bitmapInfo) or flattening that needs to occur
 *  7) Apply any channel reordering requested, if it didn't happen at an earlier step. (As indicated by src and dest bitmapInfo)
 *  8) Apply destination decode array
 *  9) Apply endianness transform given by dest bitmapInfo
 *
 * Clearly, for most common transformations not all steps need to occur and multiple steps can be collapsed into a compound operation.
 *
 *  @field  bitsPerComponent    The number of bits needed to represent one channel of data in one pixel. For ARGB8888, this would be 8. Expected values: {1, 2, 4, 5, 8, 10, 12, 16, 32}
 *  @field  bitsPerPixel        The number of bits needed to represent one pixel. For ARGB8888, this would be 32.
 *                              It is possible that bitsPerPixel > bitsPerComponent * number of components, but in practice this is rare.
 *                              The number of color components is given by the colorspace and the number of alpha components (0 or 1) is given by
 *                              by the bitmapInfo.
 *  @field  colorSpace          A description of how the pixel data in the image is positioned relative to a reference XYZ color space.
 *                                  See CoreGraphics/CGColorSpace.h.  Pass NULL as a shorthand for sRGB. The vImage_CGImageFormat is not
 *                                  capable of managing the memory held by the colorSpace. If you created the colorspace, you must
 *                                  be sure to release it before all references to it disappear from scope.
 *  @field  bitmapInfo          The CGBitmapInfo describing the color channels. See CoreGraphics/CGImage.h.
 *                                  ARGB8888 is kCGImageAlphaFirst | kCGBitmapByteOrderDefault
 *                                  BGRA8888 is kCGImageAlphaFirst | kCGBitmapByteOrder32Little
 *  @field  version             The struct is versioned for future expansion.  Pass 0 here.
 *  @field  decode              Prior to transformations caused by the colorspace, color channels are subject to a linear transformation.
 *                              This allows for a different range than the typical [0,1.0]. NULL indicates default behavior of [0,1.0]
 *                              range, and is what you should use if you don't understand this parameter. See description of CGImageCreate()
 *                              for a discussion of decode arrays. See also Decode Arrays section of Chapter 4.8 of the PDF specification.
 *                              The vImage_CGImageFormat is not capable of managing the memory held by the decode array. If you created a
 *                              decode array on the heap, you must be sure to release it before all references to it disappear from scope.
 *
 *  @field renderingIntent      See CGColorSpace.h. kCGRenderingIntentDefault is typical here. By convention, rendering intent changes that
 *                              are not accompanied by a colorspace change are ignored.
 */

我不明白如何为 YCbCr 像素格式创建 vImage_CGImageFormat。

首先，我认为它根本不受支持，但这种格式有特殊功能：Understanding YpCbCr Image Formats and vImageConverter 对多平面图像具有特殊功能，例如vImageConverter_GetNumberOfSourceBuffers和vImageConverter_GetSourceBufferOrder。在最后一个函数中，有许多vImage 缓冲区类型代码，甚至kvImageBufferTypeCode_CVPixelBuffer_YCbCr.

所以，看起来可以为 YCbCr 创建 vImageConverter，我需要帮助来了解如何去做。

标签： iosaccelerate-frameworkvimage

ios - 如何为 YCbCr 图像创建 vImage_CGImageFormat？

问题描述

解决方案

推荐阅读