ios - CMSampleBuffer 每次自动旋转

问题描述

在这里，我尝试从照片库中选择视频并逐帧读取，samplebuffer以便稍后裁剪或旋转。但问题是CMSampleBuffer默认旋转。我用于初始化的变量是

 var asset:AVAsset!     //load asset from url
 var assetReader:AVAssetReader!   
 var assetVideoOutput: AVAssetReaderTrackOutput!  // add assetreader for video
 var assetAudioOutput: AVAssetReaderTrackOutput!  // add assetreader for audio
 var readQueue: DispatchQueue!

以前变量的设置是这样的。

func resetRendering() {

    do{
        assetReader = try! AVAssetReader(asset: asset)
    }catch {
        print(error)
    }

    var tracks = asset.tracks(withMediaType: .video)
    var track : AVAssetTrack!
    if tracks.count > 0 {
        track = tracks[0]
    }

    let decompressionvideoSettings:[String:Any] = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]

    assetVideoOutput = AVAssetReaderTrackOutput(track: track, outputSettings: decompressionvideoSettings)

    tracks = asset.tracks(withMediaType: .audio)
    if tracks.count > 0 {
        track = tracks[0]
    }

    let audioReadSettings = [AVFormatIDKey: kAudioFormatLinearPCM]
    assetAudioOutput = AVAssetReaderTrackOutput(track: track, outputSettings: audioReadSettings)

    if (assetAudioOutput.responds(to: #selector(getter: AVAssetReaderOutput.alwaysCopiesSampleData))) {
        assetAudioOutput.alwaysCopiesSampleData = false
    }
    if assetReader.canAdd(assetAudioOutput) {
        assetReader.add(assetAudioOutput)
    }

    if (assetVideoOutput.responds(to: #selector(getter: AVAssetReaderOutput.alwaysCopiesSampleData))) {
        assetVideoOutput.alwaysCopiesSampleData = false
    }
    if assetReader.canAdd(assetVideoOutput) {
        assetReader.add(assetVideoOutput)
    }

}

现在，当我尝试逐帧读取视频并转换为图像时，它会自动将帧旋转 90 度。从samplebufferto的转换扩展uiimage看起来像这样

extension CMSampleBuffer {
    var uiImage: UIImage? {
        guard let imageBuffer = CMSampleBufferGetImageBuffer(self) else { return nil }

        CVPixelBufferLockBaseAddress(imageBuffer, CVPixelBufferLockFlags(rawValue: 0))
        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
        let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer)
        let width = CVPixelBufferGetWidth(imageBuffer)
        let height = CVPixelBufferGetHeight(imageBuffer)
        let colorSpace = CGColorSpaceCreateDeviceRGB()
        let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue)
        guard let context = CGContext(data: baseAddress,
                                      width: width,
                                      height: height,
                                      bitsPerComponent: 8,
                                      bytesPerRow: bytesPerRow,
                                      space: colorSpace,
                                      bitmapInfo: bitmapInfo.rawValue) else { return nil }
        guard let cgImage = context.makeImage() else { return nil }

        CVPixelBufferUnlockBaseAddress(imageBuffer,CVPixelBufferLockFlags(rawValue: 0));

        return UIImage(cgImage: cgImage)
    }
}

结果照片是

实际照片是这张

标签： iosswiftavfoundation

更新

CGImage 不持有方向属性。所以转换成UIImage的时候要手动设置。我们从视频轨道获取方向并将其设置为 UIImage。

下面的答案的右/左或上/下可能是相反的。请尝试检查。

//pls get orientation property outside of image processing to avoid redundancy
let videoTrack = videoAsset.tracks(withMediaType: AVMediaType.video).first!
let videoSize = videoTrack.naturalSize
let transform = videoTrack.preferredTransform

var orientation = UIImage.Orientation.right
switch (transform.tx, transform.ty) {
case (0, 0): orientation = UIImage.Orientation.up
case (videoSize.width, videoSize.height): UIImage.Orientation.right
case (0, videoSize.width): UIImage.Orientation.left
default: orientation = UIImage.Orientation.down
}


// then set orientation in image processing such as captureOutput
let uiImage = UIImage.init(cgImage: cgImage, scale: 1.0, orientation: orientation)

希望这可以帮助。

ios - CMSampleBuffer 每次自动旋转

问题描述

解决方案

推荐阅读