时间戳

首页 > 解决方案 > 从 ARCamera 设置 VNImageOptionCameraIntrinsics

ios - 从 ARCamera 设置 VNImageOptionCameraIntrinsics

问题描述

我正在构建一个将 ARKit 与 CoreML 相结合的应用程序。我VNImageRequestHandler使用以下几行将帧传递给：

// the frame of the current Scene
CVPixelBufferRef pixelBuffer = _cameraPreview.session.currentFrame.capturedImage;

NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
VNImageRequestHandler *handler = [[VNImageRequestHandler alloc] initWithCVPixelBuffer:pixelBuffer options:requestOptions];

注意requestOptions. 它应该包含将VNImageOptionCameraIntrinsics相机内在函数传递给 CoreML 的字段。

在使用 ARKit 之前，我使用 aCMSampleBufferRef从相机获取图像。可以使用以下方法检索和设置内在函数：

CFTypeRef cameraIntrinsicData = CMGetAttachment(sampleBuffer, kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, nil);
requestOptions[VNImageOptionCameraIntrinsics] = (__bridge id)(cameraIntrinsicData);

但是，我现在使用的是ARFrame，但我仍然想设置正确的内在函数，因为pixelBuffer旋转了。

查看文档：

https://developer.apple.com/documentation/vision/vnimageoption?language=objc

https://developer.apple.com/documentation/arkit/arcamera/2875730-intrinsics?language=objc

我们可以看到它也ARCamera提供了内在函数，但是，我如何requestOptions正确设置这个值？

到目前为止，它应该是这样的：

ARCamera *camera = _cameraPreview.session.currentFrame.camera;
NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];
// How to put camera.intrinsics here?
requestOptions[VNImageOptionCameraIntrinsics] = camera.intrinsics;

标签： iosobjective-carkitcoreml

解决方案

正如Giovanni在评论中提到的，转换UIDeviceOrientation为CGImagePropertyOrientation避免使用VNImageOptionCameraIntrinsics：

实用程序

+(CGImagePropertyOrientation) getOrientation {
    CGImagePropertyOrientation orientation;
    UIDeviceOrientation deviceOrientation = [[UIDevice currentDevice] orientation];
    switch (deviceOrientation) {
        case UIDeviceOrientationPortrait:
            orientation = kCGImagePropertyOrientationRight;
            break;
        case UIDeviceOrientationPortraitUpsideDown:
            orientation = kCGImagePropertyOrientationLeft;
            break;
        case UIDeviceOrientationLandscapeLeft:
            orientation = kCGImagePropertyOrientationUp;
            break;
        case UIDeviceOrientationLandscapeRight:
            orientation = kCGImagePropertyOrientationDown;
            break;
        default:
            orientation = kCGImagePropertyOrientationRight;
            break;
    }
    return orientation;
}

视图控制器.mm

- (void)captureOutput {
    ARFrame *frame = self.cameraPreview.session.currentFrame;
    CVPixelBufferRef pixelBuffer = frame.capturedImage;

    CGImagePropertyOrientation deviceOrientation = [Utils getOrientation];
    NSMutableDictionary<VNImageOption, id> *requestOptions = [NSMutableDictionary dictionary];

    VNImageRequestHandler *handler = [[VNImageRequestHandler alloc] initWithCVPixelBuffer:pixelBuffer orientation:deviceOrientation options:requestOptions];

    [handler performRequests:@[[self request]] error:nil];
}

推荐阅读