时间戳

首页 > 解决方案 > StartCoroutine() 修复 targetTexture.ReadPixels 错误

c# - StartCoroutine() 修复 targetTexture.ReadPixels 错误

问题描述

正如标题所示，我对行中发生的错误有疑问

targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);

错误：

调用 ReadPixels 以从系统帧缓冲区读取像素，而不是在绘图帧内。UnityEngine.Texture2D:ReadPixels(Rect, Int32, Int32)

正如我从其他帖子中了解到的那样，解决此问题的一种方法是创建一个 Ienumerator 方法，该方法产生返回新的 WaitForSeconds 或其他内容，并将其称为： StartCoroutine(methodname)以便及时加载帧，以便读取像素 -是的。

我没有得到的是在下面的代码中这种方法最有意义的地方。哪个部分没有及时加载？

    PhotoCapture photoCaptureObject = null;
    Texture2D targetTexture = null;
    public string path = "";
    CameraParameters cameraParameters = new CameraParameters();

private void Awake()
{

    var cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
    targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);

    // Create a PhotoCapture object
    PhotoCapture.CreateAsync(false, captureObject =>
    {
        photoCaptureObject = captureObject;
        cameraParameters.hologramOpacity = 0.0f;
        cameraParameters.cameraResolutionWidth = cameraResolution.width;
        cameraParameters.cameraResolutionHeight = cameraResolution.height;
        cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
    });
}

private void Update()
{
    // if not initialized yet don't take input
    if (photoCaptureObject == null) return;

    if (Input.GetKey("k") || Input.GetKey("k"))
    {
        Debug.Log("k was pressed");

        VuforiaBehaviour.Instance.gameObject.SetActive(false);

        // Activate the camera
        photoCaptureObject.StartPhotoModeAsync(cameraParameters, result =>
        {
            if (result.success)
            {
                // Take a picture
                photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
            }
            else
            {
                Debug.LogError("Couldn't start photo mode!", this);
            }
        });
    }
}

private static string FileName(int width, int height)
{
    return $"screen_{width}x{height}_{DateTime.Now:yyyy-MM-dd_HH-mm-ss}.png";
}

private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();

    targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
    targetTexture.Apply();

    byte[] bytes = targetTexture.EncodeToPNG();

    string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
    //save to folder under assets
    File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
    Debug.Log("The picture was uploaded");

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

private void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result)
{
    // Shutdown the photo capture resource
    VuforiaBehaviour.Instance.gameObject.SetActive(true);
    photoCaptureObject.Dispose();
    photoCaptureObject = null;


}

抱歉，如果这算作重复，例如。

编辑

当我达到这一点时，这个可能会很有用。

是不是我根本不需要这三行？

Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();

正如评论中所写，使用这三行与不使用这三行的区别在于保存的照片具有黑色背景+ AR-GUI。上面没有第二行代码的是一张带有 AR-GUI 的照片，但背景是我电脑网络摄像头的实时流。真的，我不想看到计算机网络摄像头，而是看到 HoloLens 看到的。

标签： c#unity3dhololens

解决方案

你的三行

Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();

targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();

对我没有多大意义。Texture2D.ReadPixels用于创建屏幕截图，以便您使用屏幕截图覆盖刚刚收到的纹理PhotoCapture？（由于相机分辨率很可能！=屏幕分辨率，因此尺寸也不正确。）

这也是原因

正如评论中所写，使用这三行与不使用这三行的区别在于保存的照片具有黑色背景+ AR-GUI。

做完之后

photoCaptureFrame.UploadImageDataToTexture(targetTexture);

您已经Texture2D从PhotoCapture.targetTexture

我想您可能会将其与Texture2D.GetPixels混淆，后者用于获取给定的像素数据Texture2D。

我想最后从中心裁剪拍摄的照片，我想也许这可以用这个代码行？在 0, 0 以外的其他像素处开始新的矩形）

Texture2D正如您在评论中提到的那样，您真正想要的是裁剪从中心收到的内容。您可以使用GetPixels(int x, int y, int blockWidth, int blockHeight, int miplevel)来做到这一点，它用于切出给定区域的某个区域Texture2D

public static Texture2D CropAroundCenter(Texture2D input, Vector2Int newSize)
{
    if(input.width < newSize.x || input.height < newSize.y)
    {
        Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!", this);
        return null;
    }

    // get the pixel coordinate of the center of the input texture
    var center = new Vector2Int(input.width / 2, input.height / 2);

    // Get pixels around center
    // Get Pixels starts width 0,0 in the bottom left corner
    // so as the name says, center.x,center.y would get the pixel in the center
    // we want to start getting pixels from center - half of the newSize 
    //
    // than from starting there we want to read newSize pixels in both dimensions
    var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);

    // Create a new texture with newSize
    var output = new Texture2D(newSize.x, newSize.y);

    output.SetPixels(pixels);
    output.Apply();

    return output;
}

为了（希望）更好地理解，这说明GetPixels了给定值的重载在这里做了什么：

在此处输入图像描述

而不是使用它

private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // for example take only half of the textures width and height
    targetTexture = CropAroundCenter(targetTexture, new Vector2Int(targetTexture.width / 2, targetTexture.height / 2);

    byte[] bytes = targetTexture.EncodeToPNG();

    string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
    //save to folder under assets
    File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
    Debug.Log("The picture was uploaded");

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

或者你可以把它作为一个扩展方法，static class比如

public static class Texture2DExtensions
{
    public static void CropAroundCenter(this Texture2D input, Vector2Int newSize)
    {
        if (input.width < newSize.x || input.height < newSize.y)
        {
            Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!");
            return;
        }

        // get the pixel coordinate of the center of the input texture
        var center = new Vector2Int(input.width / 2, input.height / 2);

        // Get pixels around center
        // Get Pixels starts width 0,0 in the bottom left corner
        // so as the name says, center.x,center.y would get the pixel in the center
        // we want to start getting pixels from center - half of the newSize 
        //
        // than from starting there we want to read newSize pixels in both dimensions
        var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);

        // Resize the texture (creating a new one didn't work)
        input.Resize(newSize.x, newSize.y);

        input.SetPixels(pixels);
        input.Apply(true);
    }
}

而是使用它

targetTexture.CropAroundCenter(new Vector2Int(targetTexture.width / 2, targetTexture.height / 2));

笔记：

UploadImageDataToTexture：如果您在 CameraParameters 中指定了BGRA32格式，则只能使用此方法。

幸运的是，无论如何你都使用它；）

请记住，此操作将发生在主线程上，因此速度很慢。

然而，唯一的选择是CopyRawImageDataIntoBuffer并在另一个线程或外部生成纹理，所以我会说留下来是可以的UploadImageDataToTexture；）

和

捕获的图像也会在 HoloLens 上翻转。您可以使用自定义着色器重新定向图像。

通过翻转，它们实际上意味着Y-Axis纹理是颠倒的。 X-Axis是正确的。

要垂直翻转纹理，您可以使用第二种扩展方法：

public static class Texture2DExtensions
{
    public static void CropAroundCenter(){....}

    public static void FlipVertically(this Texture2D texture)
    {
        var pixels = texture.GetPixels();
        var flippedPixels = new Color[pixels.Length];

        // These for loops are for running through each individual pixel and 
        // write them with inverted Y coordinates into the flippedPixels
        for (var x = 0; x < texture.width; x++)
        {
            for (var y = 0; y < texture.height; y++)
            {
                var pixelIndex = x + y * texture.width;
                var flippedIndex = x  + (texture.height - 1 - y) * texture.width;

                flippedPixels[flippedIndex] = pixels[pixelIndex];
            }
        }

        texture.SetPixels(flippedPixels);
        texture.Apply();
    }
}

并像使用它一样

targetTexture.FlipVertically();

结果：（对于这个示例和给定的纹理，我使用了 FlipVertically 并每秒裁剪到一半大小，但对于拍摄的照片应该同样有效。）

在此处输入图像描述

图片来源：http: //developer.vuforia.com/sites/default/files/sample-apps/targets/imagetargets_targets.pdf

更新

对于您的按钮问题：

不要使用

if (Input.GetKey("k") || Input.GetKey("k"))

首先，您要检查完全相同的条件两次。而不是在按键保持按下状态时GetKey触发每一帧。而是使用

if (Input.GetKeyDown("k"))

只触发一次。我猜 Vuforia 和 PhotoCapture 存在问题，因为您的原始版本经常触发，也许您有一些并发的 PhotoCapture 进程......

推荐阅读