java - 流屏幕使用 MediaRecorder 做出原生反应
问题描述
我找不到任何包来使用本机反应从应用程序中流式传输用户正在执行的操作。
我正在创建自己的插件
目标是将流从 java 发送到 javascript，然后通过 socket.io 将其发送到外部套接字服务器
这是我要求授权和流式传输的 java 类
import android.hardware.display.DisplayManager;
import android.util.DisplayMetrics;
import android.app.Activity;
import android.content.Intent;
import android.media.projection.MediaProjectionManager;
import android.media.projection.MediaProjection;
import android.media.MediaRecorder;
import android.util.Log;
import com.facebook.react.bridge.ReactApplicationContext;
import com.facebook.react.bridge.ReactContextBaseJavaModule;
import com.facebook.react.bridge.ReactMethod;
import com.facebook.react.bridge.Promise;
import com.facebook.react.bridge.ActivityEventListener;
import com.facebook.react.bridge.BaseActivityEventListener;
import static android.content.Context.MEDIA_PROJECTION_SERVICE;
public class ScreenMirroringModule extends ReactContextBaseJavaModule {
private static final int SCREEN_SHARE_REQUEST = 4242;
private static final String S_MIRROR_CANCELLED = "S_MIRROR_CANCELLED";
private Promise sMirrorPromise;
private MediaProjectionManager mMediaProjectionManager;
private MediaStreamer mMediaStreamer = new MediaStreamer();
private final ActivityEventListener mActivityEventListener = new BaseActivityEventListener() {
@Override
public void onActivityResult(Activity activity, int requestCode, int resultCode, Intent intent) {
if (requestCode == SCREEN_SHARE_REQUEST) {
if (sMirrorPromise != null) {
if (resultCode == Activity.RESULT_CANCELED) {
sMirrorPromise.reject(S_MIRROR_CANCELLED, "Screen mirroring was cancelled");
} else if (resultCode == Activity.RESULT_OK) {
DisplayMetrics dm = new DisplayMetrics();
activity.getWindowManager().getDefaultDisplay().getMetrics(dm);
mMediaStreamer.reset();
mMediaStreamer.setVideoSource(MediaRecorder.VideoSource.SURFACE);
mMediaStreamer.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
mMediaStreamer.setVideoEncoder(MediaRecorder.VideoEncoder.MPEG_4_SP);
mMediaStreamer.setVideoSize(dm.widthPixels, dm.heightPixels);
mMediaStreamer.setVideoFrameRate(30);
StreamTask st = new StreamTask();
st.execute(mMediaStreamer);
try {
st.get();
} catch (Exception e) {
sMirrorPromise.reject(e);
}
MediaProjection mMediaProjection = mMediaProjectionManager.getMediaProjection(resultCode, intent);
try {
mMediaProjection.createVirtualDisplay("MainActivity",
dm.widthPixels, dm.heightPixels, dm.densityDpi,
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
mMediaStreamer.getSurface(), null /*Callbacks*/, null/*Handler*/);
mMediaStreamer.start();
} catch (Exception e) {
//Log.v("ReactNative", Log.getStackTraceString(e));
sMirrorPromise.reject(e);
}
sMirrorPromise.resolve("Test");
}
}
sMirrorPromise = null;
}
}
};
public ScreenMirroringModule(ReactApplicationContext reactContext) {
super(reactContext);
// Add the listener for `onActivityResult`
reactContext.addActivityEventListener(mActivityEventListener);
}
@Override
public String getName() {
return "ScreenMirroring";
}
@ReactMethod
public void stream(String ip, int port, Promise promise) {
final Activity activity = getCurrentActivity();
// Store the promise to resolve/reject when picker returns data
sMirrorPromise = promise;
mMediaProjectionManager = (MediaProjectionManager) activity.getSystemService(MEDIA_PROJECTION_SERVICE);
Intent intent = mMediaProjectionManager.createScreenCaptureIntent();
activity.startActivityForResult(intent, SCREEN_SHARE_REQUEST);
}
}
这是我的课，正在将录音转换为流
package com.ijkoareactapp;
import android.media.MediaRecorder;
import android.net.LocalServerSocket;
import android.net.LocalSocket;
import android.net.LocalSocketAddress;
import android.util.Log;
import java.io.IOException;
import java.io.InputStream;
public class MediaStreamer extends MediaRecorder {
private LocalServerSocket localServerSocket = null;
private LocalSocket receiver, sender = null;
public void prepare() throws IllegalStateException, IOException {
receiver = new LocalSocket();
try {
localServerSocket = new LocalServerSocket("screen_mirror_socket");
receiver.connect(new LocalSocketAddress("screen_mirror_socket"));
receiver.setReceiveBufferSize(4096);
receiver.setSendBufferSize(4096);
sender = localServerSocket.accept();
sender.setReceiveBufferSize(4096);
sender.setSendBufferSize(4096);
} catch (Exception e) {
throw new IOException("Can't create local socket !");
}
setOutputFile(sender.getFileDescriptor());
try {
super.prepare();
} catch (Exception e) {
Log.v("ReactNative", Log.getStackTraceString(e));
closeSockets();
throw e;
}
}
public InputStream getInputStream() {
InputStream out = null;
try {
out = receiver.getInputStream();
} catch (IOException e) {
}
return out;
}
public void stop() {
closeSockets();
super.stop();
}
private void closeSockets() {
if (localServerSocket != null) {
try {
localServerSocket.close();
sender.close();
receiver.close();
} catch (IOException e) {
}
localServerSocket = null;
sender = null;
receiver = null;
}
}
}
这是在后台运行任务的类，因为我无法在主进程中启动本地套接字
package com.ijkoareactapp;
import android.os.AsyncTask;
import android.util.Log;
public class StreamTask extends AsyncTask <MediaStreamer, Integer, Long>{
protected Long doInBackground(MediaStreamer... mediaStreamers) {
Log.v("ReactNative", "Start");
for(int i = 0; i < mediaStreamers.length; i++) {
try {
mediaStreamers[i].prepare();
Log.v("ReactNative", "Prepared");
} catch (Exception e) {
Log.v("ReactNative", "Error");
Log.v("ReactNative", Log.getStackTraceString(e));
}
}
return null;
}
}
我的错误是
08-05 15:28:49.465 555 571 V ReactNative: java.lang.IllegalStateException
08-05 15:28:49.465 555 571 V ReactNative: at android.media.MediaRecorder._prepare(Native Method)
08-05 15:28:49.465 555 571 V ReactNative: at android.media.MediaRecorder.prepare(MediaRecorder.java:827)
08-05 15:28:49.465 555 571 V ReactNative: at com.ijkoareactapp.MediaStreamer.prepare(MediaStreamer.java:37)
08-05 15:28:49.465 555 571 V ReactNative: at com.ijkoareactapp.StreamTask.doInBackground(StreamTask.java:14)
08-05 15:28:49.465 555 571 V ReactNative: at com.ijkoareactapp.StreamTask.doInBackground(StreamTask.java:6)
08-05 15:28:49.465 555 571 V ReactNative: at android.os.AsyncTask$2.call(AsyncTask.java:304)
08-05 15:28:49.465 555 571 V ReactNative: at java.util.concurrent.FutureTask.run(FutureTask.java:237)
08-05 15:28:49.465 555 571 V ReactNative: at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:243)
08-05 15:28:49.465 555 571 V ReactNative: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
08-05 15:28:49.465 555 571 V ReactNative: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
08-05 15:28:49.465 555 571 V ReactNative: at java.lang.Thread.run(Thread.java:761)
阅读一些文档似乎prepare是在之前start或之后调用setOutputFormat的，但事实并非如此。
我认为我的内部套接字连接没有很好地初始化，但我找不到正确的方法。
也不确定我应该把它放在哪里mMediaProjection.createVirtualDisplay，目前我有以下错误应该是因为prepare does not works
08-05 15:43:36.077 1575 1575 V ReactNative: java.lang.IllegalStateException: failed to get surface
08-05 15:43:36.077 1575 1575 V ReactNative: at android.media.MediaRecorder.getSurface(Native Method)
08-05 15:43:36.077 1575 1575 V ReactNative: at com.ijkoareactapp.ScreenMirroringModule$1.onActivityResult(ScreenMirroringModule.java:65)
08-05 15:43:36.077 1575 1575 V ReactNative: at com.facebook.react.bridge.ReactContext.onActivityResult(ReactContext.java:262)
08-05 15:43:36.077 1575 1575 V ReactNative: at com.facebook.react.ReactInstanceManager.onActivityResult(ReactInstanceManager.java:703)
08-05 15:43:36.077 1575 1575 V ReactNative: at com.facebook.react.ReactActivityDelegate.onActivityResult(ReactActivityDelegate.java:124)
08-05 15:43:36.077 1575 1575 V ReactNative: at com.facebook.react.ReactActivity.onActivityResult(ReactActivity.java:75)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.app.Activity.dispatchActivityResult(Activity.java:6915)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.app.ActivityThread.deliverResults(ActivityThread.java:4049)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.app.ActivityThread.handleSendResult(ActivityThread.java:4096)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.app.ActivityThread.-wrap20(ActivityThread.java)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1516)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.os.Handler.dispatchMessage(Handler.java:102)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.os.Looper.loop(Looper.java:154)
08-05 15:43:36.077 1575 1575 V ReactNative: at android.app.ActivityThread.main(ActivityThread.java:6077)
08-05 15:43:36.077 1575 1575 V ReactNative: at java.lang.reflect.Method.invoke(Native Method)
08-05 15:43:36.077 1575 1575 V ReactNative: at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:866)
08-05 15:43:36.077 1575 1575 V ReactNative: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:756)
解决方案
很难在本地重现您的问题。您可能对 MediaStreamer 初始化的错误顺序是正确的。尝试逐行遵循此顺序：
- 设置视频源
- 设置输出格式
- 设置视频编码器
- setPreviewDisplay(surfaceHolder.getSurface())
- 设置输出文件
- 准备
- 开始
如果它没有帮助，我有最好的建议 - 看看正在工作的 React Native/Android 项目并尝试找出不同之处： https ://github.com/ycswaves/react-native-screen-recorder/blob/ master/android/app/src/main/java/com/screenrecorder/MainActivity.java#L131~L143 或者这个： https ://github.com/fyhertz/spydroid-ipcamera/blob/779f1035ac8fd91be5dfba99516da1b9f29f8768/src/net/majorkernelpanic/流/视频/VideoStream.java#L339~L357
根据这个 repos OutputFormat.MPEG_4 不是套接字的最佳选择。你想试试 THREE_GPP 吗？
最后一件事，你必须为你的应用添加额外的权限：
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_SETTINGS" />
祝你好运！
推荐阅读
- vue.js - 为每个创建带有 id 的动态文本字段
- android - 从 firebase 在 RecyclerView 中显示数据：错误：无法将 java.lang.String 类型的对象转换为
- big-o - 这段代码的时间复杂度是多少，其中一个循环以几何方式增长，而另一个循环以代数方式衰减？
- java - 带有 Java 和 Selenium 的 Cucumber 错误地报告重复的步骤定义
- python - OpenCV 将数据从 C++ 传递到 Python
- swift - SKLabelNode PhysicsBody 问题
- python-3.x - list() 定义类型错误
- node.js - coinbase API 问题
- python - 如何更新将其值保存在列表中的字典键？
- reactjs - React/Socket.io - 客户端只向所有其他客户端发出一次，然后只向自己发出一次