时间戳

首页 > 技术文章 > 安卓视频处理总结

安卓视频处理总结

rockylearnstodevelop 2016-04-05 08:35 原文

 

 

04-04 16:55:57.099 28015-28195/com.rockylearnstorock.testcamera D/MediaHelper: {csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], mime=video/avc, frame-rate=30, rotation-degrees=270, height=144, width=176, max-input-size=5755, durationUs=21107833, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20]}
04-04 16:55:57.106 28015-28195/com.rockylearnstorock.testcamera D/MediaHelper: {mime=audio/mpeg, durationUs=256835918, encoder-delay=576, channel-count=2, encoder-padding=1681, sample-rate=44100, bit-rate=128000}
04-04 16:55:57.106 28015-28195/com.rockylearnstorock.testcamera E/MPEG4Writer: Unsupported mime 'audio/mpeg'
04-04 16:55:57.107 28015-28195/com.rockylearnstorock.testcamera W/System.err: java.lang.IllegalStateException: Failed to add the track to the muxer
04-04 16:55:57.111 28015-28195/com.rockylearnstorock.testcamera W/System.err: at android.media.MediaMuxer.nativeAddTrack(Native Method)
04-04 16:55:57.111 28015-28195/com.rockylearnstorock.testcamera W/System.err: at android.media.MediaMuxer.addTrack(MediaMuxer.java:294)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at com.rockylearnstorock.testcamera.MediaHelper.combineAudioVideo(MediaHelper.java:181)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at com.rockylearnstorock.testcamera.MainActivity$2.run(MainActivity.java:399)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at java.lang.Thread.run(Thread.java:818)

 

Google "Unsupported mime 'audio/mpeg'， 找到StackOverflow： Impossible to mix audio file and video file using MediaMuxer?

里面有个解释是

MediaMuxer does not transcode. If you write out an MPEG4 file, it will expect the video file to be MPEG4/AAC and the audio file to be an AAC file (m4a) as well.
Once you feed it with an m4a, muxing will succeed.

尝试将MP3换成m4a文件，合成，Negtive。

这个贴子里面还有个人发了一个github 例子，尝试， 合成的视频m4a的声音根本就没有加进去。Negtive.

 

Google " java.lang.IllegalStateException: Failed to add the track to the muxer", 返回的结果，都是" java.lang.IllegalStateException: Failed to stop the track to the muxer"。这个显然不是我要问的。

再看报错行

int audioTrackIndex = muxer.addTrack(audioFormat); //MediaHelper.java:181 报错行

显示是将audioFormat添加到muxer时报错了。

关于MP4和MP3的MediaFormat，打印结果见上面黄色部分：

MP4:{csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], mime=video/avc, frame-rate=30, rotation-degrees=270, height=144, width=176, max-input-size=5755, durationUs=21107833, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20]}

MP3:{mime=audio/mpeg, durationUs=256835918, encoder-delay=576, channel-count=2, encoder-padding=1681, sample-rate=44100, bit-rate=128000}

两者的编码一个是avc, 另一个是mpeg. 

根据我的理解， 不同的编码可能造成这个方法报错的原因。因为，“Unsupported mime 'audio/mpeg'”。

 

MediaExtractor audioExtractor = new MediaExtractor();
        try {
            audioExtractor.setDataSource("/storage/emulated/0/DCIM/TestCamera/kisstherain.mp3");
        } catch (IOException e) {
            e.printStackTrace();
            Log.d("MediaHelper", "mp3 file can not be found");
            return;
        }

        MediaFormat audioFormat = null;
        for(int i = 0; i < audioExtractor.getTrackCount(); i++){
            audioFormat = audioExtractor.getTrackFormat(i);
            String mime = audioFormat.getString(MediaFormat.KEY_MIME);
            if(mime.startsWith("audio/")) {
                break;
            }
        }

        if(audioFormat == null){
            Log.d("MediaHelper", "audioFormat is null");
            return;
        }else{
            Log.d("MediaHelper", audioFormat.toString());
        }

        int audioTrackIndex = muxer.addTrack(audioFormat); //MediaHelper.java:181 报错行

 

尝试将MP3改为m4a文件，进行合并。

1.参考：Android中如何提取和生成mp4文件 （这是目前为止找到的最接近的答案）

按照文中的思路，提取video和audio， 添加到MediaMuxer。生成的视频只有m4a中的声音，没有video原来的声音。

public static void combineAudioVideo(){

        MediaMuxer muxer;
        try {
            muxer = new MediaMuxer("/storage/emulated/0/DCIM/TestCamera/temp.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
        } catch (IOException e) {
            e.printStackTrace();

            Log.d("MediaHelper", "MediaMuxer init failed");
            return;
        }
//         More often, the MediaFormat will be retrieved from MediaCodec.getOutputFormat()
//         or MediaExtractor.getTrackFormat().
        MediaExtractor mMediaExtractor = new MediaExtractor();
        try {
            mMediaExtractor.setDataSource(FileHelper.getOutputMediaFile(FileHelper.MEDIA_TYPE_VIDEO).getPath());
        } catch (IOException e) {
            e.printStackTrace();
        }

        int framerate = 0;
        MediaFormat videoFormat = null;
        for(int i = 0; i < mMediaExtractor.getTrackCount(); i++){
            videoFormat = mMediaExtractor.getTrackFormat(i);
            String mime = videoFormat.getString(MediaFormat.KEY_MIME);
            framerate = videoFormat.getInteger(MediaFormat.KEY_FRAME_RATE);
            if(mime.startsWith("video/")) {
                mMediaExtractor.selectTrack(i);
                break;
            }
        }

        if(videoFormat == null){

            Log.d("MediaHelper", "videoFormat is null");
            return;
        }else {
            Log.d("MediaHelper", videoFormat.toString());
        }

        MediaExtractor audioExtractor = new MediaExtractor();
        try {
            audioExtractor.setDataSource("/storage/emulated/0/DCIM/TestCamera/1.m4a");
        } catch (IOException e) {
            e.printStackTrace();
            Log.d("MediaHelper", "mp3 file can not be found");
            return;
        }

        int sampleRate = 0;
        MediaFormat audioFormat = null;
        for(int i = 0; i < audioExtractor.getTrackCount(); i++){
            audioFormat = audioExtractor.getTrackFormat(i);
            String mime = audioFormat.getString(MediaFormat.KEY_MIME);
            sampleRate = audioFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE);
            if(mime.startsWith("audio/")) {
                audioExtractor.selectTrack(i);
                break;
            }
        }

        if(audioFormat == null){
            Log.d("MediaHelper", "audioFormat is null");
            return;
        }else{
            Log.d("MediaHelper", audioFormat.toString());
        }

        int audioTrackIndex = muxer.addTrack(audioFormat);
        int videoTrackIndex = muxer.addTrack(videoFormat);

        muxer.start();

        MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
        MediaCodec.BufferInfo audioInfo = new MediaCodec.BufferInfo();
        info.presentationTimeUs = 0;
        ByteBuffer buffer = ByteBuffer.allocate(500*1024);
        ByteBuffer audioBuffer = ByteBuffer.allocate(500 * 1024);

        while(true) {
            int sampleSize = mMediaExtractor.readSampleData(buffer, 0);
            if(sampleSize < 0) {
                break;
            }

            int audioSampleSize = audioExtractor.readSampleData(audioBuffer, 0);
            if(audioSampleSize < 0){
                break;
            }

            mMediaExtractor.advance();
            audioExtractor.advance();

            info.offset = 0;
            info.size = sampleSize;
            info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
            info.presentationTimeUs += 1000*1000/framerate;

            audioInfo.offset = 0;
            audioInfo.size = audioSampleSize;
            audioInfo.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
//            audioInfo.presentationTimeUs += 1000 * 1000 / sampleRate;  //暂提不明白sampleRate有什么作用，如果加了这句，生在的视频就无法播放

            muxer.writeSampleData(videoTrackIndex, buffer, info);
            muxer.writeSampleData(audioTrackIndex, audioBuffer, audioInfo);
        }

        mMediaExtractor.release();
        audioExtractor.release();

        muxer.stop();
        muxer.release();

    }

 

可能的原因：

 int audioTrackIndex = muxer.addTrack(audioFormat); //添加audio时，只有m4a文件audio，所以原来mp4文件中的就被覆盖了。

尝试添加多个audio track 到MediaMuxer, google结果：Add second audio track to MediaMuxer

 

推荐阅读