问PCM - > AAC（编码器） - > PCM（解码器）如何实时进行正确的优化？
EN

Stack Overflow用户

提问于 2018-03-13 06:57:03

回答 6查看 0关注 0票数 0

我在努力实现

AudioRecord (MIC) ->

PCM -> AAC Encoder
AAC -> PCM Decode

-> AudioTrack??  (SPEAKER)

MediaCodec在安卓4.1+上(API 16)。

首先，我成功地实现了PCM -> AAC Encoder通过MediaCodec如下所示

private boolean setEncoder(int rate)
{
    encoder = MediaCodec.createEncoderByType("audio/mp4a-latm");
    MediaFormat format = new MediaFormat();
    format.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm");
    format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
    format.setInteger(MediaFormat.KEY_SAMPLE_RATE, 44100);
    format.setInteger(MediaFormat.KEY_BIT_RATE, 64 * 1024);//AAC-HE 64kbps
    format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectHE);
    encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    return true;
}

输入：PCM比特率=44100(Hz)x16(位)x1(单片)=705600位/秒

输出：aac-HE比特率=64x1024(位)=65536位/秒

解码器配置文件设置如下：

private boolean setDecoder(int rate)
{
    decoder = MediaCodec.createDecoderByType("audio/mp4a-latm");
    MediaFormat format = new MediaFormat();
    format.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm");
    format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
    format.setInteger(MediaFormat.KEY_SAMPLE_RATE, 44100);
    format.setInteger(MediaFormat.KEY_BIT_RATE, 64 * 1024);//AAC-HE 64kbps
    format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectHE);
    decoder.configure(format, null, null, 0);

    return true;
}

因为UDPserver数据包缓冲区大小是1024

UDPserver：接收1024字节由于这是压缩的aac数据，我希望解码的大小将是大约1024x11，但是实际结果是
音频解码器：8192字节解码

解码器代码如下：

    IOudpPlayer = new Thread(new Runnable()
    {
        public void run()
        {
            SocketAddress sockAddress;
            String address;

            int len = 1024;
            byte[] buffer2 = new byte[len];
            DatagramPacket packet;

            byte[] data;

            ByteBuffer[] inputBuffers;
            ByteBuffer[] outputBuffers;

            ByteBuffer inputBuffer;
            ByteBuffer outputBuffer;

            MediaCodec.BufferInfo bufferInfo;
            int inputBufferIndex;
            int outputBufferIndex;
            byte[] outData;
            try
            {
                decoder.start();
                isPlaying = true;
                while (isPlaying)
                {
                    try
                    {
                        packet = new DatagramPacket(buffer2, len);
                        ds.receive(packet);

                        sockAddress = packet.getSocketAddress();
                        address = sockAddress.toString();

                        Log.d("UDP Receiver"," received !!! from " + address);

                        data = new byte[packet.getLength()];
                        System.arraycopy(packet.getData(), packet.getOffset(), data, 0, packet.getLength());

                        Log.d("UDP Receiver",  data.length + " bytes received");

                        //===========
                        inputBuffers = decoder.getInputBuffers();
                        outputBuffers = decoder.getOutputBuffers();
                        inputBufferIndex = decoder.dequeueInputBuffer(-1);
                        if (inputBufferIndex >= 0)
                        {
                            inputBuffer = inputBuffers[inputBufferIndex];
                            inputBuffer.clear();

                            inputBuffer.put(data);

                            decoder.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0);
                        }

                        bufferInfo = new MediaCodec.BufferInfo();
                        outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);

                        while (outputBufferIndex >= 0)
                        {
                            outputBuffer = outputBuffers[outputBufferIndex];

                            outputBuffer.position(bufferInfo.offset);
                            outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                            outData = new byte[bufferInfo.size];
                            outputBuffer.get(outData);

                            Log.d("AudioDecoder", outData.length + " bytes decoded");

                            decoder.releaseOutputBuffer(outputBufferIndex, false);
                            outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);

                        }



                        //===========

                    }
                    catch (IOException e)
                    {
                    }
                }

                decoder.stop();

            }
            catch (Exception e)
            {
            }
        }
    });

完整的代码：

https：//gist.github.com/kenokabe/9029256

还需要得到许可：

 <uses-permission android:name="android.permission.INTERNET"></uses-permission>
 <uses-permission android:name="android.permission.RECORD_AUDIO"></uses-permission>

所以我的问题是

能正确地优化我的示例代码以使其正常工作吗？
这是一个正确的方式来使用AudioTrack API来动态播放解码的PCM原始数据，并且可以告诉我正确的方法吗？

回答 6

Stack Overflow用户

发布于 2018-03-13 11:46:58

在测试之后，我从修改代码中得到了以下结果：

 package com.example.app;

    import android.app.Activity;

    import android.media.AudioManager;
    import android.media.MediaCodecInfo;
    import android.media.MediaFormat;
    import android.os.Bundle;

    import android.media.AudioFormat;
    import android.media.AudioRecord;
    import android.media.AudioTrack;
    import android.media.MediaCodec;

    import android.media.MediaRecorder.AudioSource;

    import android.util.Log;

    import java.io.IOException;
    import java.net.DatagramPacket;
    import java.net.DatagramSocket;
    import java.net.InetAddress;
    import java.net.SocketAddress;
    import java.net.SocketException;
    import java.nio.ByteBuffer;

    public class MainActivity extends Activity
    {
        private AudioRecord recorder;
        private AudioTrack player;

        private MediaCodec encoder;
        private MediaCodec decoder;

        private short audioFormat = AudioFormat.ENCODING_PCM_16BIT;
        private short channelConfig = AudioFormat.CHANNEL_IN_MONO;

        private int bufferSize;
        private boolean isRecording;
        private boolean isPlaying;

        private Thread IOrecorder;

        private Thread IOudpPlayer;


        private DatagramSocket ds;
        private final int localPort = 39000;

        @Override
        protected void onCreate(Bundle savedInstanceState)
        {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.activity_main);

            IOrecorder = new Thread(new Runnable()
            {
                public void run()
                {
                    int read;
                    byte[] buffer1 = new byte[bufferSize];

                    ByteBuffer[] inputBuffers;
                    ByteBuffer[] outputBuffers;

                    ByteBuffer inputBuffer;
                    ByteBuffer outputBuffer;

                    MediaCodec.BufferInfo bufferInfo;
                    int inputBufferIndex;
                    int outputBufferIndex;

                    byte[] outData;

                    DatagramPacket packet;
                    try
                    {
                        encoder.start();
                        recorder.startRecording();
                        isRecording = true;
                        while (isRecording)
                        {
                            read = recorder.read(buffer1, 0, bufferSize);
                           // Log.d("AudioRecoder", read + " bytes read");
                            //------------------------

                            inputBuffers = encoder.getInputBuffers();
                            outputBuffers = encoder.getOutputBuffers();
                            inputBufferIndex = encoder.dequeueInputBuffer(-1);
                            if (inputBufferIndex >= 0)
                            {
                                inputBuffer = inputBuffers[inputBufferIndex];
                                inputBuffer.clear();

                                inputBuffer.put(buffer1);

                                encoder.queueInputBuffer(inputBufferIndex, 0, buffer1.length, 0, 0);
                            }

                            bufferInfo = new MediaCodec.BufferInfo();
                            outputBufferIndex = encoder.dequeueOutputBuffer(bufferInfo, 0);



                            while (outputBufferIndex >= 0)
                            {
                                outputBuffer = outputBuffers[outputBufferIndex];

                                outputBuffer.position(bufferInfo.offset);
                                outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                                outData = new byte[bufferInfo.size];
                                outputBuffer.get(outData);


                               // Log.d("AudioEncoder ", outData.length + " bytes encoded");
                                //-------------
                                packet = new DatagramPacket(outData, outData.length,
                                        InetAddress.getByName("127.0.0.1"), localPort);
                                ds.send(packet);
                                //------------

                                encoder.releaseOutputBuffer(outputBufferIndex, false);
                                outputBufferIndex = encoder.dequeueOutputBuffer(bufferInfo, 0);

                            }
                            // ----------------------;

                        }
                        encoder.stop();
                        recorder.stop();
                    }
                    catch (Exception e)
                    {
                        e.printStackTrace();
                    }
                }
            });



            IOudpPlayer = new Thread(new Runnable()
            {
                public void run()
                {
                    SocketAddress sockAddress;
                    String address;

                    int len = 2048
                    byte[] buffer2 = new byte[len];
                    DatagramPacket packet;

                    byte[] data;

                    ByteBuffer[] inputBuffers;
                    ByteBuffer[] outputBuffers;

                    ByteBuffer inputBuffer;
                    ByteBuffer outputBuffer;

                    MediaCodec.BufferInfo bufferInfo;
                    int inputBufferIndex;
                    int outputBufferIndex;
                    byte[] outData;
                    try
                    {
                        player.play();
                        decoder.start();
                        isPlaying = true;
                        while (isPlaying)
                        {
                            try
                            {
                                packet = new DatagramPacket(buffer2, len);
                                ds.receive(packet);

                                sockAddress = packet.getSocketAddress();
                                address = sockAddress.toString();

                             //   Log.d("UDP Receiver"," received !!! from " + address);

                                data = new byte[packet.getLength()];
                                System.arraycopy(packet.getData(), packet.getOffset(), data, 0, packet.getLength());

                               // Log.d("UDP Receiver",  data.length + " bytes received");

                                //===========
                                inputBuffers = decoder.getInputBuffers();
                                outputBuffers = decoder.getOutputBuffers();
                                inputBufferIndex = decoder.dequeueInputBuffer(-1);
                                if (inputBufferIndex >= 0)
                                {
                                    inputBuffer = inputBuffers[inputBufferIndex];
                                    inputBuffer.clear();

                                    inputBuffer.put(data);

                                    decoder.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0);
                                }

                                bufferInfo = new MediaCodec.BufferInfo();
                                outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);

                                while (outputBufferIndex >= 0)
                                {
                                    outputBuffer = outputBuffers[outputBufferIndex];

                                    outputBuffer.position(bufferInfo.offset);
                                    outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                                    outData = new byte[bufferInfo.size];
                                    outputBuffer.get(outData);

                                  //  Log.d("AudioDecoder", outData.length + " bytes decoded");

                                    player.write(outData, 0, outData.length);

                                    decoder.releaseOutputBuffer(outputBufferIndex, false);
                                    outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0..

票数 0

Stack Overflow用户

发布于 2018-03-13 12:44:07

package com.example.app;

import android.app.Activity;

import android.media.AudioManager;
import android.media.MediaCodecInfo;
import android.media.MediaFormat;
import android.os.Bundle;

import android.media.AudioFormat;
import android.media.AudioRecord;
import android.media.AudioTrack;
import android.media.MediaCodec;

import android.media.MediaRecorder.AudioSource;

import android.util.Log;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketAddress;
import java.net.SocketException;
import java.nio.ByteBuffer;

public class MainActivity extends Activity
{
    private AudioRecord recorder;
    private AudioTrack player;

    private MediaCodec encoder;
    private MediaCodec decoder;

    private short audioFormat = AudioFormat.ENCODING_PCM_16BIT;
    private short channelConfig = AudioFormat.CHANNEL_IN_MONO;

    private int bufferSize;
    private boolean isRecording;
    private boolean isPlaying;

    private Thread IOrecorder;

    private Thread IOudpPlayer;


    private DatagramSocket ds;
    private final int localPort = 39000;

    @Override
    protected void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        IOrecorder = new Thread(new Runnable()
        {
            public void run()
            {
                int read;
                byte[] buffer1 = new byte[bufferSize];

                ByteBuffer[] inputBuffers;
                ByteBuffer[] outputBuffers;

                ByteBuffer inputBuffer;
                ByteBuffer outputBuffer;

                MediaCodec.BufferInfo bufferInfo;
                int inputBufferIndex;
                int outputBufferIndex;

                byte[] outData;

                DatagramPacket packet;
                try
                {
                    encoder.start();
                    recorder.startRecording();
                    isRecording = true;
                    while (isRecording)
                    {
                        read = recorder.read(buffer1, 0, bufferSize);
                       // Log.d("AudioRecoder", read + " bytes read");
                        //------------------------

                        inputBuffers = encoder.getInputBuffers();
                        outputBuffers = encoder.getOutputBuffers();
                        inputBufferIndex = encoder.dequeueInputBuffer(-1);
                        if (inputBufferIndex >= 0)
                        {
                            inputBuffer = inputBuffers[inputBufferIndex];
                            inputBuffer.clear();

                            inputBuffer.put(buffer1);

                            encoder.queueInputBuffer(inputBufferIndex, 0, buffer1.length, 0, 0);
                        }

                        bufferInfo = new MediaCodec.BufferInfo();
                        outputBufferIndex = encoder.dequeueOutputBuffer(bufferInfo, 0);



                        while (outputBufferIndex >= 0)
                        {
                            outputBuffer = outputBuffers[outputBufferIndex];

                            outputBuffer.position(bufferInfo.offset);
                            outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                            outData = new byte[bufferInfo.size];
                            outputBuffer.get(outData);


                           // Log.d("AudioEncoder", outData.length + " bytes encoded");
                            //-------------
                            packet = new DatagramPacket(outData, outData.length,
                                    InetAddress.getByName("127.0.0.1"), localPort);
                            ds.send(packet);
                            //------------

                            encoder.releaseOutputBuffer(outputBufferIndex, false);
                            outputBufferIndex = encoder.dequeueOutputBuffer(bufferInfo, 0);

                        }
                        // ----------------------;

                    }
                    encoder.stop();
                    recorder.stop();
                }
                catch (Exception e)
                {
                    e.printStackTrace();
                }
            }
        });



        IOudpPlayer = new Thread(new Runnable()
        {
            public void run()
            {
                SocketAddress sockAddress;
                String address;

                int len = 1024;
                byte[] buffer2 = new byte[len];
                DatagramPacket packet;

                byte[] data;

                ByteBuffer[] inputBuffers;
                ByteBuffer[] outputBuffers;

                ByteBuffer inputBuffer;
                ByteBuffer outputBuffer;

                MediaCodec.BufferInfo bufferInfo;
                int inputBufferIndex;
                int outputBufferIndex;
                byte[] outData;
                try
                {
                    player.play();
                    decoder.start();
                    isPlaying = true;
                    while (isPlaying)
                    {
                        try
                        {
                            packet = new DatagramPacket(buffer2, len);
                            ds.receive(packet);

                            sockAddress = packet.getSocketAddress();
                            address = sockAddress.toString();

                         //   Log.d("UDP Receiver"," received !!! from " + address);

                            data = new byte[packet.getLength()];
                            System.arraycopy(packet.getData(), packet.getOffset(), data, 0, packet.getLength());

                           // Log.d("UDP Receiver",  data.length + " bytes received");

                            //===========
                            inputBuffers = decoder.getInputBuffers();
                            outputBuffers = decoder.getOutputBuffers();
                            inputBufferIndex = decoder.dequeueInputBuffer(-1);
                            if (inputBufferIndex >= 0)
                            {
                                inputBuffer = inputBuffers[inputBufferIndex];
                                inputBuffer.clear();

                                inputBuffer.put(data);

                                decoder.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0);
                            }

                            bufferInfo = new MediaCodec.BufferInfo();
                            outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);

                            while (outputBufferIndex >= 0)
                            {
                                outputBuffer = outputBuffers[outputBufferIndex];

                                outputBuffer.position(bufferInfo.offset);
                                outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                                outData = new byte[bufferInfo.size];
                                outputBuffer.get(outData);

                              //  Log.d("AudioDecoder", outData.length + " bytes decoded");

                                player.write(outData, 0, outData.length);

                                decoder.releaseOutputBuffer(outputBufferIndex, false);
                                outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);

                            }

                            //===========

                        }
                        catch (IOException e)
                        {
                        }
                    }

                    decoder.stop();
                    player.stop();

                }
                catch (Exception e)
                {
                }
            }
        });

//===========================================================
        int rate = findAudioRecord();
        if (rate != -1)
        {
            Log.v("=========media ", "ready: " + rate);
            Log.v("=========media channel ", "ready: " + channelConfig);

            boolean encoderReady = setEncoder(rate);
            Log.v("=========encoder ", "ready: " + encoderReady);
            if (encoderReady)
            {
                boolean decoderReady = setDecoder(rate);
                Log.v("=========decoder ", "ready: " + decoderReady);
                if (decoderReady)
                {
                    Log.d("=======bufferSize========", "" + bufferSize);
                    try
                    {
                        setPlayer(rate);

                        ds = new DatagramSocket(localPort);
                        IOudpPlayer.start();

                        IOrecorder.start();

                    }
                    catch (SocketException e)
                    {
                        e.printStackTrace();
                    }


                }

            }
        }
    }


    protected void onDestroy()
    {

        recorder.release();
        player.release();
        encoder.release();
        decoder.release();

    }
/*
    protected void onResume()
    {

        // isRecording = true;
    }

    protected void onPause()
    {

        isRecording = false;
    }
*/

    private int findAudioRecord()
    {
        for (int rate : new int[]{44100})
        {
            try
            {
                Log.v("===========Attempting rate ", rate + "Hz, bits: " + audioFormat + ", channel: " + channelConfig);
                bufferSize = AudioRecord.getMinBufferSize(rate, channelConfig, audioFormat);

                if (bufferSize != AudioRecord.ERROR_BAD_VALUE)
                {
                    // check if we can instantiate and have a success
                    recorder = new AudioRecord(AudioSource.MIC, rate, channelConfig, audioFormat, bufferSize);

                    if (recorder.getState() == AudioRecord.STATE_INITIALIZED)
                    {
                        Log.v("===========final rate ", rate + "Hz, bits: " + audioFormat + ", channel: " + channelConfig);

                        return rate;
                    }
                }
            }
            catch (Exception e)
            {
                Log.v("error", "" + rate);
            }

        }
        return -1;
    }

    private boolean setEncoder(int rate)
    {
        encoder = MediaCodec.createEncoderByType("audio/mp4a-latm");
        MediaFormat format = new MediaFormat();
        format.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm");
        format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
        format.setInteger(MediaFormat.KEY_SAMPLE_RATE, rate);
        format.setInteger(MediaFormat.KEY_BIT_RATE, 64 * 1024);//AAC-HE 64kbps
        format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectHE);
        encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
        return true;
    }

    private boolean setDecoder(int rate)
    {
        decoder = MediaCodec.createDecoderByType("audio/mp4a-latm");
        MediaFormat format = new MediaFormat();
        format.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm");
        format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
        format.setInteger(MediaFormat.KEY_SAMPLE_RATE, rate);
        format.setInteger(MediaFormat.KEY_BIT_RATE, 64 * 1024);//AAC-HE 64kbps
        format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectHE);

        decoder.configure(format, null, null, 0);

        return true;
    }

    private boolean setPlayer(int rate)
    {
        int bufferSizePlayer = AudioTrack.getMinBufferSize(rate, AudioFormat.CHANNEL_OUT_MONO, audioFormat);
        Log.d("====buffer Size player ", String.valueOf(bufferSizePlayer));

        player= new AudioTrack(AudioManager.STREAM_MUSIC, rate, AudioFormat.CHANNEL_OUT_MONO, audioFormat, bufferSizePlayer, AudioTrack.MODE_STREAM);


        if (player.getState() == AudioTrack.STATE_INITIALIZED)
        {

            return true;
        }
        else
        {
            return false;
        }

    }


}

票数 0

Stack Overflow用户

发布于 2018-03-13 13:44:25

网络代码正在结合数据。获得了369字节的压缩数据，但在接收端，最终获得了1024个字节。这些1024字节由两个完整帧和一个部分帧组成。两个整个帧每个都会再次解码为4096个字节，总共可以看到8192个字节。一旦你向解码器发送了足够多的数据，剩余的部分帧就可能被解码，但你通常应该只发送整个帧到解码器。

另外，MediaCodec.dequeueOutputBuffer（）不仅返回（正）缓冲区索引，还返回（负）状态码。其中一个可能的代码是MediaCodec.INFO_OUTPUT_FORMAT_CHANGED，它表示您需要调用MediaCodec.getOutputFormat（）来获取音频数据的格式。即使输入为单声道，也可能会看到编解码器输出立体声。当收到其中一个状态代码时，发布的代码就会脱离循环。

票数 0

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/-100004249

复制

相似问题

问PCM - > AAC（编码器） - > PCM（解码器）如何实时进行正确的优化？
EN

回答 6

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问PCM - > AAC（编码器） - > PCM（解码器）如何实时进行正确的优化？EN

回答 6

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问PCM - > AAC（编码器） - > PCM（解码器）如何实时进行正确的优化？
EN