麻烦将libavformat/ffmpeg与x264和RTP同步[英] Trouble syncing libavformat/ffmpeg with x264 and RTP

本文是小编为大家收集整理的关于麻烦将libavformat/ffmpeg与x264和RTP同步的处理方法,想解了麻烦将libavformat/ffmpeg与x264和RTP同步的问题怎么解决?麻烦将libavformat/ffmpeg与x264和RTP同步问题的解决办法?那么可以参考本文帮助大家快速定位并解决问题。

问题描述

我一直在研究一些流媒体软件 使用网络上各种相机和流的流程 H.264.为此,我直接使用x264编码器(使用 预设的" Zerolatency")和喂食NALS可以使用 libavformat包装到RTP(最终RTSP)中.理想情况下,这个 申请应尽可能实时.在大多数情况下, 这一直运行良好.

不幸的是,存在某种同步问题: 客户播放的任何视频播放似乎都显示了一些平滑的帧, 其次是短暂的停顿,然后是更多框架;重复.此外, 似乎有大约4秒的延迟.这发生在 我尝试过的每个视频播放器:图腾,VLC和基本GSTREAMER PIPES.

我将其全部归结为一个小的测试用例:

#include <stdio.h>
#include <stdint.h>
#include <unistd.h>
#include <x264.h>
#include <libavformat/avformat.h>
#include <libswscale/swscale.h>

#define WIDTH       640
#define HEIGHT      480
#define FPS         30
#define BITRATE     400000
#define RTP_ADDRESS "127.0.0.1"
#define RTP_PORT    49990

struct AVFormatContext* avctx;
struct x264_t* encoder;
struct SwsContext* imgctx;

uint8_t test = 0x80;


void create_sample_picture(x264_picture_t* picture)
{
    // create a frame to store in
    x264_picture_alloc(picture, X264_CSP_I420, WIDTH, HEIGHT);

    // fake image generation
    // disregard how wrong this is; just writing a quick test
    int strides = WIDTH / 8;
    uint8_t* data = malloc(WIDTH * HEIGHT * 3);
    memset(data, test, WIDTH * HEIGHT * 3);
    test = (test << 1) | (test >> (8 - 1));

    // scale the image
    sws_scale(imgctx, (const uint8_t* const*) &data, &strides, 0, HEIGHT,
              picture->img.plane, picture->img.i_stride);
}

int encode_frame(x264_picture_t* picture, x264_nal_t** nals)
{
    // encode a frame
    x264_picture_t pic_out;
    int num_nals;
    int frame_size = x264_encoder_encode(encoder, nals, &num_nals, picture, &pic_out);

    // ignore bad frames
    if (frame_size < 0)
    {
        return frame_size;
    }

    return num_nals;
}

void stream_frame(uint8_t* payload, int size)
{
    // initalize a packet
    AVPacket p;
    av_init_packet(&p);
    p.data = payload;
    p.size = size;
    p.stream_index = 0;
    p.flags = AV_PKT_FLAG_KEY;
    p.pts = AV_NOPTS_VALUE;
    p.dts = AV_NOPTS_VALUE;

    // send it out
    av_interleaved_write_frame(avctx, &p);
}

int main(int argc, char* argv[])
{
    // initalize ffmpeg
    av_register_all();

    // set up image scaler
    // (in-width, in-height, in-format, out-width, out-height, out-format, scaling-method, 0, 0, 0)
    imgctx = sws_getContext(WIDTH, HEIGHT, PIX_FMT_MONOWHITE,
                            WIDTH, HEIGHT, PIX_FMT_YUV420P,
                            SWS_FAST_BILINEAR, NULL, NULL, NULL);

    // set up encoder presets
    x264_param_t param;
    x264_param_default_preset(&param, "ultrafast", "zerolatency");

    param.i_threads = 3;
    param.i_width = WIDTH;
    param.i_height = HEIGHT;
    param.i_fps_num = FPS;
    param.i_fps_den = 1;
    param.i_keyint_max = FPS;
    param.b_intra_refresh = 0;
    param.rc.i_bitrate = BITRATE;
    param.b_repeat_headers = 1; // whether to repeat headers or write just once
    param.b_annexb = 1;         // place start codes (1) or sizes (0)

    // initalize
    x264_param_apply_profile(&param, "high");
    encoder = x264_encoder_open(&param);

    // at this point, x264_encoder_headers can be used, but it has had no effect

    // set up streaming context. a lot of error handling has been ommitted
    // for brevity, but this should be pretty standard.
    avctx = avformat_alloc_context();
    struct AVOutputFormat* fmt = av_guess_format("rtp", NULL, NULL);
    avctx->oformat = fmt;

    snprintf(avctx->filename, sizeof(avctx->filename), "rtp://%s:%d", RTP_ADDRESS, RTP_PORT);
    if (url_fopen(&avctx->pb, avctx->filename, URL_WRONLY) < 0)
    {
        perror("url_fopen failed");
        return 1;
    }
    struct AVStream* stream = av_new_stream(avctx, 1);

    // initalize codec
    AVCodecContext* c = stream->codec;
    c->codec_id = CODEC_ID_H264;
    c->codec_type = AVMEDIA_TYPE_VIDEO;
    c->flags = CODEC_FLAG_GLOBAL_HEADER;
    c->width = WIDTH;
    c->height = HEIGHT;
    c->time_base.den = FPS;
    c->time_base.num = 1;
    c->gop_size = FPS;
    c->bit_rate = BITRATE;
    avctx->flags = AVFMT_FLAG_RTP_HINT;

    // write the header
    av_write_header(avctx);

    // make some frames
    for (int frame = 0; frame < 10000; frame++)
    {
        // create a sample moving frame
        x264_picture_t* pic = (x264_picture_t*) malloc(sizeof(x264_picture_t));
        create_sample_picture(pic);

        // encode the frame
        x264_nal_t* nals;
        int num_nals = encode_frame(pic, &nals);

        if (num_nals < 0)
            printf("invalid frame size: %d\n", num_nals);

        // send out NALs
        for (int i = 0; i < num_nals; i++)
        {
            stream_frame(nals[i].p_payload, nals[i].i_payload);
        }

        // free up resources
        x264_picture_clean(pic);
        free(pic);

        // stream at approx 30 fps
        printf("frame %d\n", frame);
        usleep(33333);
    }

    return 0;
}

此测试显示在白色背景上的黑线 应该顺利移动向左移动.它是为FFMPEG编写的0.6.5 但是问题可以在 0.8 和 0.10 上复制(从我到目前为止测试的内容).我在错误处理中采取了一些捷径,使这个示例很短 在仍在显示问题的同时,请原谅一些 讨厌的代码.我还应该注意,虽然这里不使用SDP,但我 已经尝试将其与类似结果使用.测试可以是 编译:

gcc -g -std=gnu99 streamtest.c -lswscale -lavformat -lx264 -lm -lpthread -o streamtest

可以直接与gtreamer一起播放:

gst-launch udpsrc port=49990 ! application/x-rtp,payload=96,clock-rate=90000 ! rtph264depay ! decodebin ! xvimagesink

您应该立即注意到口吃.一个常见的"修复"我 在互联网上看到的是将Sync = false添加到管道:

gst-launch udpsrc port=49990 ! application/x-rtp,payload=96,clock-rate=90000 ! rtph264depay ! decodebin ! xvimagesink sync=false

这会导致播放平稳(并且接近实时),但是一个 非解决方案,仅与GSTREAMER一起使用.我想修理 来源的问题.我已经能够与几乎相同的 使用RAW FFMPEG的参数,没有任何问题:

ffmpeg -re -i sample.mp4 -vcodec libx264 -vpre ultrafast -vpre baseline -b 400000 -an -f rtp rtp://127.0.0.1:49990 -an

很显然我做错了什么.但是是什么?

推荐答案

1)您没有为发送到libx264的帧设置PTS(您可能应该看到"非刻板性单调性PTS"警告) 2)您没有为发送到libavformat的RTP Muxer发送的数据包设置PTS/DTS(我不确定需要设置它,但我想它会更好.从源代码中,从源代码看,它看起来像RTP使用PTS). 3)恕我直言(33333)不好.这导致编码器这次也停滞不前(增加延迟),而您可以在此期间编码下一帧,即使您仍然不需要通过RTP发送它.

P.S.顺便说一句,您没有将param.i_rc_method设置为x264_rc_abr,因此libx264将使用CRF 23,而忽略您的" param.rc.i_bitrate = bitrate".同样,在编码网络发送时使用VBV可能是个好主意.

本文地址:https://www.itbaoku.cn/post/359054.html