讨论广场 问答详情
相机录像反复暂停恢复出现失败
m0rty 2024-07-19 15:40:18
223 评论 分享

使用脚本压测相机录像,暂停和恢复间隔3s反复调用,相机出现录制失败

压测视频录制或者音频录制无该问题,感觉是音视频同步问题,但是日志并没有相关错误

代码版本是4.1release,社区相关暂停恢复的错误已经合入

223 评论 分享
相关文件下载
hilog.002.20240718-205905.zip
3.52 MB
下载
写回答
全部评论(7)
5 楼

定位到原因了,修改后,我这测试暂停恢复几分钟是正常的。排查不易,如果能解决你的问题,麻烦采纳一下该答案。
修改如下:
foundation/multimedia/player_framework/services/engine/gstreamer/plugins/source/videocapture/src/gst_video_capture_src.cpp
gst_video_capture_src_pause 函数,所有场景下pause后都需要清空视频buffer

    GstBufferPool *bufferpool = GST_BUFFER_POOL(surfacesrc->pool);
    if (src->stream_type != VIDEO_STREAM_TYPE_ES_AVC) {
        gst_buffer_pool_set_flushing(bufferpool, TRUE);
        gst_buffer_pool_set_flushing(bufferpool, FALSE);
    }
    GST_DEBUG_OBJECT(src, "video capture src pause");

修改成

    GstBufferPool *bufferpool = GST_BUFFER_POOL(surfacesrc->pool);
    gst_buffer_pool_set_flushing(bufferpool, TRUE);
    gst_buffer_pool_set_flushing(bufferpool, FALSE);
    GST_DEBUG_OBJECT(src, "video capture src pause");

1
2024-07-22 15:44:10
2024-07-24 09:30:06
大佬牛的呀
2024-07-24 09:30:06
4 楼

看如下打印,第二行是第10次恢复录制,第三行是恢复后拿到的首帧,比上一帧(暂停前)的pts大了3.9s,但是奇怪的是没有走到gst_video_capture_deal_with_pts 函数后面的逻辑了,此时的状态既不是RECORDER_RESUME也不是RECORDER_PAUSED,导致没有减去暂停的3.9s时间,导致了后续异常。理论上此时状态应该是RECORDER_RESUME,走line 361~368逻辑,没看懂。

可以在foundation/multimedia/player_framework/services/engine/gstreamer/plugins/source/videocapture/src/gst_video_capture_src.cpp 的gst_video_capture_deal_with_pts 函数中加打印,看此时的状态和paused_time 等。

还可以兼容操作:如果当前帧的pts比上一帧大2s以上,此时不是paused或resume状态也认为暂停了,增加total_pause_time的值

 

07-18 20:59:12.618 788 12878 D C02b2b/videocapturesrc: {gst_video_capture_src_pause():457} [gst::VideoSource:6AAEA0] video capture src pause

07-18 20:59:15.804 788 12878 D C02b2b/videocapturesrc: {gst_video_capture_src_resume():471} [gst::VideoSource:6AAEA0] video capture src resume

07-18 20:59:16.262 788 12882 D C02b2b/videocapturesrc: {gst_video_capture_deal_with_pts():337} [gst::VideoSource:6AAEA0] video capture buffer size is: 32064, pts: 812450800057

07-18 20:59:16.262 788 12882 I C02b2b/videocapturesrc: {gst_video_capture_deal_with_pts():341} [gst::VideoSource:6AAEA0] videoCapturer timestamp has increased: 3918618561

1
2024-07-19 21:04:10
2024-07-22 17:52:14
引用:“请教一下,openHarmony默认情况下GST_DEBUG_OBJECT宏的打印是不生效的,譬如:上面的日志video capture buffer size这样,需要修改什么才能开启openHarmony下GST_DEBUG_OBJECT、GST_INFO_OBJECT这些打印宏的打印啊?”
param set sys.media.log.level *:D param set hilog.loggable.global D 设置这两个系统属性
2024-07-22 17:52:14
2024-07-22 17:33:25
请教一下,openHarmony默认情况下GST_DEBUG_OBJECT宏的打印是不生效的,譬如:上面的日志video capture buffer size这样,需要修改什么才能开启openHarmony下GST_DEBUG_OBJECT、GST_INFO_OBJECT这些打印宏的打印啊?
2024-07-22 17:33:25
3 楼

出问题时音视频的pts差异4s,但是此时从src获取的音视频pts基本对齐。因为从src获取的音视频pts需要减去暂停期间的时长,怀疑恢复录制时音视频的暂停时间计算出现了偏差。
看最后3行,第10次暂停videocapturesrc未响应,导致视频pts直接比音频大了4s(未减去本次暂停时间),导致的报错。

解决方案:分析中

07-18 20:59:18.043   788 12923 D C02b2b/AudioCaptureAsImpl: {GetSegmentInfo():213} timestamp from audioCapturer: 814235163860
07-18 20:59:18.070   788 12882 D C02b2b/videocapturesrc: {gst_video_capture_deal_with_pts():337} [gst::VideoSource:6AAEA0] video capture buffer size is: 34179, pts: 814260656113
07-18 20:59:18.141   788 12882 W C02b2b/splitmuxsink: {check_completed_gop():2760} [gst::MuxSinkBin:F1C0D0] error: Timestamping error on input streams
07-18 20:59:18.141   788 12882 W C02b2b/splitmuxsink: {check_completed_gop():2760} [gst::MuxSinkBin:F1C0D0] error: Context 0x7fa1ade330 sink pad <queue_audio_0:sink> running time +0:13:01.638298898, next gop start: +0:13:05.685712872, diff +0:00:04.047413974

Line 46549: 07-18 20:59:09.424   788 12882 I C02b2b/videocapturesrc: {gst_video_capture_deal_with_pts():368} [gst::VideoSource:6AAEA0] video has 9 times pause, total PauseTime: 28595869323
Line 47157: 07-18 20:59:09.434   788 12889 D C02b2b/AudioCaptureAsImpl: {GetBuffer():357} audio has 9 times pause, total PauseTime: 29344133673
Line 118562: 07-18 20:59:15.812   788 12889 D C02b2b/AudioCaptureAsImpl: {GetBuffer():357} audio has 10 times pause, total PauseTime: 32611328920

2024-07-19 17:44:15
2 楼

07-18 20:59:18.141   788 12882 W C02b2b/structure: {priv_gst_structure_append_to_gstring():2093} [gst:: :0] No value transform to serialize field 'gerror' of type 'GError'

07-18 20:59:18.142   788 12881 E C02b2b/RecMsgProc: {ProcessErrorMsgDefault():62} [ERROR] domain:0x7ba, code:0x1, msg:Timestamping error on input streams, ../../../third_party/gstreamer/gstplugins_good/gst/multifile/gstsplitmuxsink.c(2760): check_completed_gop (): /GstPipeline:recorder-pipeline/GstSplitMuxSink:MuxSinkBin:
07-18 20:59:18.142   788 12881 E C02b2b/RecMsgProc: Context 0x7fa1ade330 sink pad <queue_audio_0:sink> running time +0:13:01.638298898, next gop start: +0:13:05.685712872, diff +0:00:04.047413974.

是时间戳相差超过3s出现录像异常,可以分析代码rk_4.1/foundation/multimedia/player_framework/services/engine/gstreamer/plugins/source下audio的时间戳计算和video的时间戳计算

2024-07-19 17:21:20