media_modules: netflix: merge patch for fix 29.97 and 23.97 output evenly [2/2]

PD#165744 : netflix: merge patch for fix 29.97 and 23.97 output evenly

NEEDLEPLAT-2057:
[Netflix][NTS] Video judders are observed during VP9/HEVC playback [1/1]

[Problem]
[Netflix][NTS] Video judders are observed during VP9/HEVC playback.
The problem happens at 'Boardwalk Twirl Ride' scene with seek point at 760
 seconds. Seek before the point or the exact point will show stuttery video
 display but a seek after the location or a short seek (swim operation in
 NF's term) will make the playback smooth again. [Solution] The problem
 is identified to be an uneven PTS timestamp for sample frames. In order
 to get 29.97fps the timestamp of video samples are switching between a
 30fps rate and some samples with longer duration so by average it's
 29.97fps. The output rendering control has very strict timestamp comparison
 between system time and the timestamp of video frame. When system time passed
 the next video frame's timestamp, a new frame is toggled to display. For the
 uneven PTS case, some frames have over 2 vsync duration and the others have
 less than 2 vsync duration. Depending on the initial vsync offset, there is a
 chance to have frames toggled in 13221322 vsync sequence, instead of a normal
 always 2 vsync toggle one frame pattern. The change is to do a small adjustment
 to the system time, based on initial system time and video timestamp so the "phase"
 of the vsync can be set up to avoid such situation.
 [Platform]
 needle,stark
 [Test]
 Verify with Netflix Chime S1E8 29.97fps titles, 'Boardwalk Twirl Ride' scene.
 Change-Id: I073481ce9e39f49555480a139e3b32d8cc047e1c
 Signed-off-by: Tim Yao <tim.yao@amlog

 NEEDLEPLAT-2604: [Netflix] Video judders during playback [1/1]
 [JIRA] NEEDLEPLAT-2604
 [Problem]
 FRC caused video judders with 23.97fps source.
 [Solution]
 When source is 23.97 fps and output is 59.94hz, in order to make it
 friendly to TV 3:2 pulldown detection, the video frame output should follow a
 323232 patten, which means for each vsync, frames are repeated 3 times, then
 switch to a new frame and repeat twice. Such repeating pattern will make the
 average vsync toggle rate become 2.5 frame per vsync, to match 23.97 to 59.94
 frame conversion rate. And TV side can also detect such patterns and do its best
 recovery for MEMC processing. The problem with Netflix is the PTS of each vidoe
 frame is not incremented in a constant duration, but with some frames bigger then
 1/23.97 seconds and some are smaller. In Jira NEEDLEPLAT-2057 we have a similar
 processing for the case when source is 29.97fps and output is 59.94hz and for this
 Jira the reason is same. The fix for NEEDLEPLAT-2057 actually caused a side effect
 because it tried to set the initial vsync phase to 0.5, to maximumly avoid the
 problem, but for the 2:3 situation, a 0.5 phase make it worse because the average
 FRC ratio is 2.5, so an initial 0.5 phase will always make the output pattern not
 aligned at 232323 pattern when the duration of each frame is not even. This fix
 is to move the initial phase to 75% so both cases should work fine.
 [Test] Play typical HD source with 23.97fps and observe the playback smoothness.
 Also double check Chimera Boardwalk Twirl Ride from seeking point 02:34:14 and
 check the playback is smooth also.
 Change-Id: I95c35d4ffa563f74b9afa7ae08f7ef22d1227706
 Signed-off-by: Tim Yao <tim.yao@amlogic.com>

 NEEDLEPLAT-3173: [NTS]Frame stuttering on 4K 60fps [1/1]
 [JIRA] NEEDLEPLAT-3173
 [Problem] The test clip is 4K 60fps and the output is 4K
 59.94hz so a frame dropping always happen every 1000 frame. The clip is special
 in that the duration (PTS growing distance) between each frame is not same. The
 frame's PTS is incremened by (1530, 1530, 1440) to get an average of 1500 90K
 unit increase. (1500 = 90k / 60), which equals two 17ms dutation then followed
 by a 16ms duration frame to get an average of 16.6ms duration. The previous
 commit for NEEDLEPLAT-2604 and 2057 are the efforts to solve this problem by
 setting an initial phase, which can solve the problem for the uneven PTS for
 23.97 and 29.97fps cases. However, for 60fps source, the initial phase can not
 solve the problem because we can not maintain 1 vsync 1 frame switching always
 because eventually we need drop a frame for every 1000 frames. It means that
 during the playback, this initial phase will be shiftted. And when the vsync
 switching happens at those uneven PTS boundary, the same problem happens.
 [Solution] The change is a general frame switching improvement to avoid two
 situations: a) In one vsync, there are two frames toggled (frame dropping),
 but the next vsync there is no frame flipped because of its PTS has not arrived.
 This will cause unnecessary frame dropping and what we do is to move the second
 video frame to next vsync window. b) In one vsync, there are no frames be flipped
 (frame repeating), but the next vsync will have more than 1 frame flipped. It's
 also another frame dropping and what we do is to flip the next video frame in
 this vsync instead. So the idea is to make the frame switching at video display
 driver more smooth by avoid uneven frame flipping for each vsync. The timestamp
 of the test clip is not correct, but we can use this method to make output
 sequence smooth and avoid frame dropping.
 [Platform]
 needle,stark
 [Test]
 Verify with the special test clip according to the instruction. Verify
 NEEDLEPLAT-2604 and 2057 also.
Change-Id: I8fd9d70fd1fee561e28b8a0c47bd4be716934ec0
Signed-off-by: shuanglong.wang <shuanglong.wang@amlogic.com>

drivers/amlogic/media_modules/frame_provider/decoder/h264_multi/vmh264.c

@@ -2850,7 +2850,7 @@ static int vh264_vf_states(struct vframe_states *states, void *op_arg)
 
 static struct vframe_s *vh264_vf_peek(void *op_arg)
 {
-	struct vframe_s *vf;
+	struct vframe_s *vf[2] = {0, 0};
 	struct vdec_s *vdec = op_arg;
 	struct vdec_h264_hw_s *hw = (struct vdec_h264_hw_s *)vdec->private;
 
@@ -2863,8 +2863,14 @@ static struct vframe_s *vh264_vf_peek(void *op_arg)
 		return &hw->vframe_dummy;
 	}
 
-	if (kfifo_peek(&hw->display_q, &vf))
-		return vf;
+	if (kfifo_out_peek(&hw->display_q, (void *)&vf, 2)) {
+		if (vf[1]) {
+			vf[0]->next_vf_pts_valid = true;
+			vf[0]->next_vf_pts = vf[1]->pts;
+		} else
+			vf[0]->next_vf_pts_valid = false;
+		return vf[0];
+	}
 
 	return NULL;
 }
@@ -2962,6 +2968,7 @@ static struct vframe_s *vh264_vf_get(void *op_arg)
 		int time = jiffies;
 		unsigned int frame_interval =
 			1000*(time - hw->last_frame_time)/HZ;
+		struct vframe_s *next_vf;
 		if (dpb_is_debug(DECODE_ID(hw),
 			PRINT_FLAG_VDEC_STATUS)) {
 			struct h264_dpb_stru *p_H264_Dpb = &hw->dpb;
@@ -2980,6 +2987,11 @@ static struct vframe_s *vh264_vf_get(void *op_arg)
 		}
 		hw->last_frame_time = time;
 		hw->vf_get_count++;
+		if (kfifo_peek(&hw->display_q, &next_vf)) {
+			vf->next_vf_pts_valid = true;
+			vf->next_vf_pts = next_vf->pts;
+		} else
+			vf->next_vf_pts_valid = false;
 		return vf;
 	}
 

drivers/amlogic/media_modules/frame_provider/decoder/h265/vh265.c

@@ -6248,7 +6248,7 @@ static int vh265_vf_states(struct vframe_states *states, void *op_arg)
 
 static struct vframe_s *vh265_vf_peek(void *op_arg)
 {
-	struct vframe_s *vf;
+	struct vframe_s *vf[2] = {0, 0};
 #ifdef MULTI_INSTANCE_SUPPORT
 	struct vdec_s *vdec = op_arg;
 	struct hevc_state_s *hevc = (struct hevc_state_s *)vdec->private;
@@ -6265,8 +6265,15 @@ static struct vframe_s *vh265_vf_peek(void *op_arg)
 		return &hevc->vframe_dummy;
 	}
 
-	if (kfifo_peek(&hevc->display_q, &vf))
-		return vf;
+
+	if (kfifo_out_peek(&hevc->display_q, (void *)&vf, 2)) {
+		if (vf[1]) {
+			vf[0]->next_vf_pts_valid = true;
+			vf[0]->next_vf_pts = vf[1]->pts;
+		} else
+			vf[0]->next_vf_pts_valid = false;
+		return vf[0];
+	}
 
 	return NULL;
 }
@@ -6360,6 +6367,7 @@ static struct vframe_s *vh265_vf_get(void *op_arg)
 #endif
 
 	if (kfifo_get(&hevc->display_q, &vf)) {
+		struct vframe_s *next_vf;
 		if (get_dbg_flag(hevc) & H265_DEBUG_PIC_STRUCT)
 			hevc_print(hevc, 0,
 				"%s(type %d index 0x%x poc %d/%d) pts(%d,%d) dur %d\n",
@@ -6371,6 +6379,13 @@ static struct vframe_s *vh265_vf_get(void *op_arg)
 
 		hevc->show_frame_num++;
 		hevc->vf_get_count++;
+
+		if (kfifo_peek(&hevc->display_q, &next_vf)) {
+			vf->next_vf_pts_valid = true;
+			vf->next_vf_pts = next_vf->pts;
+		} else
+			vf->next_vf_pts_valid = false;
+
 		return vf;
 	}
 

drivers/amlogic/media_modules/frame_provider/decoder/vp9/vvp9.c

@@ -6456,14 +6456,20 @@ static int vvp9_vf_states(struct vframe_states *states, void *op_arg)
 
 static struct vframe_s *vvp9_vf_peek(void *op_arg)
 {
-	struct vframe_s *vf;
+	struct vframe_s *vf[2] = {0, 0};
 	struct VP9Decoder_s *pbi = (struct VP9Decoder_s *)op_arg;
 
 	if (step == 2)
 		return NULL;
 
-	if (kfifo_peek(&pbi->display_q, &vf))
-		return vf;
+	if (kfifo_out_peek(&pbi->display_q, (void *)&vf, 2)) {
+		if (vf[1]) {
+			vf[0]->next_vf_pts_valid = true;
+			vf[0]->next_vf_pts = vf[1]->pts;
+		} else
+			vf[0]->next_vf_pts_valid = false;
+		return vf[0];
+	}
 
 	return NULL;
 }
@@ -6479,6 +6485,7 @@ static struct vframe_s *vvp9_vf_get(void *op_arg)
 			step = 2;
 
 	if (kfifo_get(&pbi->display_q, &vf)) {
+		struct vframe_s *next_vf;
 		uint8_t index = vf->index & 0xff;
 		if (index >= 0	&& index < pbi->used_buf_num) {
 			pbi->vf_get_count++;
@@ -6488,6 +6495,13 @@ static struct vframe_s *vvp9_vf_get(void *op_arg)
 					vf->width, vf->height,
 					vf->pts,
 					vf->pts_us64);
+
+			if (kfifo_peek(&pbi->display_q, &next_vf)) {
+				vf->next_vf_pts_valid = true;
+				vf->next_vf_pts = next_vf->pts;
+			} else
+				vf->next_vf_pts_valid = false;
+
 			return vf;
 		}
 	}