GSoC 2017: Vid enc working

This week we got the encoder but not without some workarounds. I’ll talk briefly about the fixes for the last week’s problems, the improvements still to be made and the future plan below. The commits can be seen here.

Registering  Multiple components

This blocking issue was fixed this week thanks to Juan’s support. The root of the problem was that Tizonia does not have specialised loaders to load multiple components. You can check out Juan’s much in depth answer here. So instead, we had to provide a generic name to the component and add the other “components” as roles of that component. The following code from entrypoint.c registers the generic component name:

/* Initialize the component infrastructure */
tiz_comp_init (ap_hdl, OMX_VID_COMP_NAME);

And you set the roles for the components as usual. The difference here is that the roles do not have names like they do in bellagio. For example, in vid_dec.c the three video dec roles have different names:

strcpy(comp->name, OMX_VID_DEC_BASE_NAME);
strcpy(comp->name_specific[0], OMX_VID_DEC_MPEG2_NAME);
strcpy(comp->name_specific[1], OMX_VID_DEC_AVC_NAME);
strcpy(comp->name_specific[2], OMX_VID_DEC_HEVC_NAME);

strcpy(comp->role_specific[0], OMX_VID_DEC_MPEG2_ROLE);
strcpy(comp->role_specific[1], OMX_VID_DEC_AVC_ROLE);
strcpy(comp->role_specific[2], OMX_VID_DEC_HEVC_ROLE);

So in gst-omx we can directly use the role name to select the component role as

component-name=OMX.mesa.video_decoder.mpeg2

which is the value of OMX_VID_DEC_MPEG2_NAME.

For the tizonia based component we have to provide both component name and the role, which is also more accurate:

component-name=OMX.mesa.video.all
component-role=video_decoder.avc

So with these changes the problem with loading multiple components was almost solved. Almost because we still haven’t tested the per role hook registration API which I’ll talk about later below.

Working H.264 encoder

The encoder is able to encode the video. There was a need to add new ports with some changes in functionality. The h264einport and h264eoutport are derived from tizavcport and tizvideoport respectively. The main need for them is that the h264einport replaces the pBuffer of the buffer header with a custom pointer. The replacement occurs here:

FREE((*buf)->pBuffer);
r = enc_AllocateBackTexture(ap_hdl, idx, &inp->resource, &inp->transfer, &(*buf)->pBuffer);

The other changes are regarding management of this custom pointer.

Workarounds

The encoder still uses some workarounds which will need to be addressed later on.

Freeing the buffer pointer

The new pBuffer pointer can’t be simply cleared with free(). Trying to do so causes an error to be thrown. In the bellagio based encoder simply setting the buffer to NULL solves the problem.

buf->pBuffer = NULL;

return super_FreeBuffer(typeOf (ap_obj, "h264einport"), ap_obj, ap_hdl, idx, buf);

Inside super_FreeBuffer, a check makes sure that it doesn’t try to clear an already empty buffer:

if(openmaxStandPort->pInternalBufferStorage[i]->pBuffer){

But in tizonia there doesn’t exist any check for the same. From tizport.z

if (OMX_TRUE == is_owned)
    {
      OMX_PTR p_port_priv = OMX_DirInput == p_obj->portdef_.eDir
                              ? ap_hdr->pInputPortPrivate
                              : ap_hdr->pOutputPortPrivate;
      free_buffer (p_obj, ap_hdr->pBuffer, p_port_priv);
    }

The only property it checks is the is_owned property which is internal to tizport.c. To avoid that I used this patch in the dev environement

commit f3e3f40611129c9d3f942b05b7eb66e37d198ace
Author: Gurkirpal Singh <gurkirpal204@gmail.com>
Date: Mon Jul 17 00:12:06 2017 +0530

tizport: check ap_hdr->pBuffer exists before clearing

diff --git a/libtizonia/src/tizport.c b/libtizonia/src/tizport.c
index 58bac53..8d48b3a 100644
--- a/libtizonia/src/tizport.c
+++ b/libtizonia/src/tizport.c
@@ -1187,7 +1187,9 @@ port_FreeBuffer (const void * ap_obj, OMX_HANDLETYPE ap_hdl, OMX_U32 a_pid,
OMX_PTR p_port_priv = OMX_DirInput == p_obj->portdef_.eDir
? ap_hdr->pInputPortPrivate
: ap_hdr->pOutputPortPrivate;
- free_buffer (p_obj, ap_hdr->pBuffer, p_port_priv);
+ if (ap_hdr->pBuffer) {
+ free_buffer (p_obj, ap_hdr->pBuffer, p_port_priv);
+ }
}

p_unreg_hdr = unregister_header (p_obj, hdr_pos);

Another way to avoid the crash could be to provide a fake buffer by allocating memory to the pBuffer with malloc() / calloc() just before releasing. What approach to use we’re still deciding upon it.

Releasing the input header

Another problem that arose while adding the encoder was clearing the input header. With bellagio, like in vid_enc.c you can set

priv->BufferMgmtCallback = vid_enc_BufferEncoded;

where vid_enc_BufferEncoded writes to the output buffer. The omx_base_filter_BufferMgmtFunction takes care of providing the buffers and calling the BufferMgmtCallback at the right time.

The following call sends the input buffer to the management function

return base_port_SendBufferFunction(port, buf);

Tracing the program, I found that the vid_enc_BufferEncoded is called two times before the buffer is cleared. The first time the whole function runs. The second time it hits

if (!inp || LIST_IS_EMPTY(&inp->tasks)) {
input->nFilledLen = 0; /* mark buffer as empty */
enc_MoveTasks(&priv->used_tasks, &inp->tasks);
return;
}

which marks the buffer to be cleared.

With tizonia, the component has to implement it’s own buffer management. Currently the component doesn’t uses queue for holding buffers like bellagio internally does so it has to clear the the buffer before requesting a new one. To provide similar functionality I made changes to call h264e_manage_buffers until the condition “if (!inp || LIST_IS_EMPTY(&inp->tasks))” is satisfied in this patch. Unfortunately this didn’t quite work as expected. While running it with gst-launch with this pipeline

gst-launch-1.0 filesrc location=~/mp4_tests/10\ Second\ countdown.mp4.mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! omxh264enctiz ! h264parse ! avdec_h264 ! videoconvert ! ximagesink

The video runs for a bit then stops running and hangs. I tried debugging it with gdb, setting breakpoints on all functions in h264eprc.c and adding the command “continue” to all of them so that I could just check how the control flows. Strangely while debugging in this manner the video never hanged. Instead it reached the end and gave some port flush errors. Again trying it without gdb I got same results. This made it really hard to debug. In the end I reverted the commit to try other things. The crashes caused by it didn’t help either.

The second approach I used was to just move the block that clears the buffer at the end. The commit can be found here. This assumes that the inp->tasks is empty which has been in the case from the bellagio st/omx/h264enc traces. Still this could be considered as a workaround which might need work later. With this change the video plays.

What’s next?

Since the enc is almost in working state, next week, Julien will review it while I’ll focus around adding supporting EGLimage support. Juan is working on adding a per role EGLImage registration API that would allow different roles to have different hooks. Julien and Christian provided inputs on how the task could be accomplished.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s