GSoC 2017: H.264 encoder improvements and EGLImage

With the H.264 encoder component working we moved to the big goal of the project i.e. adding EGLImage support in the H.264 decoder. In the following sections I’ll talk about the work done this week and future goals briefly.

H.264 encoder improvements

We earlier had problem with the input port that it tried to free buffer which was already NULL. The only check it had was for the “is_owned” property (from tizport.c)

if (OMX_TRUE == is_owned)
      OMX_PTR p_port_priv = OMX_DirInput == p_obj->portdef_.eDir
                              ? ap_hdr->pInputPortPrivate
                              : ap_hdr->pOutputPortPrivate;
      free_buffer (p_obj, ap_hdr->pBuffer, p_port_priv);

The is_owned value can’t be set from outside since the struct tiz_port_buf_props_t is defined in tizport.c and not exposed through any header.

The workaround was, as pointed out by Julien, to use super_UseBuffer instead of super_AllocateBuffer inside h264e_inport_AllocateBuffer.  In fact super_Use or super_AllocateBuffer do the same thing except that the latter allocates some memory and sets the flag ”owned” that is later checked while clearing the buffer. This change also has the advantage that it avoids making Tizonia to allocate buffer and to free them , which is inefficient.

Along with that there were some minor improvements too.

EGLImage support

Most of this week was spent on figuring out how to add the EGLImage support to the H.264 decoder component. We went through various approaches and it still isn’t finalised how it will be done but we have made some progress in getting the video_buffers.

From the 2 approaches to get the “struct pipe_resource *” from the EGLImage I used the first approach since, according to Julien, is safer because it calls lookup_egl_image which
seems to go all the way down and internally calls _eglCheckResource so it
does some validity check on the input pointers.

static struct pipe_resource *
st_omx_pipe_texture_from_eglimage_v1(EGLDisplay egldisplay, EGLImage
  _EGLDisplay *disp = egldisplay;
  struct dri2_egl_display *dri2_egl_dpy = disp->DriverData;
  __DRIscreen *_dri_screen = dri2_egl_dpy->dri_screen;
  struct dri_screen *st_dri_screen = dri_screen(_dri_screen);
  __DRIimage *_dri_image = st_dri_screen->lookup_egl_image(st_dri_screen,

  return _dri_image->texture;

The first commit can be found here.

This change only served as getting the pipe_resource. It was also not working since the decoder was trying to make a target from pipe_resource which wasn’t available since the output port was disabled. To prevent that case I added the changes here. This made the error to disappear but still the decoder failed later when trying to end the frame.

Next we dropped how targets were made. After some discussion it was decided to store the video buffers made from the pipe resource and use them as needed.

video_buffer = vl_video_buffer_create_ex2 (p_prc->pipe, &templat, resources);


util_hash_table_set(p_prc->video_buffer_map, p_prc->p_outhdr_, video_buffer);

Another  problem that I was getting that the decoder was getting RGBA EGLImage (1 target) whereas the decoder renders to NV12 which is 2 pipe_texture per frames. The following was added as fix for that.

assert (p_res->format == PIPE_FORMAT_R8G8B8A8_UNORM);

templat.buffer_format = PIPE_FORMAT_R8G8B8A8_UNORM; // RGBA
templat.chroma_format = PIPE_VIDEO_CHROMA_FORMAT_NONE;
templat.width = p_res->width0;
templat.height = p_res->height0;
templat.interlaced = 0;

memset(resources, 0, sizeof resources);
resources[0] = p_res;

video_buffer = vl_video_buffer_create_ex2 (p_prc->pipe, &templat, resources);

The video_buffer is then retrieved later on when h264d_fill_output needs to write to the output.

dst_buf = util_hash_table_get(p_prc->video_buffer_map, output);

The full commit message is here.

Next week’s work

After these changes the decoder compiles and runs without error but fails to write the output to the screen. I found that instead of making more than one video_buffers the decoder makes only one video_buffer. The decoder runs normally when the video isn’t running but when the video starts the main function “h264d_prc_buffers_ready” is never called by the IL core. This is the reason behind the black screen / no output from the decoder. Many parts of the decoder are still to be done/finalised like clearing the video_buffers. Next week we expect to get the decoding to work normally and some more progress.


One comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s