Subjective-Quality-Optimized Complexity Control for HEVC Decoding

Ren Yang, Mai Xu, Lai Jiang and Zulin Wang 

The School of Electronic and Information Engineering, Beihang University, China

{yangren, maixu, jianglai.china, wzulin}@buaa.edu.cn

 

 

Abstract

    The latest High Efficiency Video Coding (HEVC) standard significantly improves coding efficiency over H.264/AVC, at the cost of heavy encoding and decoding complexity. For reducing HEVC decoding complexity to a target, we propose in this paper a Subjective-Quality-Optimized Complexity Control (SQOCC) approach, which optimizes subjective quality loss caused by the decoding complexity reduction. First, a saliency detection method in HEVC domain is developed as the preliminary of subjective quality metric. Based on detected saliency, we establish a formulation to minimize subjective quality loss at the constraint of specific decoding complexity reduction, via disabling the deblocking filters of some Largest Coding Units (LCUs). Next, we utilize least square fitting to model functions in our formulation. We then provide a solution to our formulation, achieving subjective-quality-optimized complexity control for HEVC decoding. Finally, the experimental results show the effectiveness of our SQOCC approach in terms of both control accuracy and subjective quality.

---

Motivation & Basic Idea

    The past decade has witnessed an explosive growth of video data delivered over Internet. Such a growth poses a great challenge on the efficiency of video coding. To significantly improve the efficiency of video coding, the High Efficiency Video Coding (HEVC) standard was approved in April, 2013 (Sullivan et al., 2012). It has been verified that HEVC can reduce the bit-rate to around 50% with similar visual quality, compared with its former standard H.264/AVC. However, the cost for the high coding efficiency is much more computational complexity (Bossen et al., 2012). This cost becomes a great obstacle of video encoding and decoding on portable devices with limited power.

    Besides, Decoding is far more common than encoding for existing coding standards including HEVC. For example, the amount of videos encoded and uploaded to YouTube is around 65 thousands every day, while that figure is 100 millions for decoded and viewed videos per day, more than 1,500 times of the encoded videos (Dubey et al., 2014). Therefore, the study on complexity reduction is more urgent for decoding, in the latest HEVC standard.

    Also, the portable devices have recently become diverse in computational capability and battery capacity. For example, the computational capability of laptops (e.g, MacBook) is probably over twice higher than that of tablets (e.g, iPad). Besides, the battery also varies across different devices, in terms of capacity. In fact, HEVC decoding may adapt to the diverse computational capability and battery capacity. That is, decoding complexity can be reduced to a target, via complexity control in HEVC decoding. Complexity control of HEVC decoding is a promising goal in the field of decoding complexity reduction. In this paper, we propose an efficient approach to achieve this goal.

    Fig. 1 shows an example for application of our SQOCC approach. When the devices is with sufficient power (eg. 80%), the video can be decoded with full fidelity. Once the battery is insufficient (eg. 77%, 73% and 68%), the video is decoded with low complexity (complexity is reduced in the dark areas), according to the left battery. However, it incurs the quality degradation. The subjective quality should be maximized in quality degradation for the proper Quality of Experience (QoE), according to Region-of-Interests (ROI).

Fig. 1  An example for application of our SQOCC approach.

---

Fromulation for SQOCC Approach

1. Preliminary

    To optimize subjective quality by taking advantage of ROI, we proposed an HEVC domain saliency detection algorithm. Our method uses two features to detect LCU level saliency values ( ) in HEVC compressed domain: bit allocation and the contrast of bit allocation with its neighbors. Fig. 2 shows an example of our saliency detection method, and it can be seen that the detected result is tally with the saliency grountruth (from eye-tracking experiment) well.

                                                                          

Fig. 2  An example for our saliency detection method

    It has been verified that deblocking filters consume 13%-17% of decoding complexity (Bossen et al., 2012). Hence, the decoding complexity can be reduced by disabling the deblocking filters of some LCUs, at the expense of visual quality loss.  indicates whether the deblocking filter of the n-th LCU is disabled ( ) or enabled ( ). 

2. Formulation and modelling

    Our SQOCC approach aims at reducing decoding complexity to the target, meanwhile favoring subjective quality. Here, given the saliency detection result,   and   are defined as the subjective quality loss and complexity reduction of the n-th LCU in a frame caused by disabling deblocking filters, and  is the target of complexity reduction.The optimization of SQOCC can be expressed by the following formulation:

In the formulation,   and  can be calulated by               

    Then seven randomly selected training sequences from JCT-VC database are used for modelling the fromer functions. The modelling results are:
             

where N is the total number of LCUs in a frame. It obvious that when the deblocking filters are enabled ( ), there will be no quality loss and complexity reduction. As such, the functions are only modelled when the deblocking filters are disabled ( ). The training curves are shown is Fig. 3 and Fig. 4, and the coefficients a, b and l are shown in Table 1.

                     

Fig 3. The fitting cureves of  .

                     

Fig 4. The fitting cureves of  .

              Table 1 Values of the coefficients                

3. Solution for SQOCC formulation

    Our SQOCC formulation is an linear integer programming problem, and it can be solved by some mathematical programming methods. Fig. 5 shows an example of the solution. It can be seen that the decoding complexity of LCUs in non-ROI is reduced in high priority. This indicates that the subjective quality loss can be minimized by applying our SQOCC approach.

                       

Fig 5. An example of the solution to SQOCC formulation

---

Experimental results

1. Complexity control accuracy

Table 2 Actual complexity values and the control errors

                     

2. Complexity-distortion performance

                                       

Fig 6. Complexity-distortion performance of PartyScene at QP = 32

3. Subjective quality

Fig 7. Subjective quality performance of Kimono at QP = 32

    

    Fig. 7 shows one selected frame of Kimono at QP = 32, decoded by HM 16.0 with our approach and the conventional Nogues' approach. Clearly, our approach produces higher visual quality in face regions than Nogues', even when the complexity reduction is much more in our approach. Besides, the non-ROI background is with low quality in our approach, as there exist obvious blocky effects. This also indicates that our approach is capable of optimizing subjective quality by maintaining high quality in ROIs at the expense of low quality in non-ROIs.

---

References

1. Ren Yang, Mai Xu, Lai Jiang and Zulin Wang. Subjective-Quality-Optimized Complexity Control for HEVC Decoding. International Conference on Multimedia and Expo (ICME) 2016.

2. (In Submission) Ren Yang, Mai Xu, Zulin Wang and Xiaoming Tao. Saliency-Guided Complexity Control for HEVC Decoding. IEEE Transactions on Image Processing.