Self-Learning-Based Joint Super-Resolution and Deblocking for a Highly Compressed Image

 -Experiments Analysis and Discussion

 

 

Experiments Analysis

Fig. 1. Our test image dataset.

Table 1 Subjective preference votes (ratios) of the proposed SelfSR method compared to the other three methods: SelfSR, Cascade I (BM3D+SelfSR), and Cascade II (SelfSR+BM3D) for individual test images

 

Table 2 Subjective preference votes (ratios) of the proposed SCSR method compared to the other three methods: SCSR, Cascade I (BM3D+SCSR), and Cascade II (SCSR+BM3D) for individual test images

 

In the subjective paired comparison, the ratio between the preference vote for one method and that for another method is used to compare the performances between the two methods. For example, if one method is preferred 80% of the time, the SR versions of all 9 test images obtained by this method receive 80% of the preference vote, indicating that the method is visually preferred to the compared method in general.

To further analyze our method, we list in Table 1 and 2 the subjective preference votes (and the corresponding ratios in the parentheses) of the proposed method compared to the other three methods for individual test images (see Fig. 1). Note, for each test image, the SR results of two methods are pairwisely compared 20 times by 20 subjects. The highest average preference vote (ratios) between our method and compared methods is highlighted in bold-red, whereas the lowest preference vote is highlighted in bold-black. It is remarkable that the test image with the lowest average preference ratios (80% and 73.3%) for the two proposed methods is image #5. On the other hand, the test images with highest ratios (93.3% and 90%) are image #1 and image #3, respectively.

Table 3 Attributes of Individual Test Images

 

Fig. 2. Examples of image attributes. (a) Strong edges. (b) Weak edges (non-sharp edges at the top and vertical wrinkles at the bottom). (c) Textures. (d) Smooth regions.

Moreover, Fig. 2 and Table 3 show the attributes of individual test images, where each image is associated with one or more out of the four attributes: strong edge, weak edge, texture, and smooth region. The strong-edge attribute indicates that the edges in an image is with high contrast and sharpness, whereas the weak-edge attribute means that the edges in an image is relatively less sharp. The average preference ratios in Table 3 show that the proposed methods perform well for all attributes in general, but the performance for weak edges is not as good as that for the other attributes. Besides, since image #5 contains fewer high-frequency details, it is difficult to retrieve enough high-frequency information from blocking-artifact atoms for effective super-resolution using the proposed methods, thereby making the benefit of our method limited. These are why the subjective rating for the SR versions of image #5 upscaled by the proposed methods are lower than those of the other test images.

Fig. 3. SR results of some image regions: (a) smooth region; (b) region with strong edges; and (c) region with fine textures.

 Compared to image #5, test images #1 and #3 contain richer textures and smooth regions. Since the blocking artifacts are relatively visible in smooth region [e.g., see Fig. 3(a)] and fine textures contain useful high-frequency information for super-resolution, it is easier for the subjects to visually assess the deblocking and super-resolution performances of the compared methods for images #1 and #3. Figs. 3(b) and 3(c) show the reconstructed images with fine details (i.e., high-frequency components), which obviously illustrate the effectiveness of the proposed method.

Finally, we summarize the above analyses as follows:

• Compared to SR without deblocking (i.e., the original SCSR and SelfSR), our methods are significantly more effective for images with strong edges, textures, and smoothing regions.
• Compared to SR with deblocking, our methods achieve significant visual quality improvement for image regions with strong edges and textures. As for smooth regions, since blocking artifacts in such regions can be effectively mitigated using the deblocking part of the cascading-based approaches as well, our methods achieve comparable visual quality with the cascading-based approaches.  
• Compared to existing SR methods (with or without deblocking), our methods in general achieve overall visual quality improvement over the existing methods.