For a robust three-dimensional video transmission through error prone channels, an efficient multiple description coding for multi-view video based on the correlation of spatial polyphase transformed subsequences (CSPT_MDC_MVC) is proposed in this article. The input multi-view video sequence is first separated into four subsequences by spatial polyphase transform and then grouped into two descriptions. With the correlation of macroblocks in corresponding subsequence positions, these subsequences should not be coded in completely the same way. In each description, one subsequence is directly coded by the Joint Multi-view Video Coding (JMVC) encoder and the other subsequence is classified into four sets. According to the classification, the indirectly coding subsequence selectively employed the prediction mode and the prediction vector of the counter directly coding subsequence, which reduces the bitrate consumption and the coding complexity of multiple description coding for multi-view video. On the decoder side, the gradient-based directional interpolation is employed to improve the side reconstructed quality. The effectiveness and robustness of the proposed algorithm is verified by experiments in the JMVC coding platform.

The acoustic-based automatic speech recognition (ASR) technique has been a matured technique and widely seen to be used in numerous applications. However, acoustic-based ASR will not maintain a standard performance for the disabled group with an abnormal face, that is atypical eye or mouth geometrical characteristics. For governing this problem, this article develops a three-dimensional (3D) sensor lip image based pronunciation recognition system where the 3D sensor is efficiently used to acquire the action variations of the lip shapes of the pronunciation action from a speaker. In this work, two different types of 3D lip features for pronunciation recognition are presented, 3D-(x, y, z) coordinate lip feature and 3D geometry lip feature parameters. For the 3D-(x, y, z) coordinate lip feature design, 18 location points, each of which has 3D-sized coordinates, around the outer and inner lips are properly defined. In the design of 3D geometry lip features, eight types of features considering the geometrical space characteristics of the inner lip are developed. In addition, feature fusion to combine both 3D-(x, y, z) coordinate and 3D geometry lip features is further considered. The presented 3D sensor lip image based feature evaluated the performance and effectiveness using the principal component analysis based classification calculation approach. Experimental results on pronunciation recognition of two different datasets, Mandarin syllables and Mandarin phrases, demonstrate the competitive performance of the presented 3D sensor lip image based pronunciation recognition system.

Smartphones are playing an increasingly important role in helping people share information and communicate with each other, as evidenced by an increasing number of applications (or apps), including news, search engine, mailbox, and social networking apps. The more heavily people rely on their phones, the more they begin to take the term “user experience” into consideration when buying a smartphone. Obviously, the key to providing a good user experience is to minimize the response time of each graphical user interface (GUI) operation, such as a touch, slide, or flick. Thus, it is crucial for smartphone manufacturers and mobile app developers to measure the response times to enhance the performance of the device or application. In this article, the authors present the SMArtphone Response Time measuring platform (SMART), a novel and low-cost platform for measuring the response time of a smartphone. SMART consists of a normal 30-fps (frame per second) webcam and a mirror. The proposed platform first simultaneously records the changes on a smartphone screen and the GUI operation shown in the mirror. It then adopts image processing techniques to analyze the latency of the GUI operation. The authors’ validation results show that the measurement error of SMART is less than 1/30 s, which means that SMART is a practical solution for measuring the latency of GUI operations. In addition, they adopted SMART for the measurements and analysis of six different commercial smartphones to further demonstrate its applicability.

In a recent image processing system, convolution operations play a significant role in manipulating image and extracting features from images. Due to the increase of kernel sizes, the image processing hardware suffers from severe hardware complexity and power consumption. In this article, an area-efficient structure is proposed for two-dimensional separable convolution operations. Since a separable convolution allows to translate a two-dimensional convolution into two one-dimensional convolutions, it is possible to compute row-wise and column-wise convolutions independently. Whereas the previous work performs such one-dimensional convolutions in sequence, the proposed structure computes the one-dimensional convolutions simultaneously by rescheduling the computational sequence. Experimental results show that the proposed structure saves approximately 80% and 38% of the hardware resources compared to the conventional and previous structures, respectively.

This article proposes a new technique for impulsive noise removal. This technique consists of two steps: (1) impulsive noise detection, and (2) impulsive noise suppression. The proposed method exploits the effectiveness of Weber Law in detecting and locating the impulsive noise appearing in the corrupted image. The occurrence of impulsive noise is then reduced and suppressed using the Total Variation-based approach with the detected noise map obtained from the Weber Law detector. As documented in the Experimental section, the proposed method offers promising results in terms of visual investigation. In addition, it also gives superior results compared to that of the former competing schemes under objective assessment. Thus, it can be regarded as a good candidate for impulsive noise removal algorithm.

Multilayer light field three-dimensional displays are becoming popular due to their full resolution reconstruction and easy fabrication by utilizing existing display technologies such as liquid crystal display (LCD) panels. However, these displays still suffer from limited performance, achieving low angular resolution, narrow field of view, and small depth of field. One of the recent research ideas focusing on overcoming these limitations is perceptual quality improvement. But, currently introduced methods consider only specific issues/applications such as moiré fringe effect, and near-eye display technology. In this work, the authors propose a novel method of approximating light field data for dual-layered light field display considering the Human Visual and Perceptual System. The authors’ display configuration includes two liquid crystal panels with uniform backlight with no time multiplexing. It is not necessary for LCD panels to be parallel. For a wide field of view configuration, the authors introduce a quadratic penalization term to reduce ghost effects caused by neighboring views. This leads to an improved perceptual approximation of a given light field and increases the possibility of usage in design with a wider field of view configuration.

Human Visual System (HVS) contains some important psychophysical characteristics and the most important among those are the contrast sensitivity function and luminance adaptation (LA) effect, that can potentially be used for modeling better image quality assessment (IQA) methods. As a result, some existing IQA methods have intrinsically adopted the Weber’s law model to reflect the LA effects. But the authors have found that the way of adopting the LA effect was inappropriate. Recent psychophysical studies reveal that the Weber’s law model is unable to be precisely fitted in the measured data for the LA effect. Furthermore, most IQA methods attempt to measure the quality in the pixel intensity domain, while the Weber’s law works in the luminance domain and pixel intensity domain has a nonlinear relationship with the luminance domain. To address these problems, the authors derive a new LA effect model in the pixel intensity domain. Based on their derived LA effect model, the authors propose a new weight pooling method for the LA effect, denoted as LA-based Local weight Function (LALF). Their comprehensive experimental results prove that LALF can easily be applied in existing FR-IQA methods and it significantly improves the performance of the base methods in which LALF is applied.

In this study, a new CT measurement system was developed for preoperation evaluation of total knee replacement (TKR). To validate the developed imaging processing systems, TKR patients with and without a bowed lower limb were recruited. All patients underwent plain radiographs and CT scan before a TKR procedure. The precision and effectiveness of two measurements validated by an orthopedic surgeon and evaluated using the developed 3D computed tomography image system were compared. In vivo statistics data, particularly for Taiwanese patients, showed no difference between bowed lower limb and normal lower limb groups. The results of this study did not reveal a significant difference between the 3D-reconstructed CT images and 2D plain radiographs. However, the proposed 3D-reconstructed CT image measurement system demonstrated consistent measurements for TKR patients and for the bowed lower limb group in preoperative evaluation. The implemented system could be used in conjunction with the current 2D radiograph approach. The proposed system assist surgeons and patients when discussing operations and clinical approaches. Clinical results validated the system.