There has been a growing interest in recent years in the development of objective image quality assessment (IQA) models, whose roles are not only to monitor image quality degradations and benchmark image processing systems, but also to optimize various image and video processing algorithms and systems. While the past achievement is worth celebrating, a number of major challenges remain when we apply existing IQA models in realworld applications. These include obvious ones such as the challenges to largely reduce the complexity of existing IQA algorithms and to make them easy-to-use and easy-to-understand. There are also challenges regarding the applicability of existing IQA models in many real-world problems where image quality needs to be evaluated and compared across dimensionality, across viewing environment, and across the form of representations – specific examples include quality assessment for image resizing, color-togray image conversion, multi-exposure image fusion, image retargeting, and high dynamic range image tone mapping. Here we will first elaborate these challenges, and then concentrate on a specific one, namely the generalization challenge, which we believe is a more fundamental issue in the development, validation and application of IQA models. Specifically, the challenge is about the generalization capability of existing IQA models, which achieve superior quality prediction performance in lab testing environment using a limited number of subject-rated test images, but the performance may not extend to the real-world where we are working with images of a much greater diversity in terms of content and complexity. We will discuss some principle ideas and related work that might help us meet the challenges in the future.