Having a methodology for assessing smartphone camera image quality is advantageous for both those who design and develop the cameras as well as those who use them. Camera engineers need to quickly and reliably assess the impact of the system decisions they make. Smartphone customers who are armed with a quantitative understanding of the image quality can include this information to make informed decisions between products. This research project was undertaken to develop a procedure for evaluating pictorial image quality for smartphone camera captures. Experiments were conducted to evaluate tone quality, color quality, and sharpness and noise using images captured with 20 cameras that were released primarily in the period between 2012 to late 2014. A variety of scenes were captured with each device. In each test, observers rated the test images for overall quality and then for a specific image quality characteristic using an anchored scaling experimental protocol. The results indicated high correlations between the individual characteristics and overall quality. It was also determined that high correlations could be achieved between the visual results and objective measurements for sharpness and noise. Both analyses indicated that a two-step process in which devices are first sorted into categories of high and low quality followed by a second sort to further refine device quality may be required to successfully predict the visual results.