This paper presents a subjective image quality experiment design that specifically targets understanding consumer perception of visual quality. With this consumer-oriented goal in mind, images were captured from a large number of representative consumer devices. This design choice is fundamentally different from previous work in the field, in that it is not based on simulated software impairments, but rather on naturally occurring artifacts due to the device and conditions. New subjective questions were posed, which allowed exploration of differences in the quality of experience (QoE) provided by various devices. This paper presents the experimental design implemented to capture ratings and feedback on these consumer devices. The study is analyzed to highlight and demonstrate the unique capabilities provided by this method.