Consumer cameras are indispensable tools for communication, content creation, and remote work, but image and video quality can be affected by various factors such as lighting, hardware, scene content, face detection, and automatic image processing algorithms. This paper investigates how web and phone camera systems perform in face-present scenes containing diverse skin tones, and how performance can be objectively measured using standard procedures and analyses. We closely examine image quality factors (IQFs) commonly impacted by scene content, emphasizing automatic white balance (AWB), automatic exposure (AE), and color reproduction according to Valued Camera Experience (VCX) standard procedures. Video tests are conducted for scenes containing standard compliant mannequin heads, and across a novel set of AI-generated faces with 10 additional skin tones based on the Monk Skin Tone Scale. Findings indicate that color shifts, exposure errors, and reduced overall image fidelity are unfortunately common for scenes containing darker skin tones, revealing a major short-coming in modern-day automatic image processing algorithms, highlighting the need for testing across a more diverse range of skin tones when developing automatic processing pipelines and the standards that test them.
Accurate characterization (profiling) of a capture system is essential to have the system accurately reproduce the colors in a scene. ISO 17321 [1][2] describe two methods to achieve this calibration. One based on standard reflective targets (chart-based method) and the other on making accurate measurements of the cameras responsivity functions and the spectral power distribution of the deployed illuminant (spectral characterization). The most prominent of the two is the chart-based method for the reason that it involves a simple capture of an inexpensive, standard color pattern (e.g., Macbeth/Xrite Color Checker). However, the results obtained from this method are illuminant specific and are very sensitive to the technique used in the capture process. Lighting non-uniformity on the chart, incorrect framing, and flare can all erroneously affect the results. ISO also recommends a more robust technique, involving the measurement of the camera's responsivity and the spectral power distribution of the capture illuminant. Measurements of these features can require the use of expensive and sophisticated instruments such as monochromators and spectro-radiometers. Both methods involve tradeoffs in cost, ease of use, and most importantly in the accuracy of the final capture system characterization. The results obtained are very sensitive to the technique of capture and precision of measurements of the various parameters involved. The end-user is often left confused asking such questions as, What accuracy is needed in individual measurements?, What are the tradeoffs (particularly in color accuracy) in using the chart-based method vs. the spectral characterization based method?, also, How sensitive is the system to the various parameters? In this study, both of the ISO recommended techniques are utilized for camera calibration on a broad range of professional cameras and illuminants. Such characterization was conducted by approximately ten different users so as to capture the variability of the deployed capture technique. The collected data was used to calculate and quantify the system characterization accuracy using the color inconstancy index for a set of evaluation colors as the metric. Sensitivity analysis techniques were used to attempt to answer the question "How much of an advantage, if any, does the spectral characterization of the camera offer over the chartbased approach?" In answering the question, parameters (and their sensitivities) were identified to most influence the results.