Using smartphone images to quantify color presents a noninvasive way to assess jaundice and other color-related biomarkers of the human body. Here we focus on assessing jaundice through accurate bilirubin measurement in adult liver patients, the first time optical imaging has been applied to this cohort. These patients can suffer from very high levels of bilirubin, indicating their severity of liver disease. A smartphone assessment technique for jaundice based around the color of the sclera (white of the eye) extracted from images is being developed, as smartphone imaging enables cheap, non-invasive and quantitative readings. Variations in ambient light cause large changes to recorded pixel values so must be accounted for to ensure that any changes detected are due to changes in jaundice level. Here we suggest the use of an ambient subtraction approach to minimise the effects of ambient light. Pairs of flash/ no-flash images are captured and the extracted values subtracted to yield data as though under a pure flash illumination. We present data demonstrating the technique with a group of healthy adult volunteers. We also present data from a patient study involving adults with liver disease. Images were captured and the bilirubin (jaundice) level predicted from these images before and after subtraction was compared to the ground truth value obtained via blood test. The linear correlation coefficient increased from 0.47 to 0.85 (p < 0.001 in both cases) upon application of subtraction, demonstrating the effectiveness of the technique.