Evaluating the utility of polarimetric imaging for material identification, as compared to conventional irradiance imaging, motivates this work. Images of diffuse objects captured with a wide field of view Mueller matrix polarimeter are used to demonstrate a classification and measurement optimization method. This imaging study is designed to test polarimetric utility in discriminating white fabric from white wood. The material color is constrained to be similar so that classification from only total radiance imaging is difficult, i.e., metamerism. A statistical divergence between two distributions of measured intensity is used to optimize the Polarization State Generator (PSG) and the Polarization State Analyzer (PSA) given two classes of Mueller matrices. The classification performance as a function of number of polarimetric measurements is computed. This work demonstrates that two polarimetric measurements of white fabric and white wood offer nearly perfect classification. The utility and design of partial Mueller imaging is supported by this optimization of PSG/PSA states and number of measurements. c 2020 Society for Imaging Science and Technology.