This article introduces a novel algorithm to learn optimal incident illumination for material classification using spectral bidirectional reflectance distribution function (BRDF) images. The method performs a joint selection of incident angle and spectral band in two steps: (1) clustering and selecting incident angles using statistics on the spectral BRDF images for a specific material, and (2) searching for the optimal angles and spectral bands that maximize material discriminability, which we measure in classification performance. The benefits of reducing the number of incident illumination angles include improving material classification, reducing computational time and storage, and allowing for a less cumbersome and potentially mobile imaging system. The authors show that their approach provides comparable material classification performance when using a reduced number of incident illuminations as compared with when using a larger number. They also compare their approach with prior work. © 2015 Society for Imaging Science and Technology.