Efficient plant phenotyping methods are necessary in order to accelerate the development of high yield biofuel crops. Manual measurement of plant phenotypes, such as width, is slow and error-prone. We propose a novel approach to estimating the width of corn and sorghum stems from color and depth images obtained by mounting a camera on a robot which traverses through plots of plants. We use deep learning to detect individual stems and employ an image processing pipeline to model the boundary of each stem and estimate the pixel and metric width of each stem. This approach results in 13.5% absolute error in the pixel domain on corn averaged over 153 estimates and 13.2% metric absolute error on phantom sorghum averaged over 149 estimates.