The rise of cheaper and more accurate genotyping techniques has lead to significant advances in understanding the genotype-phenotype map. However, this is currently bottlenecked by manually intensive or slow phenotype data collection. We propose an algorithm to automatically estimate the canopy height of a row of plants in field conditions in a single pass on a moving robot. A stereo sensor pointed down collects a series of stereo image pairs. The depth images are then converted to height-above-ground images to extract height contours. Separate height contours corresponding to each frame are then concatenated to construct a height contour representing one row of plants in the plot. Since the process is automated, data can be collected throughout the growing season with very little manual labor complementing the already abundantly available genotypic data. Using experimental data from seven plots, we show our proposed approach achieves a height estimation error of approximately 3.3%.