We used deep neural network image analysis to automatically extract head pose angles—roll, yaw, pitch (or tilt)—and figure display length (quarter-length, half-length, full-length) from 11,000 digital images of portrait paintings in a wide variety of styles, from the early Renaissance through Modern eras. We tracked trends and exposed anomalies in such formal properties of these portraits, which sheds light upon the social and aesthetic forces to which portrait artists respond. For example, we find that the so-called Primitive or Naive portraitists favor a highly restricted range of pose angles (primarily frontal) while Expressionist, Mannerist, and Ukiyo-e portraitists employ a far greater range of angles. We also analyzed these formal properties to reveal the different trends throughout the careers of several individual artists, such as Paul Cezanne, Edouard Manet, and Francisco Goya. Our methods can be expanded to incorporate additional computed visual and contextual information—such as genders and ages of figures—and thus form a foundation for addressing a large range of problems in the history of art.