Color filter array (CFA) based acquisition schemes allow for the acquisition of color images with a single optical sensor and are an integral part of most consumer-level digital camera pipelines. The CFA is a mosaic of color filters overlaid on the optical sensor such that only a single spectral band is acquired at a particular location. There is thus simultaneous sampling in the spectral and spatial dimensions. In this paper we propose a joint spatial-chromatic framework for the design of optimal spectral sensitivity functions for CFA filters. We design optimal sensitivity functions for the Bayer and seven alternate CFA arrangements and demonstrate the superior performance of CFAs with optimized color filters with respect to standard RGB and CMY filters in terms of both perceived quality and the s-CIELab metric.