Video processing algorithms tend to improve over time in terms of image quality while increasing in implementation complexity. Generally, video algorithms are developed and evaluated in isolation from the video processing system of which they will be a part, in a consumer product. The final image quality obtained by that system, however, strongly depends on the interaction of its constituent algorithms. Current methods for optimizing the overall image quality are ad-hoc, time consuming and don't guarantee the best possible result. In this paper we propose a scalable method for optimizing a video, taking into consideration the possibility of adding/removing different components to this video system. Our method utilizes genetic algorithms (GAs), which evolves toward the optimum system configuration (the best image quality). GAs are heuristic optimization search methods, which when driven by an objective video quality metric, finds the optimum system configuration.