Mobile is increasingly going to be all about video. The major drivers in video growth are mobile devices such as smartphones, which enable people to: watch what I want, when I want, and where I want. However, due to the large data size and intensive computation, video processing consumes a large amount of power, which limits battery life and frustrates mobile users. Mobile system designers typically focus on hardware-level power optimization techniques without considering how hardware performance interfaces with viewer experience. Here we investigate how viewing context factors affect mobile viewer experience, where such factors include viewer movement, viewing distance, and ambient illumination. We connect hardware design techniques to viewer experience to develop a simple but effective decision tree model to enable video system adaption. Finally, we implement a viewer-aware intelligent mobile video system which can optimize power efficiency automatically in real-time according to the viewing context while maintaining the viewer experience. Our research has opened a door for development of future viewer-aware mobile system design, accelerating low-cost mobile devices with longer battery life.