Surveillance video of vehicle camera are widely used to support driver's safe driving, especially for taxi and truck drivers. Long-term videos are often inspected by human operators manually to find dangerous driving events, which is tedious time consuming work. We propose a new method to detect dangerous driving events automatically from the surveillance videos. Events such as rocket start, red light ignored dangerous driving can be detected. In our method, traffic light recognition is made at first. Then speed and acceleration data of car, traffic light recognition results are used as features to detect dangerous driving events. Color and shape of traffic lights are different in different countries and areas. Color and shape of traffic light images are also different at different shooting time, background and weather condition. It is difficult for conventional method to obtain both high recognition rate and low false positive rate. We proposed to use color, shape and context features to recognize traffic light more accurately. Vehicle road testing in both USA and Japan were made to demonstrate effectiveness of our proposed method. Real-time processing recognition experiments were made by vehicle camera video stream. Surveillance videos taken by driving recorder camera were also used to do traffic light recognition and dangerous driving events detection experiments. Traffic light recognition rate of 93%, false positive detection rate of 0.1%, realtime processing time less than 30ms results were obtained by our method.