A real-time violent behavior detection algorithm based on a new descriptor is proposed. This descriptor reflects a common observation that the changes in both the magnitude and direction of movement in violent images are more abrupt than non-violent ones. During several frames, descriptor feature vectors consisting of descriptor values are generated, and they are inputs to the Support Vector Machine (SVM) classifier for discriminating violent actions from non-violent actions. Comparison experiments among the Motion Binary Pattern (MBP), the Violent Flow (ViF) and the proposed algorithm were conducted with three different types of datasets. In all datasets, the proposed algorithm was above 80% in the F-measure and outperformed the other methods in every case.