Sparse coding - modelling data vectors as sparse linear combinations of basis elements - has been widely and successfully used in image classification, noise reduction, texture synthesis, audio processing, etc. Although traditional sparse coding with fixed dictionaries like wavelet and curvelet can produce promising results, unsupervised sparse coding has shown its advantage by optimizing the dictionary based on target data provided. However, most of the existing unsupervised sparse coding method failed to consider the high dimensional manifold information. Recently, graph regularized sparse coding has been proposed to incorporate manifold information. Better classification and clustering results have been shown compared with naive unsupervised sparse coding. The authors utilize modified feature-sign search and Lagrange dual algorithm to solve the objective function as two consecutive convex functions. This method relies on large number of iterations to get state-of-art classification and clustering results, which is computational intensive. In this paper, we proposed a novel modified online dictionary learning method which iteratively utilizes modified least angle regression and block coordinate descent method to solve the problem. Instead of getting entire coefficient matrix then generate dictionary matrix, our method updates coefficient vector and dictionary matrix in each inner iteration. Thus, efficiency and accuracy are reserved at same time.