Actually in most applications, optical flow is computed from one luminance Y-plane and only a few methods refer to color optical flow. In fact, a brief analysis shows that these methods are either marginal approaches, whether dramatically time consuming techniques. Here, we propose a vectorial approach based upon a joint analysis of a structure tensor and a so called flow tensor, both computed from image derivatives.
B. Augereau, B. Tremblais, C. Fernandez-Maloigne, "Vectorial Computation of the Optical Flow in Color Image Sequences" in Proc. IS&T 13th Color and Imaging Conf., 2005, pp 130 - 134, https://doi.org/10.2352/CIC.2005.13.1.art00025