Glare due to sunlight, moonlight, or other light sources can be a serious impediment during autonomous or manual driving. Automatically detecting the presence, location, and severity of such glare can be of critical importance for an autonomous driving system, which may then give greater priority to other sensors or cues/parts of the scene. We present an algorithm for automatic real-time glare detection that uses a combination of: (1) the intensity, saturation, and local contrast of the input frame; (2) shape detection; and (3) solar azimuth and elevation computed based on the position and heading information from the GPS (used under daylight conditions). These data are used to generate a glare occurrence map that indicates the center location(s) and extent(s) of the glare region(s). Testing on a variety of daytime and nighttime scenes demonstrates that the proposed system is effective at glare detection and is capable of real-time operation.