Free space is an essential component of any autonomous driving system. It describes the region, which is typically the road surface, around the vehicle which is free from obstacles. However, in practice, free space should not solely describe the area where a vehicle can plan a trajectory. For instance, in a single lane road with two way traffic the opposite lane should not be included as an area where the vehicle can plan a driving path although it will be detected as free space. In this paper, we introduce a new conceptual representation called DriveSpace which corresponds to semantic understanding and context of the scene. We formulate it based on combination of dense 3d reconstruction and semantic segmentation. We use a graphical model approach to fuse and learn the drivable area. As the drivable region is highly dependent on the situation and dynamics of other objects, it remains a bit subjective. We analyze various scenarios of DriveSpace and propose a general method to detect all scenarios. As it is a new concept, there are no datasets available for development and test, however, we are working on creating the same to show quantitative results of the proposed method.