Cryo-electron microscopy (Cryo-EM) is a popular imaging modality used to visualize a wide range of bio-molecules in their 3D form. The goal in Cryo-EM is to reconstruct the 3D density map of a molecule from projection images taken from random and unknown orientations. A critical step in the Cryo-EM pipeline is 3D refinement. In this procedure, an initial 3D map and a set of estimated projection orientations is refined to obtain higher resolution maps. State-of-the-art refinement techniques rely on projection matching steps in order to refine the initial projection orientations. Unfortunately projection matching is computationally inefficient and it requires a finite discretization of the space of orientations. To avoid repeated projection matching steps, in this work we consider the orientation variables in their continuous form. This enables us to formulate the refinement problem as a joint optimization problem that refines the underlying density map and orientations. We use alternating direction method of multipliers (ADMM) and gradient descent steps in order to update the density map and the orientations, respectively. Our results and their comparison with several baselines demonstrate the feasibility and performance of the proposed refinement framework.