We present an image recovery approach to improve amplitude and phase reconstruction from single shot digital holograms, using iterative reconstruction with alternating updates. This approach allows the flexibility to apply different priors to amplitude and phase, improves phase reconstruction in image areas with low amplitudes, and does not require phase unwrapping for regularization. Phantom simulations and experimental measurements of a grating sample both demonstrate that the proposed method helps to reduce noise and resolve finer features. The improved image reconstruction from this technique will benefit the many applications of digital holography.