Colorization of grayscale images is a severely ill-posed inverse problem among computer vision tasks. We present a novel end-to-end deep learning method for the automatic colorization of grayscale images. Past methods employ multiple deep networks, use auxiliary information, and/or are trained on massive datasets to understand the semantic transfer of colors. The proposed method is a 38-layer deep convolutional residual network that utilizes the CIELAB color space to reduce the problem’s solution space. The network comprises 16 residual blocks, each with 128 convolutional filters to address the ill-posedness of colorization, followed by 4 convolutional blocks to reconstruct the image. Experiments under challenging heterogeneous scenarios and using the Imagenet, Intel, and MirFlickr datasets show significant generalization when assessed visually and against PSNR, SSIM, and PIQE. The proposed method is relatively simpler (16 million parameters), faster (15 images/sec), and resource-efficient (just 50000 training images) when compared to the state-of-the-art.