In recent years, image processing has proven to be a great tool to help document, preserve, and restore art pieces, especially visual art. One example is using colorization techniques when the original color information of the image has been lost or is unavailable. To expand on that, we used Ghent Altarpiece as the study case. One of the panels of this polyptych has been lost, and only an archival photograph exists. Using the state-of-the-art colorization method, we want to digitally restore what has been lost in its original form. In this work, we proposed a pipeline that consists of a colorization transformer (ColTran) trained on the captured patch-based imaging data of the Ghent Altarpiece. We evaluated the proposed pipeline and addressed its strengths and weaknesses. Moreover, we presented planned future steps and improvements for this project.