Real-time three-dimensional (3D) medical imaging requires both high memory and high computational bandwidth due to the massive amount of data to be processed. Application-specific systems have been designed to speed up a few selected 3D imaging algorithms. When applied to other algorithms, all such systems require complicated redesign procedures or get lower performance than expected. In this article, we present an FPGA-based computing platform that can be used to accelerate a broad range of 3D medical imaging algorithms dominated by local operations. Its generality and reconfigurability make it easy to be customized to differing algorithm requirements. The architecture in the platform exploits intrinsic parallelisms in 3D imaging algorithms in order to achieve high computational bandwidth. Along with the architecture, a new caching scheme called brick caching was designed to dynamically partition the input data and buffer them in distributed internal caches to provide multiple memory accesses. The caching scheme exploits locality of reference in three dimensions, thus greatly reducing the total data flow from external system memory to internal caches. It is also a deterministic caching method, which allows the input data to be prefetched into the processor before they are processed. We also present application of the platform in FDK (Feldkamp–Davis–Kress) cone-beam computed tomography (CT) image reconstruction so as to demonstrate its potential in real-time 3D medical imaging.