An Experimental Comparative Analysis of Spectral Reconstruction Methods for Multispectral Camera Systems

Majid Ansari-Asl; Tanzima Habib; Jon Yngve Hardeberg

doi:10.2352/J.ImagingSci.Technol.2026.70.2.020402

Abstract

This paper presents a comprehensive experimental evaluation of spectral reconstruction methods in multispectral imaging systems, focusing on two multispectral camera technologies with differing spectral characteristics: spectral filter array and filter wheel. These systems were assessed under a controlled LED-based illumination setup. A range of reconstruction methods, encompassing both model-based and training-based approaches, were analyzed in their baseline forms as well as in adaptive configurations, which select optimal local training subsets based on spectral reflectance or camera response similarity. Experiments were conducted using a custom-built imaging setup and two well-characterized spectral reflectance datasets: the standard Munsell and the Munsell Student Color sets.

Results demonstrate that training-based methods significantly outperform model-based methods in both spectral and colorimetric accuracy. Adaptive dataset selection further enhances performance in many cases, particularly for the SpectroCam filter wheel camera. The influence of illumination on reconstruction accuracy is also examined, revealing that model-based methods are especially sensitive to the spectral power distribution of the light source. These findings offer practical and technical guidance for the design and calibration of multispectral imaging systems aimed at achieving high-accuracy spectral recovery.

jist

JIMTE6

Journal of Imaging Science and Technology

J. Imaging Sci. Technol.

1062-3701

1943-3522

Society for Imaging Science and Technology

020402

10.2352/J.ImagingSci.Technol.2026.70.2.020402

2025008

Work Presented at Electronic Imaging 2026

An Experimental Comparative Analysis of Spectral Reconstruction Methods for Multispectral Camera Systems

Spectral reconstruction in multispectral cameras

Ansari-AslMajid

HabibTanzima

HardebergJon Yngve

NTNU - Norwegian University of Science and Technology, Faculty of Information Technology and Electrical Engineering, Department of Computer Science, Teknologivegen 22, 2815 Gjøvik, Norway

majid.ansari-asl@ntnu.no

Ansari-Asl, Habib, and Hardeberg

032026

882025

1512026

2026

Abstract

spectral reconstructionspectral recoveryreflectance estimationmultispectral imagingmultispectral camera.

ccc

1062-3701/2026/70(2)/020402/22/$25.00

printed

Printed in the USA

Introduction

Multispectral imaging has emerged as a pivotal technology with applications spanning fields such as remote sensing [1], medical imaging [2], cultural heritage [3], and material analysis [4]. To ensure the accuracy and consistency of measurements, radiometric calibration is indispensable for these systems as it establishes the relationship between the physical scene and the recorded image values. Accurate radiometric calibration enables devices to deliver reproducible measurements, fostering reliable comparisons across different imaging systems.

In the context of conventional cameras, radiometric calibration is typically understood to involve two interrelated components: calibration and characterization. Calibration refers to configuring the imaging system to exhibit repeatable and reproducible behavior under well-defined conditions. Characterization, by contrast, establishes the mapping between the device-dependent color space and the standard CIE colorimetric system [5, 6]. The latter is commonly referred to as colorimetric or conventional characterization, whereas spectral characterization is defined as the measurement or estimation of the camera’s spectral sensitivity functions [7].

Multispectral imaging systems, although sharing fundamental optical principles with conventional cameras, often require distinct radiometric calibration approaches due to their higher-dimensional data, nonuniversal sensor designs, diverse filter layouts, and spectral channel scanning mechanisms. By extending the characterization framework established for conventional color imaging devices, existing studies on multispectral cameras can be broadly categorized into two primary approaches.

The first approach, referred to as multispectral camera colorimetric characterization, treats a multispectral camera analogously to a conventional camera by deriving a transformation matrix that maps the device-dependent sensor space to a device-independent colorimetric space, such as CIE XYZ [8–11]. This process typically involves reducing the higher-dimensional multispectral data to a lower-dimensional representation.

The second approach, multispectral camera spectral characterization, reframes the problem as spectral reconstruction, also known as spectral recovery or spectral estimation [7, 8, 12–14]. In this case, the objective is to map the recorded multispectral measurements to a higher-dimensional, device-independent spectral reflectance space. This approach is particularly well suited to applications requiring detailed spectral information. For example, Bauer et al. employed a variant of Wiener estimation to reconstruct spectral reflectance from Spectral Filter Array (SFA) cameras [13]. Klein et al. considered it as the measurement of camera spectral sensitivity functions and as a prerequisite for both color calibration and spectral estimation [12, 14], and further regarded spectral characterization as the complete process culminating in spectral reflectance reconstruction. The present study primarily focuses on this second approach, emphasizing spectral reconstruction while incorporating colorimetric evaluation as a complementary analysis.

The literature on spectral reconstruction methods is extensive and can be broadly classified into two strategies: model-based reconstruction and training-based reconstruction. Model-based reconstruction methods, also known as physical-model-based estimations or direct methods, leverage the known physical properties of the camera system. These methods derive a forward model that maps spectral data to the camera’s responses, and an inverse model is used to estimate spectral reflectances. Notable examples include methods based on Wiener estimation [15, 16].

Conversely, training-based reconstruction methods, often referred to as empirical-model-based estimations or indirect methods, avoid reliance on the physical properties of the camera. Instead, they utilize empirical data and learning-based techniques to approximate the camera’s behavior in converting spectral data to its responses. These methods, including the Moore–Penrose pseudoinverse [17], typically involve training with known datasets and may incorporate physical constraints to enhance accuracy. A detailed overview of this strategy and associated methods is provided in [18].

Although traditional spectral reconstruction methods train on the entire dataset, recent studies suggest that using a sample-specific subset, known as the local optimal training dataset, can lead to more accurate estimations [19–24]. This subset consists of spectral reflectances from the training data that are most similar or closest to the test sample. The enhanced performance of reconstruction methods using such local datasets has been demonstrated particularly in multispectral imaging contexts [20, 24–26]. These approaches are generally categorized into two groups: those that identify the closest samples in the spectral reflectance space [25] and those that search within the camera response space [24, 26].

Given the critical importance of accurate spectral reconstruction, this paper offers a comparative analysis of current methods, with a focus on practical considerations. By synthesizing insights from both model-based and training-based reconstruction strategies, as well as local dataset selection techniques, this work aims to deepen the understanding of radiometric calibration and spectral reconstruction for imaging systems based on multispectral cameras and LED light sources. The main contributions of this paper are summarized as follows:

∙

investigation of various multispectral-to-spectral reflectance reconstruction methods,

∙

exploration of different local optimal training dataset selection strategies,

∙

analysis of multispectral camera technologies,

∙

discussion of practical and technical considerations in designing imaging systems that utilize multispectral cameras.

The remainder of the paper is structured as follows. Section 2 introduces the image formation model for multispectral cameras, details both model-based and training-based spectral reconstruction approaches, and explains the local optimal dataset selection techniques and evaluation protocols. Section 3 outlines the imaging setup, including specifications of the multispectral cameras, analysis of the light sources, and the imaging geometry. It also discusses technical challenges encountered during the experiments, such as image capture issues and optimizing camera exposure times to address dynamic range and bracketing constraints. Section 4 presents the experimental results in tables and graphs, categorized for improved readability and comparison, and provides a detailed discussion of the results, challenges, and key findings. Finally, Section 5 concludes the paper and outlines potential directions for future research.

Algorithms and Methodology

2.1

Problem Formulation

The mathematical formulation of multispectral imaging parallels that of conventional tristimulus imaging, with the key distinction being its higher number of spectral channels. The camera response for a given stimulus can be expressed as

(1)

ρ_{i} = g_{i} \int_{λ} s_{n} (λ) i (λ) r (λ) d λ + n_{i} .

Here, ρn represents the response of the camera’s nth channel while r(λ) and i(λ) denote the spectral reflectance of the object and the spectral power distribution (SPD) of the light source, respectively. The variable λ corresponds to the wavelength, and sn(λ) is the spectral responsivity of the nth channel. For a multispectral camera with N channels, there exist N spectral responsivities, collectively represented as sn(λ) ∈{s1(λ), s2(λ), …, sN(λ)}.

In Eq. (1), the gain parameter gi is a coefficient that converts the result of the integral into digital counts, representing the camera output. This parameter depends on the imaging system configuration, including factors such as imaging geometry and the SPD of the light source. Furthermore, ni denotes the imaging system noise, which is modeled as white noise with zero mean and variance ai, and is assumed to be uncorrelated with the spectral reflectance [24, 26, 27].

This relationship can be reformulated in matrix notation as

(2)

ρ = g S I R + n .

In this expression, the variables are defined as follows:

∙

N, P, and J denote the number of channels, the number of samples, and the number of wavelengths, respectively.

∙

ρ

is an N × P matrix representing the camera responses for P recorded samples, where each sample contains N values corresponding to the camera sensors.

∙

g is an N × N diagonal matrix representing the gain parameters of the camera spectral channels.

∙

S is an N × J matrix containing the spectral responsivities of the camera’s sensors.

∙

I is a J × J diagonal matrix representing the SPD of the illumination.

∙

R is a J × P matrix containing the spectral reflectances of P samples, with each reflectance characterized over J wavelengths.

The problem of reflectance reconstruction involves estimating the spectral reflectances R from the measured multichannel responses

ρ

. By introducing a matrix M, defined as the product of the gain parameter, the camera responsivities, and the illumination SPD, M = gSI, Eq. (2) can be simplified as

(3)

ρ = M R + n .

Assuming a noiseless system, if M were known and invertible, the reconstruction of spectral reflectance would reduce to solving a straightforward inverse problem:

(4)

\hat{R} = M^{- 1} ρ,

where

\hat{R}

represents the estimated spectral reflectance. However, in practice, M is often not invertible, and this direct solution is not robust especially in the presence of noise. Consequently, alternative approaches are required to achieve accurate and stable spectral reflectance reconstruction.

2.2

Spectral Reconstruction Methods

The estimation of spectral reflectance from multispectral data has been actively investigated in the literature for more than five decades [15, 28]. The central goal is to find a transformation matrix W that maps the measured multispectral data to the spectral reflectance space as expressed in

(5)

\hat{R} = W ρ .

2.2.1

Model-Based Methods

Model-based spectral reconstruction methods are applicable in scenarios where the imaging model, comprising the camera’s spectral responsivities and the illumination spectrum, is known. This corresponds to the case in which the matrix M in Eq. (3) is given. Since the transformation is from a higher-dimensional spectral space to a lower-dimensional multispectral space, the system is underdetermined and admits infinitely many solutions. The following methods are employed to address this problem.

Minimum Norm Approach.

A common approach for solving the underdetermined camera model is the minimum norm method [17]. This technique seeks a solution that not only satisfies the measurement equation (Eq. (3)) but also minimizes the Euclidean norm of the reconstructed reflectance. Its mathematical formulation is as follows:

(6)

W = M^{T} {(M M^{T})}^{- 1} = {(M)}^{+},

where (M)+ denotes the Moore–Penrose pseudoinverse of M. This method offers a fast, closed-form solution and is particularly useful when no prior knowledge or training data is available. It is hereafter referred to as MINN.

Wiener Estimation.

Wiener estimation methods are among the most widely used and dependable approaches for spectral reconstruction, particularly when the camera model parameters are available [25, 27]. In Wiener estimation, the reconstruction matrix W in Eq. (5) is derived such that the mean squared error (MSE) between the estimated and the original spectra is minimized. Given the camera response model in Eq. (2), the matrix W is computed as

(7)

W = C_{R} M^{T} {(M C_{R} M^{T} + a)}^{- 1},

where CR denotes the autocorrelation matrix of the training spectra, defined as the expected value of the product of the spectral reflectance matrix and its transpose (viz., E[RRT]), and a is the N × N diagonal matrix representing the variance of the camera noise. This method is hereafter referred to as WINE.

Linear Minimum Mean-Squared Error.

The linear minimum mean-squared error (LMMSE) estimator [27] assumes a linear relationship between the spectral reflectance of objects in a scene and their corresponding camera responses. It is essentially an extension of the conventional Wiener estimator, also based on minimizing the MSE between the estimated and true spectral reflectances. Unlike the Wiener estimator, however, the LMMSE introduces an additional constant vector to account for bias in the reconstruction. Accordingly, compared to Eq. (4), the LMMSE formulation is given by

(8)

\hat{R} = W ρ + b,

where b is a constant vector with the same dimensionality as the spectral reflectances. Let

ρ_{0}

and R0 denote the mean camera responses and their corresponding mean spectral reflectances, respectively. Then, the estimated spectral reflectance from the camera responses can be computed as

(9)

\hat{R} = R_{0} + C_{P} M^{T} {(M C_{P} M^{T} + a)}^{- 1} (ρ - M R_{0}),

where CP is the autocorrelation matrix of the mean-centered spectral reflectances defined as

(10)

C_{P} = E [(R - R_{0}) {(R - R_{0})}^{T}] = \frac{(R - R_{0}) {(R - R_{0})}^{T}}{P},

with P denoting the number of spectral reflectance samples.

Principal Component Estimation.

The principal component (PC) estimation method reconstructs the spectral reflectance by projecting the measured reflectance spectra onto a low-dimensional subspace defined by the leading eigenvectors of the reflectance covariance matrix. Let V ∈RJ×K denote the matrix of the first K principal components (eigenvectors), and let V0 ∈RJ be the mean spectrum of the training set. For a given test reflectance R ∈RJ, its principal component coordinates C ∈RK are computed as

(11)

C = V^{T} (R - V_{0}) .

The estimated spectral reflectance is then given by

(12)

\hat{R} = V_{0} + V C .

This method minimizes the mean squared error between the original and estimated spectra in the least-squares sense. This method assumes that reflectance spectra lie approximately within a low-dimensional linear subspace and are most effective when the training set is sufficiently representative of the test spectra [29].

Taking inspiration from [29], where this simple method was modified to estimate spectral reflectances from tristimulus values, this method is expanded to fit spectral reflectance estimation from multispectral camera responses. Then principal component coordinate C, in this case, is given by

(13)

C = {(gMV + a)}^{- 1} (ρ - ({gMV}_{0} + a)) .

The estimated spectral reflectance can be derived by substituting Eq. (13) in Eq. (12). This method is hereafter referred to as PCAE.

Weighted Principal Component Estimation.

The principal component estimation method described in the previous section was further modified by Agahian et al. by adding weights to the training reflectance set R [30]. The weighted reflectance Rw is then given by

(14)

R_{W} = W_{R} R^{T},

where WR is a diagonal matrix with weights as follows:

(15)

W_{R} = [\begin{matrix} w_{1} \\ ⋱ \\ w_{P} \end{matrix}] .

The weights can be calculated by finding the inverse of the Euclidean distance calculated between the test camera responses and the training camera responses such that if the Euclidean distance is 0, then the weight becomes 1.

The principal components are then computed from the weighted covariance matrix of RW. This method is hereafter referred to as WPCA.

The system-dependent selection of the optimal number of PCs for the PCAE and WPCA methods is described in Section 3.7.

2.2.2

Training-Based Methods

In cases where the imaging system model, including the spectral characteristics of the camera and illumination, is unknown, training-based methods are employed to estimate spectral reflectances from camera responses. In this approach, a mapping function from the camera response space to the spectral reflectance space, which is typically of higher dimensionality, is learned using a set of paired camera responses and spectral reflectances acquired as training data. These methods do not necessarily assume a linear relationship between camera responses and spectral reflectances, which allows them to be effective even when nonlinear processes have been applied to the camera signals. Although numerous training-based spectral reconstruction methods have been proposed, including regression-based and neural-network-based approaches, this work focuses on representative baseline techniques; interested readers are referred to [18, 24, 31] for comprehensive surveys. In what follows, we briefly describe the training-based methods used in this study. Specifically, the pseudoinverse (denoted as PINV) and second-order polynomial estimation (denoted as POLY) methods are employed as representative training-based spectral reconstruction techniques. These methods are widely adopted baseline approaches that respectively model linear and low-order nonlinear relationships while maintaining interpretability and enabling controlled experimental analysis.

Pseudoinverse Method.

Assuming a linear relationship between the camera response space and the spectral reflectance space leads to one of the most fundamental data-driven solutions for spectral reconstruction. This method minimizes the squared reconstruction error over the training set. Accordingly, the solution is given by the Moore–Penrose pseudoinverse as follows:

(16)

W = R_{t} ρ_{t}^{T} {(ρ_{t} ρ_{t}^{T})}^{- 1} .

Here, Rt and

ρ_{t}

represent the training datasets, and T denotes matrix transposition. Although effective, this method is often enhanced using dimensionality reduction techniques [32–36], which help reduce computational complexity and improve performance.

Second-Order Polynomial Estimation.

The pseudoinverse method can be modified using polynomial regression [37]. In this case, the multispectral camera responses

ρ

are expanded to its second-order polynomial form and the least-squares fit is then used to estimate spectral reflectance. Higher-order expansions increase model flexibility but can lead to overfitting. Therefore, in this work, a second-order polynomial fit is chosen, which is a commonly employed technique in the domain. This choice limits the number of terms well below the number of training samples, ensuring stable model estimation while maintaining sufficient flexibility. Let

ρ_{t}^{'}

be the second-order polynomial expansion of the training multispectral camera responses. The new basis W′ is given by

(17)

W^{'} = R_{t} ρ_{t}^{' T} {(ρ_{t}^{'} ρ_{t}^{' T})}^{- 1} .

2.3

Optimal Training Dataset Selection

Unlike global methods, which apply a fixed transformation or model across all inputs, adaptive spectral reconstruction methods dynamically tailor the estimation process to each individual sample [24, 25]. This adaptability often enhances both accuracy and robustness. These methods typically involve identifying a subset of training samples that are most similar to the target input and performing reconstruction using only this localized data. In this study, we employ two adaptive strategies: one that selects samples based on spectral similarity and another based on similarity in camera response space. Both approaches are detailed in Sections 2.3.1 and 2.3.2.

2.3.1

Selection in Spectral Reflectance Space

Shen et al. [25] introduced an adaptive Wiener estimation method for multispectral imaging. This approach improves reconstruction accuracy by adaptively selecting training samples closer to the test sample for autocorrelation matrix calculation.

In Shen’s method, the spectral similarity between the estimated reflectance

\hat{R}

and each training reflectance Ri is measured using a combined distance metric:

(18)

\begin{matrix} d_{i} & = & α mean \{∥\frac{R_{i}}{∥ R_{i} ∥} - \frac{\hat{R}}{∥ \hat{R} ∥}∥\} \\ + (1 - α) max \{∥\frac{R_{i}}{∥ R_{i} ∥} - \frac{\hat{R}}{∥ \hat{R} ∥}∥\}, \end{matrix}

where α is a scaling factor, ∥x∥ denotes normalization, and di indicates the spectral distance. Training samples with smaller di values are deemed more similar and selected. These samples are weighted based on their proximity using

(19)

q_{i} = [\frac{d_{L}}{d_{i}} + 0.5],

where qi represents the repetition count for the ith sample, dL is the distance of the farthest acceptable sample, and di is the distance of the ith sample. In the remainder of this paper, this method is denoted as specLO.

2.3.2

Selection in Camera Space

In adaptive methods that utilize training sample selection in the spectral space, the objective is to identify and employ the spectrum closest to the reflectance of a given test sample during the reconstruction process. Accordingly, for each set of test camera responses, the typical procedure begins by estimating the reflectance using all available training reflectances. The closest reflectances to this initial estimation are then selected for use in the final reconstruction.

However, Tominaga et al. [24] argue that this initial estimation is inherently affected by reconstruction errors, which may compromise the efficiency of the selection process. To mitigate this issue, they propose an alternative strategy that performs the search in the camera response space, thereby avoiding the potential inaccuracies introduced by the initial spectral reconstruction.

In their approach, the entire set of training reflectances is first transformed into corresponding camera responses using the camera model. Then, the distance between the test camera response and each simulated training response is computed. The reflectances associated with the closest responses are subsequently used for spectral reconstruction.

Assuming MRi as the simulated camera response (neglecting noise) corresponding to the ith training reflectance Ri for i = 1,2, …, P, the distance from a given test camera response

ρ

is computed as

(20)

L_{i} = {∥ρ - M R_{i}∥}_{2}^{2} (i = 1, 2, \dots, P),

where ∥⋅∥2 denotes the Euclidean (L2) norm defined as ∥⋅∥2 = z12 + z22 + ⋯ + zN2. In the subsequent step, the Li values are sorted and the K closest samples in the camera space are selected. Their corresponding reflectances are then used for the final spectral reconstruction. This approach is referred to as camLO throughout the paper.

2.4

Evaluation Metrics

To evaluate the performance of spectral reconstruction methods, several widely used metrics are employed. Colorimetric accuracy is assessed using CIEDE2000, which provides improved perceptual uniformity compared to earlier color difference formulas. Spectral accuracy is evaluated using the root mean squared error (RMSE) and the goodness-of-fit coefficient (GFC), capturing both magnitude and shape differences between spectra. Although other metrics such as PSNR, mean absolute error, ΔE76, and ΔE94 are also reported in the literature and were computed during the analysis, they are omitted here to limit redundancy and maintain conciseness as they exhibited trends consistent with the selected measures. The following provides a brief description and mathematical formulation of each metric.

2.4.1

CIEDE2000 (ΔE00)

The CIEDE2000 color difference metric (ΔE00) is a refinement over previous models (e.g., ΔE76 and ΔE94), designed to better align with human visual perception [38]. It introduces corrections for lightness, chroma, and hue differences as well as compensations for neutral colors and the interaction between chroma and hue. The CIEDE2000 formula is defined as

(21)

\begin{matrix} Δ E_{00} & = & [{(\frac{Δ L^{'}}{k_{L} S_{L}})}^{2} + {(\frac{Δ C^{'}}{k_{C} S_{C}})}^{2} + {(\frac{Δ H^{'}}{k_{H} S_{H}})}^{2} \\ {+ R_{T} \cdot (\frac{Δ C^{'}}{k_{C} S_{C}}) \cdot (\frac{Δ H^{'}}{k_{H} S_{H}})]}^{1 ∕ 2} . \end{matrix}

Here, the notation is defined as follows:

∙

ΔL′, ΔC′, and ΔH′ are the differences in lightness, chroma, and hue between two color samples.

∙

SL, SC, and SH are weighting functions for lightness, chroma, and hue.

∙

kL, kC, and kH are parametric correction terms typically set at 1 under reference viewing conditions.

∙

RT is a rotation term accounting for the interaction between chroma and hue, especially in the blue region.

This metric is widely used as one of the most accurate measures for quantifying perceived color differences in color science and imaging applications.

2.4.2

Root Mean Squared Error

The RMSE quantifies the average squared difference between the estimated and reference spectra:

(22)

RMSE = \sqrt{\frac{1}{n} \sum_{i = 1}^{n} {({\hat{R}}_{i} - R_{i})}^{2}},

where Ri is the ground truth reflectance at wavelength i and

{\hat{R}}_{i}

is the estimated reflectance.

The RMSE is typically calculated separately for each reconstructed test sample, and its average over a set of test samples is obtained by summing all individual RMSE values and dividing by the number of samples in the test set. However, Tominaga et al. [24] proposed computing the average RMSE as the square root of the average of the squared norm of the estimation error per wavelength over all test samples. Therefore, this performance metric is called the root average mean squared error (RAMSE) in this paper, which calculates the square root of the average of the mean squared errors across all test samples. It is defined as follows. Given N test samples and each spectrum having n wavelengths, let

{\hat{R}}^{(j)}

and R(j) denote the estimated and ground truth spectra, respectively, of the jth sample:

(23)

RAMSE = \sqrt{\frac{1}{N} \sum_{j = 1}^{N} \frac{1}{n} {∥{\hat{R}}^{(j)} - R^{(j)}∥}_{2}^{2}},

where ⋅ 2 denotes the Euclidean (L2) norm and

(24)

{∥{\hat{R}}^{(j)} - R^{(j)}∥}_{2}^{2} = \sum_{i = 1}^{n} {({\hat{R}}_{i}^{(j)} - R_{i}^{(j)})}^{2} .

Here, the notation is defined as follows:

∙

N is the number of test samples.

∙

n is the number of wavelengths per spectrum.

∙

{\hat{R}}_{i}^{(j)}

and Ri(j) are the estimated and true reflectance values at wavelength i, respectively, for sample j.

2.4.3

Goodness-of-fit Coefficient

The GFC measures the spectral shape similarity using cosine similarity [39]:

(25)

GFC = \frac{\sum_{i = 1}^{n} {\hat{R}}_{i} R_{i}}{\sqrt{\sum_{i = 1}^{n} {\hat{R}}_{i}^{2}} \sqrt{\sum_{i = 1}^{n} R_{i}^{2}}} .

The GFC values span from 0 to 1, with 1 representing an exact spectral match. When the reconstructed spectrum is normalized, the GFC corresponds to the cosine of the angle between the original and reconstructed spectra, treated as a 61-dimensional vector (see Section 4 for details on spectral truncation and sampling).

Experiments

In this work, two distinct multispectral cameras are utilized, each based on a different imaging technology: the Silios CMS-C, which employs an SFA architecture, and the Pixelteq SpectroCam, a multispectral filter wheel camera. Table I presents a comprehensive comparison of their key specifications. These cameras along with the Konica Minolta CS-2000 telespectroradiometer (TSR), used as the reference measurement device, were integrated into a 0/45 imaging geometry.

Table I.

Summarized specifications of cameras.

Specification	Silios CMS-C	SpectroCam VIS + NIR
Technology	SFA	Filter wheel
Spectral range (nm)	430–700	350–1050
Spectral channels	8 + 1*	8
Sensor technology	CMOS	CCD
Resolution	426 × 339	2456 × 2058
Pixel pitch (μm)	5.3 × 3	3.45
Bit depth	10	12
Dimension (mm)	62 × 62 × 31	136 × 124 × 105
Weight (g)	110	680

8 color bands + 1 panchromatic.

In this setup, the LED illumination (see Section 3.3 for detailed spectral specifications of the lighting system) is positioned perpendicular to the sample surface while all the three measurement devices are placed at a 45-degree angle relative to the sample surface normal. This configuration is implemented based on the imaging system introduced in [40, 41]. Figure 1 illustrates the experimental setup used in this work, with the primary components annotated in the image. In subsequent sections, a detailed description of each camera is provided, emphasizing their key features and explaining how they were configured to achieve optimal performance for the experimental requirements.

Figure 1.

The experimental setup used in this study for capturing camera responses and reference measurements for the STUD dataset (see Section 3.5 for details).

3.1

Silios CMS-C Camera

The Silios CMS-C [42] is a multispectral imaging system that utilizes an SFA integrated with a standard CMOS sensor. This camera employs a Bayer-like mosaic filter positioned over a sensor with a native resolution of 1280 × 1024 pixels. However, the mosaic’s 3 × 3 configuration reduces the effective spectral image resolution to 426 × 339 macropixels. The device captures data across eight discrete narrowband channels and one broad panchromatic band. The narrowband filters cover central wavelengths from 430 nm to 700 nm, with Gaussian-like transmission profiles, each having an average full width at half maximum of 40 nm. These filters are centered at specific wavelengths: 440 nm, 473 nm, 511 nm, 549 nm, 585 nm, 623 nm, 665 nm, and 703 nm. Meanwhile, the panchromatic band provides consistent sensitivity across the visible spectrum. The sensor features a 5.3 μm pixel pitch and delivers 10-bit digital output.

The quantum efficiency of the Silios CMS-C camera’s spectral channels is shown in Figure 2(a) [13]. In this camera, each pixel is designed to capture light from a specific spectral band, as the filters are integrated directly on the sensor. However, imperfections in the filters and optical system can cause light from one spectral band to overlap with adjacent channels, leading to inaccuracies in the recorded spectral and spatial data. This issue, known as crosstalk, is typically corrected numerically during post-processing. However, this correction is not addressed in this paper. Instead, Fig. 2(b) presents the quantum efficiency of the spectral channels multiplied by the SPD of the employed LED source. This figure illustrates how crosstalk affects the spectral responsivity of the channels, with secondary peaks becoming comparable in magnitude to the main spectral peak under LED illumination. The impact of this issue is further discussed in Section 4.

Figure 2.

Spectral specification of the Silios CMS-C camera: (a) quantum efficiency of spectral channels [13]; (b) quantum efficiency of spectral channels multiplied by the SPD of the LED.

3.2

Pixelteq SpectroCam Camera

The SpectroCam (VIS + NIR) multispectral camera [43] utilizes a silicon-based CCD panchromatic sensor capable of detecting wavelengths ranging from 350 nm to 1050 nm. A motorized filter wheel, positioned in front of the sensor, accommodates up to eight interchangeable filters. Users have access to over 145 commercially available interference filters with diverse spectral properties [44]. The spectral transmittance characteristics of the filters pre-installed in the camera are illustrated in Figure 3. These filters are centered at wavelengths of 375 nm, 425 nm, 475 nm, 525 nm, 570 nm, 625 nm, 680 nm, and 930 nm. The first seven filters have a bandwidth of approximately 50 nm while the last filter features a broader bandwidth of around 100 nm. The camera outputs images with a resolution of 2456 × 2058 pixels and a pixel pitch of 3.45 μm. Its digital output supports up to 12-bit precision.

Figure 3.

Spectral specification of the SpectroCam filter wheel camera: (a) normalized sensor responsivity; (b) normalized transmittance of filters. Data provided by the manufacturer.

The spectral sensor responsivity and filter transmittances initially installed in the camera are depicted in Fig. 3. All the eight filters are exhibited; however, Filter UV and Filter IR, shown in gray and black in Fig. 3, were omitted from the experiment due to inadequate light intensity.

3.3

LED Light Source

The light source in our setup integrates multiple components to ensure efficient and controlled illumination. As detailed in our previous work [41], at the core of our light source is the Thorlabs MCWHL6 LED, emitting cold white light at 6500 K with an output power of 1430 mW, mounted on a PCB for stability and thermal management. Heat dissipation is achieved using a heat sink while a Thorlabs SM2F32-A adjustable collimation adapter is employed to produce collimated light, essential for accurate spectral imaging and surface data capture. Light intensity is regulated via the upLEDTM LED Driver, offering fine control with a USB interface and C++ SDK support. This configuration provides uniform, collimated lighting tailored to our application requirements.

Figure 4 presents the results of radiometric analysis of the LED. The SPD measured at the precise imaging geometry and location is shown in Fig. 4(a). The Thorlabs MCWHL6 LED was selected for its exceptional reconstruction capabilities and advantageous technical attributes, including higher CCT and output power, as detailed in [41]. Figure 4(b) reveals slight inhomogeneity in the light source across the surface of the sample holder table in the measurement setup shown in Fig. 1, with illumination intensity generally decreasing as the distance from the table center (marked 0 on the X-coordinate) increases. To mitigate this variability, all measurements were conducted at the center of the sample holder table.

Figure 4.

Specification of the Thorlabs MCWHL6 mounted LED used as the light source in this work: (a) the SPD of MCWHL6 mounted LED spectrum; (b) spatial distribution of light source intensity; (c) consistency of the LED with time.

To assess the LED’s temporal stability, its SPD was recorded every minute over six hours. As shown in Fig. 4(c), the radiance decreases by about 1% within the first 75 minutes after activation before stabilizing. Consequently, a warm-up period was included before measurements. Additionally, spectral analysis indicates minimal change, with a shift of approximately 1 nm over the six-hour measurement period.

3.4

Camera Dynamic Range and Radiometry

Dynamic range represents the ratio between the maximum and minimum light intensities that a camera sensor can accurately capture. Essentially, it reflects the sensor’s ability to preserve details in both high- and low-light regions of a scene. In multispectral imaging, dynamic range is crucial because spectral bands may have vastly different camera responsivities due to differences in sensor quantum efficiency, filter transmission, and lens imperfections. Furthermore, nonuniform illumination across the spectral range from the light source creates significant intensity variations that the camera must accommodate in different channels.

Each spectral band has a unique dynamic range that determines its usable range for a given exposure time. The overall dynamic range required in an experiment is influenced by the light source’s SPD, the reflectance properties of the samples, and the imaging setup. If the required dynamic range exceeds the camera’s capability, high dynamic range (HDR) imaging is necessary. Otherwise, exposure time adjustments can ensure sufficient dynamic range overlap.

Managing exposure time is essential for effective multispectral imaging. Some spectral bands may require shorter exposure times to avoid saturation while others may need longer exposure times to avoid noise. The Silios and SpectroCam cameras handle exposure time differently. While the Silios camera applies a single exposure time across all channels, the SpectroCam allows for channel-specific adjustments.

For the Silios camera, the dynamic range was evaluated through a stepwise process that follows the usual practice in the field. First, dark noise was measured by capturing ten images in a dark room with no light source or ambient illumination, and the average image histogram revealed an average dark noise of about ten digital counts in the 10-bit output (0–1023 range). Next, a Spectralon reflectance standard, representing the maximum diffuse reflectance, was imaged. The exposure time was empirically adjusted so that all spectral bands remained below saturation levels. The reflectance of the least reflective sample was then measured to confirm that its values exceeded the dark noise across all bands. This confirmed that the required dynamic range could be covered with a single exposure time, eliminating the need for HDR imaging.

3.5

Datasets

Two datasets were used to evaluate the performance of the spectral reconstruction methods. The first contains 264 samples of the Munsell Student Color Set (STUD), measured using the Konica Minolta CS2000 reference spectroradiometer, along with the corresponding camera responses captured by both the SpectroCam and Silios multispectral cameras. This dataset was employed in the evaluation of both model-based and training-based reconstruction methods. To obtain the camera responses, each sample was imaged ten times, and the resulting images were averaged to reduce random noise. Following this, dark noise was subtracted from the averaged image. The pixel values corresponding to the same spatial area measured by the Konica Minolta CS2000 were then spatially averaged to produce the final camera response for each sample, a commonly adopted practice intended to reduce sensor noise and mitigate local spatial nonuniformities in radiometric measurements.

The second dataset is the full standard Munsell (MUNS) dataset, comprising 1269 spectral reflectances, which was also used in the experiments. However, it could only be utilized for evaluating the model-based methods as it does not include the corresponding camera responses.

All spectral reflectance data were truncated to the 400–700 nm range and sampled at 5 nm intervals, resulting in spectral vectors of 61 dimensions per reflectance.

The spectral reflectance datasets used for method evaluation are illustrated in Figure 5: the STUD dataset, which was measured in the frame of the present study, along with its gamut in the CIE a∗–b∗ space (a and b); the MUNS dataset and its associated gamut (c and d), which were obtained from previously published data [37].

Figure 5.

Spectral reflectance datasets used for method evaluation: (a, b) Munsell Student Color Set, measured alongside their corresponding camera responses, and its gamut in the CIE a∗–b∗ space; (c, d) full standard Munsell dataset and its corresponding gamut.

3.6

Camera Gain and Noise Measurement

The camera gain and noise parameters were measured following the method proposed in [27] with minor modifications. In this approach, a Spectralon diffuse white reference sample is imaged by the cameras under the system’s illumination. The gain parameter is then estimated by minimizing the L1 and L2 norm errors between the theoretical and observed mean camera responses. Similarly, the noise variance is estimated by minimizing the L1 and L2 norm errors between the theoretical and empirical covariance matrices of the pixel values. In this work, the L1 norm was employed, and both the gain and noise variance were calculated separately for each spectral channel of both cameras. This is necessary because in multispectral cameras, the channel-specific settings can vary significantly.

3.7

Optimal Number of Principal Components

The number of PCs used in the PCAE and WPCA methods has a significant impact on spectral reconstruction accuracy and is dependent on the imaging system. In this study, the optimal number of PCs was determined experimentally for each camera instead of adopting values from prior work. Specifically, the baseline PCA estimation method (PCAE without local dataset selection) was evaluated by varying the number of PCs over all feasible values using the STUD and MUNS datasets independently. The reconstruction performance was assessed using the median RMSE, maximum RMSE, and RAMSE metrics, and the optimal number of PCs was selected as the value minimizing these error measures. Consistent results were obtained across both datasets, yielding an optimal choice of five PCs for the SpectroCam and four PCs for the Silios camera. These values were subsequently used throughout all analyses in this work.

Results and Discussion

The experiments were conducted in two stages. In the first stage, each reconstruction method was evaluated over a range of values for the local training dataset size parameter K, representing the number of closest samples used for training. For each method, the value of K yielding the minimum RAMSE, as defined in Eq. (23), was selected for use in the second stage. The RAMSE was chosen as the optimization criterion at this stage because it provides a global spectral error measure by aggregating squared errors across all test samples, which is preferred in the literature over per-sample RMSE followed by statistical summarization [24]. Furthermore, as spectral accuracy is the primary focus of this work, a spectral-domain metric was prioritized. Employing multiple metrics for parameter selection would substantially increase computational complexity. Consequently, the comparative evaluation in the second stage is based on the best-performing configurations of each method. In both stages, performance evaluation is carried out using Leave-One-Out Cross-Validation (LOOCV) [45]. The LOOCV is a robust validation approach in which each data point is used once as a test sample while the remainder form the training set. This method offers a nearly unbiased performance estimate and is especially useful for small datasets, such as those in multispectral imaging, where collecting larger datasets is often cumbersome. Its comprehensive use of available data ensures reliable and generalizable results.

4.1

Determining the Optimal Number of Closest Neighbors

Figures 6 and 7 present the performance trends of various methods, which are categorized into four groups. In these figures, the vertical axis denotes the RAMSE while the horizontal axis indicates the number of closest neighbors K. The minimum of each curve identifies the best performance point for each method. Figure 6(a) illustrates the performance of both model-based and training-based methods using the SpectroCam camera, where both training and testing are performed on the STUD dataset. Each method is evaluated with different local dataset selection strategies: “None” (original method using the entire dataset), “camLO” (local optimal selection in camera space), and “specLO” (local optimal selection in spectral space). For consistency, each method is plotted with a distinct line color and style in figures. The ”None” methods appear as horizontal lines since they are evaluated with training on the full dataset.

Figure 6.

RAMSE estimation errors of spectral reconstruction methods, each combined with different local optimal dataset selection strategies, plotted as a function of the number of closest training samples K. Results are obtained using the SpectroCam camera. (a) Both training and testing are performed on the STUD dataset. The evaluated methods include both model-based and training-based approaches. (b) Training is performed on the MUNS dataset and testing on the STUD dataset, where only model-based methods are evaluated. For improved clarity, the methods are grouped into two or three subfigures. Zoomed-in views are included where necessary to enhance readability.

Figure 7.

RAMSE estimation errors of spectral reconstruction methods, each combined with different local optimal dataset selection strategies, plotted as a function of the number of closest training samples K. Results are obtained using the Silios camera. (a) Both training and testing are performed on the STUD dataset. The evaluated methods include both model-based and training-based approaches. (b) Training is performed on the MUNS dataset and testing on the STUD dataset, where only model-based methods are evaluated. For improved clarity, the methods are grouped into two or three subfigures. Zoomed-in views are included where necessary to enhance readability.

Figure 7(a) displays results similar to those in Fig. 6(a), using the Silios camera. Here too, training and testing are performed on the STUD dataset. Figures 6(b) and 7(b) show the performance of the SpectroCam and Silios cameras, respectively, when trained on the MUNS dataset and tested on the STUD dataset. As discussed in Section 3.5, training-based methods cannot be used in this scenario due to the lack of corresponding camera responses in the MUNS dataset. Therefore, only model-based methods are included in these figures. For better clarity, a zoomed-in view highlighting the region near the minimum RAMSE is also provided in the figures where necessary to enhance readability.

As shown in Figs. 6(a) and 7(a), the POLY method consistently outperforms the other approaches. The WINE and LMMSE models exhibit similar performance trends and produce comparable results. Although LMMSE is theoretically expected to yield lower estimation errors than WINE [27], this advantage primarily holds when the model is trained on a full dataset with a sufficiently large number of samples [24]. However, the plots indicate that the superiority of LMMSE over WINE does not generalize across different cameras: it consistently holds for the Silios camera, but the opposite trend is observed for the SpectroCam. Investigating the underlying cause of this discrepancy is beyond the scope of this paper.

Another important observation is that methods using camLO selection converge to the performance of the original method when K equals the full dataset size. This occurs because in camLO, the selected local dataset becomes identical to the full dataset at that point. In contrast, specLO does not replicate the original dataset exactly as it involves repeated use of the closest samples, which leads to a dataset with a higher density of similar samples. Consequently, specLO can potentially yield lower errors than the original method, even when K is maximal.

4.2

Comparison of Methods at Optimal Performance

After determining the optimal number of closest training samples K, each method was executed using its corresponding best K, that is, each method was evaluated in its optimal configuration. The experimental results are presented in Tables II and III. As outlined in Section 4.1, the methods are categorized into four groups based on the camera and training dataset. Accordingly, Tables II and III report the performance of both model-based and training-based methods using SpectroCam and Silios cameras, respectively. In these tables, methods with both training and testing conducted on the STUD dataset are shown above a double-line separator while those with training on the MUNS dataset and testing on the STUD dataset appear below it. The best performance (viz., the minimum error) within each category is highlighted in bold for each metric and statistical measure, and model-based and training-based approaches are distinguished by a single solid line separator.

Table II.

The estimation errors of spectral reconstruction methods, each combined with different local optimal dataset selection strategies, evaluated using their optimal number of closest training samples K. Results are obtained using the SpectroCam camera. Methods with both training and testing conducted on the STUD dataset are shown above the double-line separator while those with training on the MUNS dataset and testing on the STUD dataset appear below it. The best performance (the minimum error) within each category is highlighted in bold for each metric and statistical measure, and model-based and training-based approaches are distinguished by a single solid line separator.

Table III.

The estimation errors of spectral reconstruction methods, each combined with different local optimal dataset selection strategies, evaluated using their optimal number of closest training samples K. Results are obtained using the Silios camera. Methods with both training and testing conducted on the STUD dataset are shown above the double-line separator while those with training on the MUNS dataset and testing on the STUD dataset appear below it. The best performance (the minimum error) within each category is highlighted in bold for each metric and statistical measure, and model-based and training-based approaches are distinguished by a single solid line separator.

The tables present the performance of the spectral reconstruction methods under different local dataset selection strategies, with the corresponding optimal training set size shown in the K column. Each table includes ΔE00 under CIE D65 [46] and CIE LED-B5 [47] illuminants as the colorimetric errors and two spectral metrics: RMSE and GFC. These illuminants were selected to provide standardized and widely accepted reference conditions for color difference assessment, facilitating reproducibility and comparison with existing studies. Although the experimental LED source used for image acquisition does not exactly match the spectral characteristics of these standard illuminants, the reconstructed outputs represent device-independent spectral reflectance estimates, allowing colorimetric evaluation under arbitrary illuminants. Furthermore, within the CIE LED illuminant series, CIE LED-B5 exhibits the most similar behavior to the employed LED source in terms of peak wavelength positions and relative amplitude in the blue spectral region.

The first four metrics are reported using four statistics—median, standard deviation (std), 95th percentile, and maximum values—while the RAMSE is included under the RMSE statistics instead of maximum. Following [48, 49], GFC values are categorized into four intervals and reported as the percentage of estimation errors falling within each interval. The intervals are defined as follows: GFC < 0.995 (poor), 0.995 ≤ GFC < 0.999 (accurate), 0.999 ≤ GFC < 0.9999 (good), and GFC ≥ 0.9999 (excellent).

The MINN method is presented separately in the tables and serves as a baseline for comparison. As a simple, interpretable, and closed-form solution—requiring no iterative optimization, training data, or prior assumptions—it provides a strong benchmark for validating the effectiveness of more advanced models.

According to Table II, the best overall performance for the SpectroCam is achieved by the POLY method combined with specLO and trained on the STUD dataset. This configuration yields an RAMSE of 0.0090, a median RMSE of 0.0063, and colorimetric errors with a median of 0.03 and a maximum of 0.35. Among the methods trained on the MUNS dataset, WINE demonstrates the best performance with an RAMSE of 0.0206, a median RMSE of 0.0175, and colorimetric metrics showing a median of approximately 0.32 and a maximum of 0.35.

When comparing model-based methods across training sets, WINE and LMMSE consistently perform better when trained on STUD, PCAE tend to perform more favorably when trained on MUNS, and WPCA obtains comparable results for both datasets. Nevertheless, WINE and LMMSE generally yield superior results to PCAE and WPCA in both training scenarios. The table clearly illustrates that training-based methods significantly outperform model-based methods.

These findings support the conclusion that when designing a new system with a configuration similar to those investigated in this study, acquiring a dedicated training dataset comprising paired camera responses and corresponding ground truth reflectances can be highly beneficial, compared to relying solely on existing reflectance datasets for model-based methods. This conclusion is drawn from the explicit comparison performed in this work between reconstruction methods trained on an acquired, system-specific dataset and those relying on generic reflectance datasets. To the best of our knowledge, this study represents the first experimental attempt to evaluate these approaches under such differing training data assumptions for multispectral cameras of this type. A broader investigation of this topic across systems with different configurations and acquisition conditions constitutes an important topic for future comparative and review studies.

According to Table III, similar to the SpectroCam, the best overall performance for the Silios camera is achieved by the POLY method trained on the STUD dataset, with nearly identical results across different adaptive strategies. This configuration achieves an RAMSE of 0.0090, a median RMSE of 0.0064, and colorimetric errors with a median of 0.03 and a maximum of 0.13. Among the methods trained on the MUNS dataset, PCAE provides the best performance, yielding an RAMSE of 0.0443, a median RMSE of 0.0356, and a ΔE00 under CIE D65 with a median of approximately 0.79 and a maximum of 3.85. However, the best ΔE00 under CIE LED-B5 is obtained by the MINN method.

In contrast to the SpectroCam case, the model-based methods behave differently for the Silios camera. Specifically, WINE and LMMSE perform better when trained on the STUD dataset in terms of colorimetric accuracy and on the MUNS dataset from a spectral metrics perspective. The PCAE method also tends to perform more favorably when trained on STUD. Notably, although PCAE and WPCA generally outperform WINE and LMMSE in both training conditions, WPCA produces the poorest colorimetric results overall.

As with the SpectroCam, the results for the Silios camera clearly demonstrate that training-based methods significantly outperform model-based approaches. This reinforces the conclusion that for the design of a new spectral imaging system, it is highly advantageous to collect a dedicated training dataset containing paired camera responses and ground truth spectral reflectances rather than relying solely on model-based methods trained on existing datasets.

The findings also suggest that a well-distributed and compact set of reflectance samples, such as those in the dedicated STUD dataset, can be sufficient to achieve high reconstruction accuracy despite being smaller than the standard MUNS dataset. Similar observations have been reported in previous studies, where carefully selected or application-specific reflectance datasets were shown to provide effective training for spectral reconstruction methods [50–52]. The problem of optimally selecting such a dataset, however, lies beyond the scope of this study.

One notable observation from the tables is the behavior of the MINN method. Overall, MINN performs better with the SpectroCam than with the Silios camera. A possible explanation for this lies in the interaction between the camera spectral sensitivities and the LED light source used in this study. Although the Silios camera has more channels, its spectral responsivities, shown in Fig. 2(a), are significantly affected by the system’s LED illumination as illustrated in Fig. 2(b). The high intensity near 440 nm shifts the response curves of certain channels, and in others, the secondary peak in the blue region becomes comparable to the main peak. This crosstalk effect is much less pronounced in the SpectroCam, whose filters exhibit nearly rectangular spectral transmittance without significant secondary peaks.

In the camera forward model, the SPD of the illuminant is multiplied by the camera sensitivities. If some regions of the spectrum become poorly observable or if channel responses become more redundant due to illumination-induced crosstalk, the resulting system matrix M can become ill-conditioned or rank-deficient. Since MINN relies on the Moore–Penrose pseudoinverse of M, such degeneracy can lead to instability or reduced accuracy, which the method does not compensate for, as it lacks regularization or priors.

Another important characteristic of MINN is its illumination dependence. Although it shows the poorest performance in terms of ΔE00 under CIE D65, it achieves comparable, and in the case of the Silios camera, even the best ΔE00 under CIE LED-B5 among all model-based methods. This can be attributed to the strong influence of the illumination on the MINN method. Because the camera responses and the system matrix are both derived under LED illumination, the reconstructed reflectances are implicitly optimized for that specific light source. When evaluated under D65, a spectral mismatch occurs, leading to larger colorimetric errors. In contrast, evaluation under LED, the system’s native light, preserves the consistency between reconstruction and assessment conditions, resulting in improved accuracy.

Although the detailed numerical results are reported in tabular form to ensure completeness and reproducibility, the large number of evaluated methods and performance metrics can make direct comparison across camera–dataset configurations challenging. To address this, a compact visual summary is provided in Figure 8, which presents bar plots of the key evaluation metrics for the best-performing spectral reconstruction method associated with each camera–dataset combination. Specifically, the figure summarizes the colorimetric performance under CIE D65 and CIE LED-B5 illuminants, as well as the spectral reconstruction accuracy measured by RMSE, for SpectroCam–STUD, SpectroCam–MUNS, Silios–STUD, and Silios–MUNS. This visualization highlights the relative performance trends across systems and datasets, complements the detailed tables, and facilitates a more intuitive comparison of the main findings.

Figure 8.

Summary of the best-performing spectral reconstruction methods for each camera–dataset combination. Bar plots show (a) ΔE00 under CIE D65, (b) ΔE00 under CIE LED-B5, and (c) spectral RMSE. The reported results correspond to the best-performing methods for each configuration: SpectroCam–STUD (POLY-specLO), SpectroCam–MUNS (WINE-camLO), Silios–STUD (POLY-camLO), and Silios–MUNS (PCAE-camLO). Error bars over the median and mean bars indicate the standard deviation.

Among all evaluated configurations, the POLY method, a training-based approach trained and tested on the STUD dataset, achieved the highest overall performance. The results are nearly identical across both cameras, with SpectroCam showing slightly better spectral and colorimetric accuracy. However, Silios performs better in terms of the standard deviation and maximum values of the colorimetric errors.

Although the choice of adaptive dataset selection has minimal impact on the performance of POLY for the Silios camera, it has a more noticeable effect on the SpectroCam, particularly when considering the percentage of reconstructed spectra falling within the Excellent category of GFC. The highest GFC performance is observed with the specLO strategy, where 15.91% of the results fall into this category.

It is evident from the results that in all cases, training-based methods consistently achieve higher accuracy than model-based approaches. A potential explanation for this discrepancy lies in the inherent reliance of model-based methods on an accurate characterization of the imaging system. These methods require detailed knowledge of multiple system parameters, including sensor spectral responsivities, filter transmittances, lens transmittance, cut-off filters, and the SPD of the illumination. Accurate measurement of these characteristics is essential for reliable performance. However, in this study, the spectral characteristics of the cameras were obtained from manufacturer-provided data, which may be subject to inaccuracies. Although these uncertainties may partially account for the increased reconstruction errors observed in model-based methods, this interpretation is based on the present experimental results and does not establish a causal relationship. A systematic investigation of the impact of uncertainties in spectral sensitivity data on reconstruction performance is therefore identified as an important direction for future work. Within the scope of the present experiments, these observations also suggest a practical advantage of data-driven, training-based approaches for real-world applications.

4.2.1

Example Reconstructed Reflectances

Figures 9 and 10 present the reconstructed spectral reflectances for nine selected samples from the STUD dataset. These samples are chosen to represent distinct hues, collectively covering nearly the entire hue range of the dataset, and are marked with red diamonds in Fig. 5(b).

Figure 9.

Reconstructed spectral reflectances for nine selected samples from the STUD dataset. The results are obtained using the SpectroCam camera and the baseline spectral reconstruction methods trained on the full set of training samples. (a) Both training and testing are conducted on the STUD dataset. The evaluated methods include both model-based and training-based approaches. (b) Training is performed on the MUNS dataset and testing on the STUD dataset, where only model-based methods are evaluated.

Figure 10.

Reconstructed spectral reflectances for nine selected samples from the STUD dataset. The results are obtained using the Silios camera and the baseline spectral reconstruction methods trained on the full set of training samples. (a) Both training and testing are conducted on the STUD dataset. The evaluated methods include both model-based and training-based approaches. (b) Training is performed on the MUNS dataset and testing on the STUD dataset, where only model-based methods are evaluated.

Figure 9 shows the results obtained using the SpectroCam camera and baseline spectral reconstruction methods trained on the full set of training samples. The results from methods where both training and testing are performed on the STUD dataset are displayed in Fig. 9(a), which includes both model-based and training-based approaches. In contrast, Fig. 9(b) presents results where training is performed on the MUNS dataset and testing on the STUD dataset, for which only model-based methods are applicable. Figure 10 displays the corresponding results obtained using the Silios camera.

A visual inspection of these figures reveals that for these samples, POLY, PCAE, and especially WPCA perform better with the Silios camera than with SpectroCam in both the STUD- and MUNS-trained scenarios. Conversely, the remaining methods exhibit superior performance with the SpectroCam. Additionally, for SpectroCam, training on the STUD dataset yields slightly better results compared to training on MUNS. Notably, PCAE and WPCA consistently show the poorest performance across both training setups. These observations are consistent with the quantitative results presented in Tables II and III.

To assess the computational cost of the adaptive methods, the runtime of camLO and specLO was measured using MATLAB R2024b on a MacBook Pro 16-inch (2023) with an Apple M2 Pro and 16 GB of memory. For comparison, the best-performing POLY method was executed for all test samples in the STUD dataset, using a local dataset size K approximately equal to the size of STUD. On the SpectroCam, the runtime was 6.102 s for specLO and 1.700 s for camLO. On the Silios camera, the runtime was 8.150 s for specLO and 2.815 s for camLO. These results show that while specLO achieved slightly higher reconstruction accuracy, camLO provided a substantial reduction in computational cost, which may be preferable for time-sensitive applications.

Conclusions

This study presents a thorough comparative analysis of spectral reconstruction methods for multispectral imaging under controlled LED illumination, with a strong emphasis on practical performance evaluation. Using two distinct multispectral cameras and standardized spectral reflectance datasets, the strengths and limitations of both model-based and training-based approaches augmented with adaptive local dataset selection are comprehensively demonstrated.

Experimental results show that training-based methods, particularly POLY and PINV, consistently outperform model-based techniques across various spectral and colorimetric accuracy metrics. Among all configurations, POLY trained and tested on the STUD dataset delivers the best overall performance for both cameras. Moreover, adaptive strategies that dynamically select locally optimal training samples further improve reconstruction quality, notably reducing RAMSE errors and increasing robustness.

Although model-based methods offer advantages in terms of interpretability and speed, their reliance on accurate system modeling limits their practical effectiveness. In this study, the use of manufacturer-provided spectral sensitivity data likely introduced inaccuracies that affected their performance. Furthermore, the illumination dependency of the MINN method was evident, with significantly better results under the LED light source compared to standard daylight (D65), emphasizing its sensitivity to SPD.

The findings support the conclusion that in designing a new multispectral imaging system, acquiring a dedicated training dataset, including both camera responses and corresponding spectral reflectances, combined with training-based reconstruction methods provides significant advantages over relying solely on model-based approaches and existing standard datasets. This study, therefore, provides practical guidance for selecting suitable spectral reconstruction strategies based on system-specific data availability and operational constraints.

Future work could focus on further characterizing multispectral imaging systems by investigating channel-wise noise and gain parameters as well as their variation with respect to input intensity levels. Such analysis would improve the accuracy of camera models and support better calibration practices. Additionally, refining adaptive dataset selection methods, such as exploring the repetition of nearest neighbors in the camLO method, analogous to the strategy used in specLO, could enhance reconstruction accuracy. Another promising direction involves evaluating the impact of different light sources on reconstruction performance, analyzing both spectral and colorimetric outcomes under varied illumination conditions. These investigations would deepen the understanding of factors influencing spectral recovery and further optimize multispectral system design.

In this study, the Munsell dataset is employed as a controlled benchmark for spectral estimation. Although validation on more diverse real-world spectral datasets is important for assessing generalization, such an evaluation lies beyond the scope of the present work and will be addressed in future studies.

Disclosures

The authors declare that there are no financial interests, commercial affiliations, or other potential conflicts of interest that could have influenced the objectivity of this research or the writing of this paper.

Code, Data, and Material Availability

Data are available on request from the authors.

Acknowledgment

This project has received funding from ApPEARS (Appearance Printing European Advanced Research School) by the European Union’s Horizon 2020 research and innovation programme under Marie Skłodowska-Curie grant agreement no. 814158.

References

1PrieurC.RabatelA.ThomasJ.-B.FarupI.ChanussotJ.2022Machine learning approaches to automatically detect glacier snow lines on multi-spectral satellite imagesRemote Sensing14386810.3390/rs14163868

2EwerlöfM.LarssonM.SalerudE. G.2017Spatial and temporal skin blood volume and saturation estimation using a multispectral snapshot imaging cameraImaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues XV10068105116105–16SPIEBellingham, WA, USA

3ArteagaY.BoustC.HardebergJ.2022HDR multispectral imaging-based BRDF measurement using a flexible robotic arm systemColor and Imaging Conf.30758075–80Society for Imaging Science and TechnologySpringfield, VA, USA10.2352/CIC.2022.30.1.15

4YuanY.SunR.XuC.MaS.LiuY.WangY.SongW.2023Design of an image-based BRDF measurement method using a catadioptric multispectral capture and a real-time Lambert calibration,Opt. Express32425443425–4310.1364/OE.510627

5JohnsonT.1996Methods for characterizing colour scanners and digital cameras,Displays16183191183–9110.1016/0141-9382(96)01012-8To Achieve WYSIWYG Colour

6HameyL.Radiometric Camera Calibration20094John Wiley & SonsUnited States232923382329–38

7QuanS.“Evaluation and Optimal Design of Spectral Sensitivities for Digital 1178 Color Imaging,” Ph.D. dissertation (Rochester Institute of Technology, 1179 Rochester, NY, USA, 2002).

8BurnsP. D.BernsR. S.1996Analysis multispectral image captureColor and Imaging Conf.4192219–22Society of Imaging Science and TechnologySpringfield, VA, USA

9KhanH. A.GreenP.2018Color characterization methods for a multispectral cameraInt’l. Symp. on Electronic Imaging 2018: Color Imaging XXIII: Displaying, Processing, Hardcopy, and ApplicationsSociety for Imaging Science and TechnologySpringfield, VA, USA

10HerzogP. G.KnippD.StiebigH.KönigF.1999Colorimetric characterization of novel multiple-channel sensors for imaging and metrologyJ. Electron. Imaging8342353342–5310.1117/1.482704

11GaoH.MantiukR. K.FinlaysonG. D.2024Color-accurate camera capture with multispectral illumination and multiple exposures,Comput. Graph. Forum43Wiley Online LibraryHoboken, NJ, USA

12KleinJ.BrauersJ.AachT.2011Methods for spectral characterization of multispectral camerasDigital Photography VII7876102112102–12SPIEBellingham, WA, USA

13BauerJ. R.ThomasJ.-B.HardebergJ. Y.VerdaasdonkR. M.2019An evaluation framework for spectral filter array cameras to optimize skin diagnosisSensors19480510.3390/s19214805

14KleinJ.Multispectral imaging: aberrations and acquisitions from different viewing positions. Dissertation, Aachen, Techn. Hochsch., Aachen (2016). Veröffentlicht auf dem Publikationsserver der RWTH Aachen University 2016; Aachen, Techn. Hochsch., Diss., 2015

15PrattW. K.MancillC. E.1976Spectral estimation techniques for the spectral calibration of a color image scannerAppl. Opt.15737573–510.1364/AO.15.000073

16ShimanoN.2006Recovery of spectral reflectances of objects being imaged without prior knowledgeIEEE Trans. Image Process.15184818561848–5610.1109/TIP.2006.877069

17MaloneyL. T.1986Evaluation of linear models of surface spectral reflectance with small numbers of parametersJ. Opt. Soc. Am. A3167316831673–8310.1364/JOSAA.3.001673

18EckhardT. M.“Design considerations for line-scan multi-spectral imaging systems,” PhD thesis (Universidad de Granada, 2016)

19SafdarM.EmmelP.2022Toward non-metameric reflectance recovery by emulating the spectral neighborhood using corresponding color informationJ. Opt. Soc. Am. A39106610751066–7510.1364/JOSAA.451931

20YooJ.-H.KimD.-C.HaH.-G.HaY.-H.2016Adaptive spectral reflectance reconstruction method based on wiener estimation using a similar training setJ. Imaging Sci. Technol.6010.2352/J.ImagingSci.Technol.2016.60.2.020503

21LiuZ.LiuQ.GaoG.-a.LiC.2017Optimized spectral reconstruction based on adaptive training set selectionOpt. Express25124351244512435–4510.1364/OE.25.012435

22ShenH.-L.XinJ. H.2004Spectral characterization of a color scanner by adaptive estimation,J. Opt. Soc. Am. A21112511301125–3010.1364/JOSAA.21.001125

23ShenH.-L.XinJ. H.2006Spectral characterization of a color scanner based on optimized adaptive estimationJ. Opt. Soc. Am. A23156615691566–910.1364/JOSAA.23.001566

24TominagaS.SakaiH.2023Spectral reflectance estimation from camera responses using local optimal datasetJ. Imaging94710.3390/jimaging9020047

25ShenH.-L.CaiP.-Q.ShaoS.-J.XinJ. H.2007Reflectance reconstruction for multispectral imaging by adaptive wiener estimationOpt. Express15155451555415545–5410.1364/OE.15.015545

26TominagaS.SakaiH.2024Spectral reflectance estimation from camera response using local optimal dataset and neural networks,J. Imaging1022210.3390/jimaging10090222

27TominagaS.NishiS.OhteraR.SakaiH.2022Improved method for spectral reflectance estimation and application to mobile phone camerasJ. Opt. Soc. Am. A39494508494–50810.1364/JOSAA.449347

28LiT.LiuT.WangY.LiX.GuY.2022Spectral reconstruction network from multispectral images to hyperspectral images: a multitemporal caseIEEE Trans. Geosci. Remote Sens.601161–16

29FairmanH. S.BrillM. H.2004The principal components of reflectancesColor Res. Appl.: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur29104110104–10

30AgahianF.AmirshahiS. A.AmirshahiS. H.2008Reconstruction of reflectance spectra using weighted principal component analysisColor Res. Appl.: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur33 360371 360–71

31ZhangJ.SuR.FuQ.RenW.HeideF.NieY.2022A survey on computational spectral reconstruction methods from RGB to hyperspectral imagingSci. Rep.1211905

32ValeroE. M.HuY.Hernández-AndrésJ.EckhardT.NievesJ. L.RomeroJ.SchnitzleinM.NowackD.2014Comparative performance analysis of spectral estimation algorithms and computational optimization of a multispectral imaging system for print inspectionColor Res. Appl.39162716–2710.1002/col.21763

33ShiM.HealeyG.2002Using reflectance models for color scanner calibrationJ. Opt. Soc. Am. A19645656645–5610.1364/JOSAA.19.000645

34JaaskelainenT.ParkkinenJ.ToyookaS.1990Vector-subspace model for color representationJ. Opt. Soc. Am. A7725730725–3010.1364/JOSAA.7.000725

35ImaiF. H.BernsR. S.1999Spectral estimation using trichromatic digital camerasPro. Int’l. Symp. on Multispectral Imaging and Color Reproduction for Digital Archives42181–8Chiba University Chiba, Japan

36ConnahD.WestlandS.ThomsonM.2001Recovering spectral information using digital camera systemsColoration Technol.117309312309–1210.1111/j.1478-4408.2001.tb00080.x

37HabibT.GreenP.NussbaumP.2023Spectral estimation: its behaviour as a sat and implementation in colour managementJ. Imaging Sci. Technol.67

38GreenP. J.A Colour Engineering Toolbox (International Color Consortium, Reston, VA, USA, 2003). https://www.color.org/resources/ColourEngineeringToolbox.xalter. Accessed: 2025-07-20

39RomeroJ.Garcıa-BeltránA.Hernández-AndrésJ.1997Linear bases for representation of natural and artificial illuminantsJ. Opt. Soc. Am. A14100710141007–1410.1364/JOSAA.14.001007

40Ansari-AslM.BarbieriM.ObeinG.HardebergJ. Y.2023Advancing material appearance measurement: a cost-effective multispectral imaging system for capturing SVBRDF and BTFLondon Imaging MeetingSociety for Imaging Science and TechnologySpringfield, VA, USA

41Ansari-AslM.BarbieriM.ObeinG.HardebergJ. Y.2024Optical and electromechanical design and implementation of an advanced multispectral device to capture material appearanceJ. Imaging105510.3390/jimaging10030055

42Silios, Silios Multispectral Cameras (Silios Technologies, Peynier, France, 2024). https://www.silios.com/cms-series. Accessed on 24 December 2024

43SpectroCam, Spectrocam Multispectral Wheel Cameras (Torrent Photonics, Largo, FL, USA, 2024). https://salvocoatings.com/products/spectrocam/. Accessed on 24 December 2024

44Martí-nez DomingoM.Multispectral High Dynamic Range Polarimetric Imaging Applied to Scene Segmentation and Object Classification. PhD thesis, Universidad de Granada, Granada, Spain (2017). Available online: http://hdl.handle.net/10481/47628 (Accessed on 24 December 2024)

45BengioY.GrandvaletY.2004No unbiased estimator of the variance of k-fold cross-validationJ. Mach. learn. Res.5108911051089–105

46CIE – International Commission on Illumination, CIE Standard Illuminant D65 (International Commission on Illumination, Vienna, Austria, 2019). https://doi.org/10.25039/CIE.DS.hjfjmt59. CIE, Vienna, Austria

47International Commission on Illumination (CIE), Relative Spectral Power Distributions of Illuminants Representing Typical LED Lamps, 1nm Spacing (International Commission on Illumination, Vienna, Austria, 2018). https://doi.org/10.25039/CIE.DS.dhcw57sd. CIE, Vienna, Austria

48BiancoS.2010Reflectance spectra recovery from tristimulus values by adaptive estimation with metameric shape correctionJ. Opt. Soc. Am. A27186818771868–7710.1364/JOSAA.27.001868

49Hernández-AndrésJ.RomeroJ.Lee JrR. L.2001Colorimetric and spectroradiometric characteristics of narrow-field-of-view clear skylight in Granada, SpainJ. Opt. Soc. Am. A18412420412–2010.1364/JOSAA.18.000412

50LiangJ.ZhuQ.LiuQ.XiaoK.2022Optimal selection of representative samples for efficient digital camera-based spectra recoveryColor Res. Appl.47107120107–2010.1002/col.22718

51EckhardT.ValeroE. M.Hernández-AndrésJ.SchnitzleinM.2014Adaptive global training set selection for spectral estimation of printed inks using reflectance modeling,Appl. Opt.53709719709–1910.1364/AO.53.000709

52HardebergJ. Y.Acquisition and Reproduction of Color Images: Colorimetric and Multispectral Approaches2001Universal-PublishersBoca Raton, FL, USA

articleview.keywords