The mathematical formulation of multispectral imaging parallels that of conventional tristimulus imaging, with the key distinction being its higher number of spectral channels. The camera response for a given stimulus can be expressed as(1)ρi=gi ∫λsn(λ)i(λ)r(λ)dλ+ni.

Here, ρ_n represents the response of the camera’s nth channel while r(λ) and i(λ) denote the spectral reflectance of the object and the spectral power distribution (SPD) of the light source, respectively. The variable λ corresponds to the wavelength, and s_n(λ) is the spectral responsivity of the nth channel. For a multispectral camera with N channels, there exist N spectral responsivities, collectively represented as s_n(λ) ∈{s₁(λ), s₂(λ), …, s_N(λ)}.

In Eq. (1), the gain parameter g_i is a coefficient that converts the result of the integral into digital counts, representing the camera output. This parameter depends on the imaging system configuration, including factors such as imaging geometry and the SPD of the light source. Furthermore, n_i denotes the imaging system noise, which is modeled as white noise with zero mean and variance a_i, and is assumed to be uncorrelated with the spectral reflectance [24, 26, 27].

This relationship can be reformulated in matrix notation as (2)ρ=gSIR+n.

In this expression, the variables are defined as follows:

The problem of reflectance reconstruction involves estimating the spectral reflectances R from the measured multichannel responses ρ. By introducing a matrix M, defined as the product of the gain parameter, the camera responsivities, and the illumination SPD, M = gSI, Eq. (2) can be simplified as (3)ρ=MR+n.

Assuming a noiseless system, if M were known and invertible, the reconstruction of spectral reflectance would reduce to solving a straightforward inverse problem: (4)R̂=M−1ρ, where R̂ represents the estimated spectral reflectance. However, in practice, M is often not invertible, and this direct solution is not robust especially in the presence of noise. Consequently, alternative approaches are required to achieve accurate and stable spectral reflectance reconstruction.

The estimation of spectral reflectance from multispectral data has been actively investigated in the literature for more than five decades [15, 28]. The central goal is to find a transformation matrix W that maps the measured multispectral data to the spectral reflectance space as expressed in(5)R̂=Wρ.

Unlike global methods, which apply a fixed transformation or model across all inputs, adaptive spectral reconstruction methods dynamically tailor the estimation process to each individual sample [24, 25]. This adaptability often enhances both accuracy and robustness. These methods typically involve identifying a subset of training samples that are most similar to the target input and performing reconstruction using only this localized data. In this study, we employ two adaptive strategies: one that selects samples based on spectral similarity and another based on similarity in camera response space. Both approaches are detailed in Sections 2.3.1 and 2.3.2.

To evaluate the performance of spectral reconstruction methods, several widely used metrics are employed. Colorimetric accuracy is assessed using CIEDE2000, which provides improved perceptual uniformity compared to earlier color difference formulas. Spectral accuracy is evaluated using the root mean squared error (RMSE) and the goodness-of-fit coefficient (GFC), capturing both magnitude and shape differences between spectra. Although other metrics such as PSNR, mean absolute error, ΔE₇₆, and ΔE₉₄ are also reported in the literature and were computed during the analysis, they are omitted here to limit redundancy and maintain conciseness as they exhibited trends consistent with the selected measures. The following provides a brief description and mathematical formulation of each metric.

The Silios CMS-C [42] is a multispectral imaging system that utilizes an SFA integrated with a standard CMOS sensor. This camera employs a Bayer-like mosaic filter positioned over a sensor with a native resolution of 1280 × 1024 pixels. However, the mosaic’s 3 × 3 configuration reduces the effective spectral image resolution to 426 × 339 macropixels. The device captures data across eight discrete narrowband channels and one broad panchromatic band. The narrowband filters cover central wavelengths from 430 nm to 700 nm, with Gaussian-like transmission profiles, each having an average full width at half maximum of 40 nm. These filters are centered at specific wavelengths: 440 nm, 473 nm, 511 nm, 549 nm, 585 nm, 623 nm, 665 nm, and 703 nm. Meanwhile, the panchromatic band provides consistent sensitivity across the visible spectrum. The sensor features a 5.3 μm pixel pitch and delivers 10-bit digital output.

The quantum efficiency of the Silios CMS-C camera’s spectral channels is shown in Figure 2(a) [13]. In this camera, each pixel is designed to capture light from a specific spectral band, as the filters are integrated directly on the sensor. However, imperfections in the filters and optical system can cause light from one spectral band to overlap with adjacent channels, leading to inaccuracies in the recorded spectral and spatial data. This issue, known as crosstalk, is typically corrected numerically during post-processing. However, this correction is not addressed in this paper. Instead, Fig. 2(b) presents the quantum efficiency of the spectral channels multiplied by the SPD of the employed LED source. This figure illustrates how crosstalk affects the spectral responsivity of the channels, with secondary peaks becoming comparable in magnitude to the main spectral peak under LED illumination. The impact of this issue is further discussed in Section 4.

The SpectroCam (VIS + NIR) multispectral camera [43] utilizes a silicon-based CCD panchromatic sensor capable of detecting wavelengths ranging from 350 nm to 1050 nm. A motorized filter wheel, positioned in front of the sensor, accommodates up to eight interchangeable filters. Users have access to over 145 commercially available interference filters with diverse spectral properties [44]. The spectral transmittance characteristics of the filters pre-installed in the camera are illustrated in Figure 3. These filters are centered at wavelengths of 375 nm, 425 nm, 475 nm, 525 nm, 570 nm, 625 nm, 680 nm, and 930 nm. The first seven filters have a bandwidth of approximately 50 nm while the last filter features a broader bandwidth of around 100 nm. The camera outputs images with a resolution of 2456 × 2058 pixels and a pixel pitch of 3.45 μm. Its digital output supports up to 12-bit precision.

The spectral sensor responsivity and filter transmittances initially installed in the camera are depicted in Fig. 3. All the eight filters are exhibited; however, Filter UV and Filter IR, shown in gray and black in Fig. 3, were omitted from the experiment due to inadequate light intensity.

The light source in our setup integrates multiple components to ensure efficient and controlled illumination. As detailed in our previous work [41], at the core of our light source is the Thorlabs MCWHL6 LED, emitting cold white light at 6500 K with an output power of 1430 mW, mounted on a PCB for stability and thermal management. Heat dissipation is achieved using a heat sink while a Thorlabs SM2F32-A adjustable collimation adapter is employed to produce collimated light, essential for accurate spectral imaging and surface data capture. Light intensity is regulated via the upLED^TM LED Driver, offering fine control with a USB interface and C++ SDK support. This configuration provides uniform, collimated lighting tailored to our application requirements.

Figure 4 presents the results of radiometric analysis of the LED. The SPD measured at the precise imaging geometry and location is shown in Fig. 4(a). The Thorlabs MCWHL6 LED was selected for its exceptional reconstruction capabilities and advantageous technical attributes, including higher CCT and output power, as detailed in [41]. Figure 4(b) reveals slight inhomogeneity in the light source across the surface of the sample holder table in the measurement setup shown in Fig. 1, with illumination intensity generally decreasing as the distance from the table center (marked 0 on the X-coordinate) increases. To mitigate this variability, all measurements were conducted at the center of the sample holder table.

To assess the LED’s temporal stability, its SPD was recorded every minute over six hours. As shown in Fig. 4(c), the radiance decreases by about 1% within the first 75 minutes after activation before stabilizing. Consequently, a warm-up period was included before measurements. Additionally, spectral analysis indicates minimal change, with a shift of approximately 1 nm over the six-hour measurement period.

Dynamic range represents the ratio between the maximum and minimum light intensities that a camera sensor can accurately capture. Essentially, it reflects the sensor’s ability to preserve details in both high- and low-light regions of a scene. In multispectral imaging, dynamic range is crucial because spectral bands may have vastly different camera responsivities due to differences in sensor quantum efficiency, filter transmission, and lens imperfections. Furthermore, nonuniform illumination across the spectral range from the light source creates significant intensity variations that the camera must accommodate in different channels.

Each spectral band has a unique dynamic range that determines its usable range for a given exposure time. The overall dynamic range required in an experiment is influenced by the light source’s SPD, the reflectance properties of the samples, and the imaging setup. If the required dynamic range exceeds the camera’s capability, high dynamic range (HDR) imaging is necessary. Otherwise, exposure time adjustments can ensure sufficient dynamic range overlap.

Managing exposure time is essential for effective multispectral imaging. Some spectral bands may require shorter exposure times to avoid saturation while others may need longer exposure times to avoid noise. The Silios and SpectroCam cameras handle exposure time differently. While the Silios camera applies a single exposure time across all channels, the SpectroCam allows for channel-specific adjustments.

For the Silios camera, the dynamic range was evaluated through a stepwise process that follows the usual practice in the field. First, dark noise was measured by capturing ten images in a dark room with no light source or ambient illumination, and the average image histogram revealed an average dark noise of about ten digital counts in the 10-bit output (0–1023 range). Next, a Spectralon reflectance standard, representing the maximum diffuse reflectance, was imaged. The exposure time was empirically adjusted so that all spectral bands remained below saturation levels. The reflectance of the least reflective sample was then measured to confirm that its values exceeded the dark noise across all bands. This confirmed that the required dynamic range could be covered with a single exposure time, eliminating the need for HDR imaging.

Two datasets were used to evaluate the performance of the spectral reconstruction methods. The first contains 264 samples of the Munsell Student Color Set (STUD), measured using the Konica Minolta CS2000 reference spectroradiometer, along with the corresponding camera responses captured by both the SpectroCam and Silios multispectral cameras. This dataset was employed in the evaluation of both model-based and training-based reconstruction methods. To obtain the camera responses, each sample was imaged ten times, and the resulting images were averaged to reduce random noise. Following this, dark noise was subtracted from the averaged image. The pixel values corresponding to the same spatial area measured by the Konica Minolta CS2000 were then spatially averaged to produce the final camera response for each sample, a commonly adopted practice intended to reduce sensor noise and mitigate local spatial nonuniformities in radiometric measurements.

The second dataset is the full standard Munsell (MUNS) dataset, comprising 1269 spectral reflectances, which was also used in the experiments. However, it could only be utilized for evaluating the model-based methods as it does not include the corresponding camera responses.

All spectral reflectance data were truncated to the 400–700 nm range and sampled at 5 nm intervals, resulting in spectral vectors of 61 dimensions per reflectance.

The spectral reflectance datasets used for method evaluation are illustrated in Figure 5: the STUD dataset, which was measured in the frame of the present study, along with its gamut in the CIE a^∗–b^∗ space (a and b); the MUNS dataset and its associated gamut (c and d), which were obtained from previously published data [37].

The camera gain and noise parameters were measured following the method proposed in [27] with minor modifications. In this approach, a Spectralon diffuse white reference sample is imaged by the cameras under the system’s illumination. The gain parameter is then estimated by minimizing the L1 and L2 norm errors between the theoretical and observed mean camera responses. Similarly, the noise variance is estimated by minimizing the L1 and L2 norm errors between the theoretical and empirical covariance matrices of the pixel values. In this work, the L1 norm was employed, and both the gain and noise variance were calculated separately for each spectral channel of both cameras. This is necessary because in multispectral cameras, the channel-specific settings can vary significantly.

The number of PCs used in the PCAE and WPCA methods has a significant impact on spectral reconstruction accuracy and is dependent on the imaging system. In this study, the optimal number of PCs was determined experimentally for each camera instead of adopting values from prior work. Specifically, the baseline PCA estimation method (PCAE without local dataset selection) was evaluated by varying the number of PCs over all feasible values using the STUD and MUNS datasets independently. The reconstruction performance was assessed using the median RMSE, maximum RMSE, and RAMSE metrics, and the optimal number of PCs was selected as the value minimizing these error measures. Consistent results were obtained across both datasets, yielding an optimal choice of five PCs for the SpectroCam and four PCs for the Silios camera. These values were subsequently used throughout all analyses in this work.

Figures 6 and 7 present the performance trends of various methods, which are categorized into four groups. In these figures, the vertical axis denotes the RAMSE while the horizontal axis indicates the number of closest neighbors K. The minimum of each curve identifies the best performance point for each method. Figure 6(a) illustrates the performance of both model-based and training-based methods using the SpectroCam camera, where both training and testing are performed on the STUD dataset. Each method is evaluated with different local dataset selection strategies: “None” (original method using the entire dataset), “camLO” (local optimal selection in camera space), and “specLO” (local optimal selection in spectral space). For consistency, each method is plotted with a distinct line color and style in figures. The ”None” methods appear as horizontal lines since they are evaluated with training on the full dataset.

Figure 7(a) displays results similar to those in Fig. 6(a), using the Silios camera. Here too, training and testing are performed on the STUD dataset. Figures 6(b) and 7(b) show the performance of the SpectroCam and Silios cameras, respectively, when trained on the MUNS dataset and tested on the STUD dataset. As discussed in Section 3.5, training-based methods cannot be used in this scenario due to the lack of corresponding camera responses in the MUNS dataset. Therefore, only model-based methods are included in these figures. For better clarity, a zoomed-in view highlighting the region near the minimum RAMSE is also provided in the figures where necessary to enhance readability.

As shown in Figs. 6(a) and 7(a), the POLY method consistently outperforms the other approaches. The WINE and LMMSE models exhibit similar performance trends and produce comparable results. Although LMMSE is theoretically expected to yield lower estimation errors than WINE [27], this advantage primarily holds when the model is trained on a full dataset with a sufficiently large number of samples [24]. However, the plots indicate that the superiority of LMMSE over WINE does not generalize across different cameras: it consistently holds for the Silios camera, but the opposite trend is observed for the SpectroCam. Investigating the underlying cause of this discrepancy is beyond the scope of this paper.

Another important observation is that methods using camLO selection converge to the performance of the original method when K equals the full dataset size. This occurs because in camLO, the selected local dataset becomes identical to the full dataset at that point. In contrast, specLO does not replicate the original dataset exactly as it involves repeated use of the closest samples, which leads to a dataset with a higher density of similar samples. Consequently, specLO can potentially yield lower errors than the original method, even when K is maximal.

After determining the optimal number of closest training samples K, each method was executed using its corresponding best K, that is, each method was evaluated in its optimal configuration. The experimental results are presented in Tables II and III. As outlined in Section 4.1, the methods are categorized into four groups based on the camera and training dataset. Accordingly, Tables II and III report the performance of both model-based and training-based methods using SpectroCam and Silios cameras, respectively. In these tables, methods with both training and testing conducted on the STUD dataset are shown above a double-line separator while those with training on the MUNS dataset and testing on the STUD dataset appear below it. The best performance (viz., the minimum error) within each category is highlighted in bold for each metric and statistical measure, and model-based and training-based approaches are distinguished by a single solid line separator.

The tables present the performance of the spectral reconstruction methods under different local dataset selection strategies, with the corresponding optimal training set size shown in the K column. Each table includes ΔE₀₀ under CIE D65 [46] and CIE LED-B5 [47] illuminants as the colorimetric errors and two spectral metrics: RMSE and GFC. These illuminants were selected to provide standardized and widely accepted reference conditions for color difference assessment, facilitating reproducibility and comparison with existing studies. Although the experimental LED source used for image acquisition does not exactly match the spectral characteristics of these standard illuminants, the reconstructed outputs represent device-independent spectral reflectance estimates, allowing colorimetric evaluation under arbitrary illuminants. Furthermore, within the CIE LED illuminant series, CIE LED-B5 exhibits the most similar behavior to the employed LED source in terms of peak wavelength positions and relative amplitude in the blue spectral region.

The first four metrics are reported using four statistics—median, standard deviation (std), 95th percentile, and maximum values—while the RAMSE is included under the RMSE statistics instead of maximum. Following [48, 49], GFC values are categorized into four intervals and reported as the percentage of estimation errors falling within each interval. The intervals are defined as follows: GFC < 0.995 (poor), 0.995 ≤ GFC < 0.999 (accurate), 0.999 ≤ GFC < 0.9999 (good), and GFC ≥ 0.9999 (excellent).

The MINN method is presented separately in the tables and serves as a baseline for comparison. As a simple, interpretable, and closed-form solution—requiring no iterative optimization, training data, or prior assumptions—it provides a strong benchmark for validating the effectiveness of more advanced models.

According to Table II, the best overall performance for the SpectroCam is achieved by the POLY method combined with specLO and trained on the STUD dataset. This configuration yields an RAMSE of 0.0090, a median RMSE of 0.0063, and colorimetric errors with a median of 0.03 and a maximum of 0.35. Among the methods trained on the MUNS dataset, WINE demonstrates the best performance with an RAMSE of 0.0206, a median RMSE of 0.0175, and colorimetric metrics showing a median of approximately 0.32 and a maximum of 0.35.

When comparing model-based methods across training sets, WINE and LMMSE consistently perform better when trained on STUD, PCAE tend to perform more favorably when trained on MUNS, and WPCA obtains comparable results for both datasets. Nevertheless, WINE and LMMSE generally yield superior results to PCAE and WPCA in both training scenarios. The table clearly illustrates that training-based methods significantly outperform model-based methods.

These findings support the conclusion that when designing a new system with a configuration similar to those investigated in this study, acquiring a dedicated training dataset comprising paired camera responses and corresponding ground truth reflectances can be highly beneficial, compared to relying solely on existing reflectance datasets for model-based methods. This conclusion is drawn from the explicit comparison performed in this work between reconstruction methods trained on an acquired, system-specific dataset and those relying on generic reflectance datasets. To the best of our knowledge, this study represents the first experimental attempt to evaluate these approaches under such differing training data assumptions for multispectral cameras of this type. A broader investigation of this topic across systems with different configurations and acquisition conditions constitutes an important topic for future comparative and review studies.

According to Table III, similar to the SpectroCam, the best overall performance for the Silios camera is achieved by the POLY method trained on the STUD dataset, with nearly identical results across different adaptive strategies. This configuration achieves an RAMSE of 0.0090, a median RMSE of 0.0064, and colorimetric errors with a median of 0.03 and a maximum of 0.13. Among the methods trained on the MUNS dataset, PCAE provides the best performance, yielding an RAMSE of 0.0443, a median RMSE of 0.0356, and a ΔE₀₀ under CIE D65 with a median of approximately 0.79 and a maximum of 3.85. However, the best ΔE₀₀ under CIE LED-B5 is obtained by the MINN method.

In contrast to the SpectroCam case, the model-based methods behave differently for the Silios camera. Specifically, WINE and LMMSE perform better when trained on the STUD dataset in terms of colorimetric accuracy and on the MUNS dataset from a spectral metrics perspective. The PCAE method also tends to perform more favorably when trained on STUD. Notably, although PCAE and WPCA generally outperform WINE and LMMSE in both training conditions, WPCA produces the poorest colorimetric results overall.

As with the SpectroCam, the results for the Silios camera clearly demonstrate that training-based methods significantly outperform model-based approaches. This reinforces the conclusion that for the design of a new spectral imaging system, it is highly advantageous to collect a dedicated training dataset containing paired camera responses and ground truth spectral reflectances rather than relying solely on model-based methods trained on existing datasets.

The findings also suggest that a well-distributed and compact set of reflectance samples, such as those in the dedicated STUD dataset, can be sufficient to achieve high reconstruction accuracy despite being smaller than the standard MUNS dataset. Similar observations have been reported in previous studies, where carefully selected or application-specific reflectance datasets were shown to provide effective training for spectral reconstruction methods [50–52]. The problem of optimally selecting such a dataset, however, lies beyond the scope of this study.

One notable observation from the tables is the behavior of the MINN method. Overall, MINN performs better with the SpectroCam than with the Silios camera. A possible explanation for this lies in the interaction between the camera spectral sensitivities and the LED light source used in this study. Although the Silios camera has more channels, its spectral responsivities, shown in Fig. 2(a), are significantly affected by the system’s LED illumination as illustrated in Fig. 2(b). The high intensity near 440 nm shifts the response curves of certain channels, and in others, the secondary peak in the blue region becomes comparable to the main peak. This crosstalk effect is much less pronounced in the SpectroCam, whose filters exhibit nearly rectangular spectral transmittance without significant secondary peaks.

In the camera forward model, the SPD of the illuminant is multiplied by the camera sensitivities. If some regions of the spectrum become poorly observable or if channel responses become more redundant due to illumination-induced crosstalk, the resulting system matrix M can become ill-conditioned or rank-deficient. Since MINN relies on the Moore–Penrose pseudoinverse of M, such degeneracy can lead to instability or reduced accuracy, which the method does not compensate for, as it lacks regularization or priors.

Another important characteristic of MINN is its illumination dependence. Although it shows the poorest performance in terms of ΔE₀₀ under CIE D65, it achieves comparable, and in the case of the Silios camera, even the best ΔE₀₀ under CIE LED-B5 among all model-based methods. This can be attributed to the strong influence of the illumination on the MINN method. Because the camera responses and the system matrix are both derived under LED illumination, the reconstructed reflectances are implicitly optimized for that specific light source. When evaluated under D65, a spectral mismatch occurs, leading to larger colorimetric errors. In contrast, evaluation under LED, the system’s native light, preserves the consistency between reconstruction and assessment conditions, resulting in improved accuracy.

Although the detailed numerical results are reported in tabular form to ensure completeness and reproducibility, the large number of evaluated methods and performance metrics can make direct comparison across camera–dataset configurations challenging. To address this, a compact visual summary is provided in Figure 8, which presents bar plots of the key evaluation metrics for the best-performing spectral reconstruction method associated with each camera–dataset combination. Specifically, the figure summarizes the colorimetric performance under CIE D65 and CIE LED-B5 illuminants, as well as the spectral reconstruction accuracy measured by RMSE, for SpectroCam–STUD, SpectroCam–MUNS, Silios–STUD, and Silios–MUNS. This visualization highlights the relative performance trends across systems and datasets, complements the detailed tables, and facilitates a more intuitive comparison of the main findings.

Among all evaluated configurations, the POLY method, a training-based approach trained and tested on the STUD dataset, achieved the highest overall performance. The results are nearly identical across both cameras, with SpectroCam showing slightly better spectral and colorimetric accuracy. However, Silios performs better in terms of the standard deviation and maximum values of the colorimetric errors.

Although the choice of adaptive dataset selection has minimal impact on the performance of POLY for the Silios camera, it has a more noticeable effect on the SpectroCam, particularly when considering the percentage of reconstructed spectra falling within the Excellent category of GFC. The highest GFC performance is observed with the specLO strategy, where 15.91% of the results fall into this category.

It is evident from the results that in all cases, training-based methods consistently achieve higher accuracy than model-based approaches. A potential explanation for this discrepancy lies in the inherent reliance of model-based methods on an accurate characterization of the imaging system. These methods require detailed knowledge of multiple system parameters, including sensor spectral responsivities, filter transmittances, lens transmittance, cut-off filters, and the SPD of the illumination. Accurate measurement of these characteristics is essential for reliable performance. However, in this study, the spectral characteristics of the cameras were obtained from manufacturer-provided data, which may be subject to inaccuracies. Although these uncertainties may partially account for the increased reconstruction errors observed in model-based methods, this interpretation is based on the present experimental results and does not establish a causal relationship. A systematic investigation of the impact of uncertainties in spectral sensitivity data on reconstruction performance is therefore identified as an important direction for future work. Within the scope of the present experiments, these observations also suggest a practical advantage of data-driven, training-based approaches for real-world applications.