Visual Cortex based Material Appearance Transfer Model

Hiroaki Kotera; Norimichi Tsumura

doi:10.2352/J.ImagingSci.Technol.2022.66.5.050406

Abstract

MA (Material Appearance) is a perceptual phenomenon that our brain deciphers from the retinal image. What features of retinal image are most closely related to the stimulus inside the visual cortex of V1 ∼ V5? The function of V1 is the most well-studied. V1 has the function of seeing fine in the fovea and rough in the periphery, and is mathematically described by LPT (Log-Polar Transform).

Since LPT samples the retinal image at a higher rate in the fovea but at lower rate peripherally, the color information tends to gather in center of V1. Paying attention to this LPT features in V1, we reported a novel method to transfer MA from one to another scenes. After LPT, PCM (Principal Component Matching) is applied to match the color distribution between source and target scenes. By just showing the target scene as an example, our previously reported LPT-PCM model can transfer the MA of target to that of source without any a priori information. However, this model had drawbacks such as changes in appearance depending on the background margins and unpredictable results for the scenes consisting of multiple color clusters.

This article explores measures to overcome such drawbacks and discusses the applicability of proposed LPT-PCM. Finally we propose a new numerical index to evaluate the similarity between the target and the transferred with the examined samples.

jist

JIMTE6

Journal of Imaging Science and Technology

J. Imaging Sci. Technol.

1062-3701

1943-3522

Society for Imaging Science and Technology

050406

10.2352/J.ImagingSci.Technol.2022.66.5.050406

1318

Papers based on Work Presented at CIC30: Color and Imaging 2022

Visual Cortex based Material Appearance Transfer Model

Visual cortex based material appearance transfer model

KoteraHiroaki

▴

Kotera Imaging Laboratory, 3-20-20, Oyumino-Cyuo, Midori-ku, Chiba, 266-0032, Japan

hiroimage@gmail.com

TsumuraNorimichi

▴

Chiba University, 1-33 Yayoi-chi, Inage-ku, Chiba, 263-8522, Japan

Kotera and Tsumura

▴

IS&T Members.

092022

050406-1

050406-10

852022

782022

2022

Abstract

ccc

1062-3701/2022/66(5)/050406/10/$25.00

printed

Printed in the USA

Background

Human observers can recognize material property at a glance through sensory organs. Without touching materials, we can tell whether they would feel hard or soft, cool or warm, rough or smooth, wet or dry. Furthermore, it is possible to define the material such as metal or wood, leather or cloth.

The material appearance is a perceptual phenomenon of feeling or sensation that our brain perceives from optical image projected onto retina. Though, it is hard to untangle what information of the retinal image stimulates the visual cortex of V1 ∼ V5 and how it induces the material feeling in our brain. The mechanism of INNER VISION in brain is still a black box at present [1].

As a framework for material perception, N. Tsumura initiated the skin color appearance and proposed the concept of appearance delivering system [2].

In Brain Information Science research on SHITSUKAN (material perception) [3] by MEXT (Ministry of Education, Culture, Sports, Science and Technology) in Japan, the first stage (2010–2014 led by Dr. H. Komatsu) has concluded and the second stage (2015–2019 led by Dr. S. Nishida) stepped forward into “multi-dimensional” material perception and currently, further research on “deep texture” is progressing. The results so far are being put onto practical use.

Despite the complexity of MA mechanism, human sensations such as “gloss/matte”, “transparent/translucent”, “metal/cloth” are controllable by an intuitive, but a smart technique.

For instance, Motoyoshi, Nishida, et al. [4] noticed the “gloss” perception appears when the luminance histogram is skewed. If it’s stretched smoothly to the higher luminance, the object looks “glossy” but looks “matte” if compressed to the lower. Sawayama and Nishida [5] developed “wet” filter by a combination of exponent-shaped TRC (Tone Reproduction Curve) and boosted color saturation. It is very interesting that any “skew” in the image features induces a sensational material perception. However, the mechanism why and how such sensations as “gloss” or “wet” are activated by the “skew” effect in our cerebral visual cortex is not untangled yet.

Meanwhile, R & Ds for practical applications are making steady progress in private enterprises. BRDF (Bidirectional Reflectance Distribution Function) describes the specular and diffusion components of optical surface reflection, which carry “gloss” or “texture” appearance and used to adjust MA. As a successful example, a specular reflection control algorithm based on BRDF is implemented in LSI chip used for a commercial 4K HDTV set [6].

Motivated by that LPT mimics the structure of visual cortex V1, we intended to apply it to MR transfer from a different perspective.

Scene Color Transfer Models

Since the material perceptions such as gloss or clarity are related to a variety of factors [7], it is hard to specify the cause of perceptual feeling to a single factor. Nevertheless, trials on material or textual appearances transfer between CG images [8] or 3D objects [9] are reported. Especially, color appearance plays an important role in the MA. Historically, Reinhard’s color transfer model [10] was epoch-making, where the color atmosphere of source scene A was transferred into that of target scene B. The clustered color distribution of A is roughly matched with that of B. There, the use of vision-based lαβ color space by Ruderman [11] attracted interest.

So far, various example-based models have been reported, but they needed troublesome segmentation processes prior to define the target color area.

Most recently, Chunzhi Gu et al. [12] improved this drawback by an extended Gaussian Mixed Model (GMM) instead of segmentation. But, it incurs high computation costs to optimize GMM by iterative EM (Expectation Maxmization) algorithm.

The author developed a novel joint LPT-PCM model by combining LPT and PCM to transfer scene colors between different materials [13]. The LPT-PCM is also an example-based model, but the principle is fundamentally different from the conventional methods. This is because we do not see the object with our eyes but with the visual cortex, that is, LPT-PCM is a cortex-based model, where LPT reflects the variable resolution sampling characteristics of visual cortex V1.

The LPT-PCM model worked well between the objects that form relatively simple color clusters, but had the problems such as the changes in appearance depending on the background margin size or unpredictable/unintended results for the targets with complex mixed color clusters.

At the last CIC29, we announced how to emphasize MA. This time, we discuss how to transfer MA between scenes, focusing on the visual cortex V1, which is deeply related to MA.

2.1

lαβ Color Transfer Model

The lαβ is known as an orthogonal luminance-chrominance color space simply transformed from RGB by the following Step 1 and Step 2 and the color distribution of source image is changed to match the target (reference) image by the scaling process in Step 3 and the color atmosphere of target is transferred to the source via the inverse transform in Step 4 as follows

Step 1: RGB to LMS cone response transform

(1)

[\begin{matrix} L \\ M \\ S \end{matrix}] = [\begin{matrix} 0.381 & 0.578 & 0.040 \\ 0.197 & 0.724 & 0.078 \\ 0.024 & 0.129 & 0.844 \end{matrix}] [\begin{matrix} R \\ G \\ B \end{matrix}] .

Step 2: LMS to lαβ transform with orthogonal luminance l and chrominance αβ

(2)

\begin{matrix} [\begin{matrix} l \\ α \\ β \end{matrix}] & = & [\begin{matrix} 1 ∕ \sqrt{3} & 0 & 0 \\ 0 & 1 ∕ \sqrt{6} & 0 \\ 0 & 0 & 1 ∕ \sqrt{2} \end{matrix}] [\begin{matrix} 1 & 1 & 1 \\ 1 & 1 & - 2 \\ 1 & - 1 & 0 \end{matrix}] \\ \times [\begin{matrix} log L \\ log M \\ log S \end{matrix}] . \end{matrix}

Step 3: Scaling of lαβ around the mean values

{\bar{l}, \bar{α}, \bar{β}}

by the ratio of standard deviation to make match the color distributions between source and target images.

(3)

\begin{matrix} l^{'} = (σ_{DST}^{l} ∕ σ_{ORG}^{l}) (l - \bar{l}) \\ α^{'} = (σ_{DST}^{α} ∕ σ_{ORG}^{α}) (α - \bar{α}) \\ β^{'} = (σ_{DST}^{β} ∕ σ_{ORG}^{β}) (β - \bar{β}), \end{matrix}

where, σORGl and σDSTα denote the standard deviation of luminance l for the source image and that of chrominance α for the target image, and so on.

Step 4: Inverse transform [l′α′β′] ⇒ [L′M′S′] ⇒ [R′G′B′].

Finally, the scaled l′α′β′ source image with color distribution matched to the target image is displayed on sRGB monitor.

2.2

PCM Color Transfer Model

Prior to lαβ model, the author et al. developed PCM (Principal Component Matching) model [14, 15] for transferring the color atmosphere from one scene to another as illustrated in Figure 1. The lαβ model works well between the scenes with color similarity but not for the scenes with color dissimilarity and often fails. While, PCM model works almost stable between the scenes with color dissimilarities and advanced toward automatic scene color interchange [16–18].

In our basic object-to-object PCM model a vector X in a color cluster is projected onto a vector Y in PC space by Hotelling Transform as

(4)

Y = A (X - μ) .

Where,

μ

denotes the mean vector and the matrix A is formed by the set of eigenvectors {e1 e2 e3} of covariance matrix

Σ_{X}

(5)

A = [e_{1} e_{2} e_{3}] .

The covariance matrix

Σ_{Y}

of {Y} is diagonalized in terms of A and

Σ_{X}

with the elements composed of the eigenvalues {λ1 λ2 λ3} of

Σ_{X}

(6)

Σ_{Y} = A (Σ_{X}) A^{t} = [\begin{matrix} λ_{1} & 0 & 0 \\ 0 & λ_{2} & 0 \\ 0 & 0 & λ_{3} \end{matrix}] .

Figure 1.

Scene Color Transfer Model based on PCM.

Thus the color vectors in source and target images are mapped to the same PC space and the following equations are formed to make match a source vector YORG to a target vector YDST through the scaling matrix S as follows.

(7)

\begin{matrix} Y_{DST} = A_{DST} (X_{DST} - μ_{DST}) and \\ Y_{ORG} = A_{ORG} (X_{ORG} - μ_{ORG}), \end{matrix}

where,

(8)

\begin{matrix} Y_{DST} = S \cdot Y_{ORG} \end{matrix}

(9)

\begin{matrix} S = [\begin{matrix} \sqrt{λ_{1 DST} ∕ λ_{1 ORG}} & 0 & 0 \\ 0 & \sqrt{λ_{2 DST} ∕ λ_{2 ORG}} & 0 \\ 0 & 0 & \sqrt{λ_{3 DST} ∕ λ_{3 ORG}} \end{matrix}] \end{matrix}

Solving Eqs. (7) and (8), we get the following relation between a source color XORG and a target color XDST to be transferred and matched.

(10)

X_{DST} - μ_{DST} = M_{PCM} (X_{ORG} - μ_{ORG}) .

The matching matrix MPCM is given by

(11)

M_{PCM} = (A_{DST}^{- 1}) (S) (A_{ORG}),

where, AORG and ADST denote the eigen matrices for the source color cluster and the target color cluster. In the scaling matrix S, λ1ORG means the 1st eigenvalue of the source and λ2DST the 2nd eigenvalue of the target, etc. These are obtained from each covariance matrix.

In general, the PCM model works better than lαβ even for the scenes with color dissimilarities, because of using the statistical characteristics of covariance matrix.

Figure 2 shows a successful example in both lαβ and PCM models for the images with color similarity. While, in case of Figure 3, lαβ fails to change the color atmosphere of A into that of B due to their color dissimilarities, but works well in PCM.

2.3

Spectral Decomposition Color Transfer Models

Following the lαβ model, a variety of improved or alternative color transfer models have been reported. As a basic drawback in lαβ model, Pitie et al. [18] pointed out that it’s not based on the statistical covariance but only on the mean values and variances in the major lαβ axes. Hence PCM model is better than lαβ because of using the statistical covariance matrix

Σ_{X}

with the Hotelling transform onto the PC space. At the same time, Pitie [19] suggested to make use of orthogonal spectral decomposition paying the attention to the Hermitian (Self adjoint) property of symmetric matrix

Σ_{X}

with real eigenvalues.

Figure 2.

Successful example between images with color similarity.

Figure 3.

Comparison in lαβ versus PCM for images with color dissimilarity.

In general, the covariance matrix

Σ

in a clustered color distribution of image is a real symmetric matrix. The square root of

Σ

for source and target images is decomposed by eigenvalues as

(12)

Σ_{ORG}^{1 ∕ 2} = A_{ORG}^{- 1} D_{ORG}^{1 ∕ 2} A_{ORG} and Σ_{DST}^{1 ∕ 2} = A_{DST}^{- 1} D_{DST}^{1 ∕ 2} A_{DST} .

AORG and ADST denote the eigen matrices for source and target images. DORG and DDST are given by the diagonal matrices with the entries of their eigenvalues respectively.

(13)

\begin{matrix} D_{ORG} = [\begin{matrix} λ_{1 ORG} & 0 & 0 \\ 0 & λ_{2 ORG} & 0 \\ 0 & 0 & λ_{3 ORG} \end{matrix}], \\ D_{DST} = [\begin{matrix} λ_{1 DST} & 0 & 0 \\ 0 & λ_{2 DST} & 0 \\ 0 & 0 & λ_{3 DST} \end{matrix}] . \end{matrix}

Now, the color matching matrix MEigen corresponding to Eq. (11) is given by

(14)

\begin{matrix} M_{Eigen} & = & Σ_{DST}^{1 ∕ 2} Σ_{ORG}^{- 1 ∕ 2} \\ = & (A_{DST}^{- 1} D_{DST}^{1 ∕ 2} A_{DST}) {(A_{ORG}^{- 1} D_{ORG}^{1 ∕ 2} A_{ORG})}^{- 1} \\ = & (A_{DST}^{- 1} D_{DST}^{1 ∕ 2} A_{DST}) (A_{ORG}^{- 1} D_{ORG}^{- 1 ∕ 2} A_{ORG}) . \end{matrix}

2.4

Singular Value Decomposition (SVD)

A m × n matrix

Σ

is decomposed by SVD as the product of matrices U, V, and W

(15)

Σ = U W V .

Where, U and V are m × m and n × n orthogonal matrices. If

Σ

is a m × n rectangular matrix of rank-r, matrix W is composed of r × r diagonal matrix with the singular values as its entries and the remaining small null matrices.

Since the covariance

Σ

is a 3 × 3 real symmetric matrix, the singular values equal to the eigenvalues and SV D equals EV D.

2.5

Cholesky Decomposition

Cholesky, a compact spectral decomposition method, decomposes the covariance

Σ

as a simple product of lower triangular matrix and its transpose as follows.

(16)

\begin{matrix} Σ_{ORG} = L_{ORG} L_{ORG}^{T} for L_{ORG} = Chol [Σ_{ORG}] Chol {[Σ_{ORG}]}^{T} \\ Σ_{DST} = L_{DST} L_{DST}^{T} for L_{DST} = Chol [Σ_{DST}] Chol {[Σ_{DST}]}^{T} \end{matrix}

2.6

Eigenvalue Decomposition (EVD)

Where, Chol[∗] denotes the Cholesky decomposition. The lower triangular matrix L is obtained by the iteration just like as Gaussian elimination method (details omitted).

The color matching matrix MChol to transfer the color atmosphere of target image into the source is given by

(17)

M_{Chol} = L_{DST} {(L_{ORG})}^{- 1} .

Visual Cortex Based LPT-PCM Color Transfer

3.1

Retina to Visual Cortex Log Polar Transform

The PCM model works well to transfer the color atmosphere between the images even with color dissimilarities. However, any human visual characteristic has not been taken into account. In this paper, a striking feature in the spatial color distributions in our visual cortex image is introduced to improve the performance in PCM.

The mapping to visual cortex from retina is mathematically described by Schwartz’s complex Logarithmic Polar Transform [20].

The complex vector z pointing a pixel located at (x, y) in the retina is transformed to a new vector log(z) by LPT as follows.

(18)

\begin{matrix} z = x + j y = ρ e^{j θ}; ρ = | z | and θ = {tan}^{- 1} (y ∕ x) \\ log (z) = u + j v = log (ρ) + j θ; j = \sqrt{- 1} . \end{matrix}

The retinal image is sampled at spatially-variant resolution on the polar coordinate (ρ, θ), that is, in the radial direction, fine in the fovea but coarser towards peripheral according to the logarithm of ρ, while in the angle direction, at a constant pitch Δθ and stored to the coordinate (u, v) in the striate cortex of V1.

Figure 4 illustrates a sketch how the retinal image is sampled, stored in the striate cortex, and played back to retina.

Figure 4.

Outline of spatially-variant mapping to visual cortex from retina.

In the discrete LPT system, (ρ, θ) is digitized to R number of rings and S number of sectors. The striate cortex image is stored in the new Cartesian coordinates (u, v) as

(19)

\begin{matrix} \begin{matrix} (u, v) \underset{̲}{\underset{̲}{Δ}} {ρ (u), θ (v)} \\ ρ (u) = ρ_{0} a^{u} for ρ \geq ρ_{0}, u = 1, 2, \dots, R \\ a = exp [log (ρ_{max} ∕ ρ_{0}) ∕ R] \\ θ (v) = v Δ θ = (2 π ∕ S) v for v = 1, 2, \dots, S . \end{matrix} \end{matrix}

ρ0 denotes the radius of blind spot and ρ ≥ ρ0 prevents the points near origin not to be mapped to the negative infinite-point. This regulation is called CBS (Central Blind Spot) model. Figure 5 illustrates how the image “sunflower” is sampled in LPT lattice and transformed to striate cortex image, and then stored in the coordinates (u, v).

The height h(u) and width w(u) of an unit cell between u + 1 and u are given by the following equations. Hence the area α(u) of unit cell increases exponentially with u.

(20)

\begin{matrix} h (u) = ρ (u + 1) - ρ (u) = ρ_{0} (a - 1) a^{u} \\ w (u) = \frac{1}{2} (2 π ∕ S) \{ρ (u + 1) + ρ (u)\} = (π ∕ S) (1 + a) a^{u} ρ_{0} \\ α (u) = h (u) w (u) = π {ρ_{0}}^{2} (a^{2} - 1) a^{2 u} S^{- 1} . \end{matrix}

As sensed in Fig. 5, the color is sampled finer in the center but coarser towards peripheral. The pixels in the yellow petals occupy a larger area than peripheral. This color distribution in striate cortex reflects that V1 has the spatial processing function that collects color information in the center of the viewpoint.

Figure 5.

“sunflower” sampled in LPT lattice, and stored in Striate Cortex.

Figure 6 is another example for a pink rose named “cherry shell”. It shows how the color distribution is concentrated on the pinkish petal area around at the central viewpoint in the striate cortex image. Hence it’ll be better for applying PCM not on the original, but on the striate cortex image after LPT to perform the color matching more effective for the object of attention.

Figure 6.

Spatially-variant color concentration effect in Striate Cortex by LPT.

Now the basic PCM matrix MPCM in Eq. (11) is applied to the covariance after LPT and we get the novel color transfer matrix as

(21)

\begin{matrix} \begin{matrix} M_{LPTPCM} = (_{LPT} A_{DST}^{- 1}) (_{LPT} S) (_{LPT} A_{ORG}) \\ _{LPT} S = [\begin{matrix} \sqrt{{L P T}^{'} 1 D S T ∕ L P T^{'} 1 O R G} 0 0 \\ 0 \sqrt{{L P T}^{'} 2 D S T ∕ L P T^{'} 2 O R G} 0 \\ 0 0 \sqrt{{L P T}^{'} 3 D S T ∕ L P T^{'} 3 O R G} \end{matrix}] . \end{matrix} \end{matrix}

Figure 7 illustrates the color transfer process in LPT-PCM model. In this sample, both the source image A and target image B are first transformed to the visual cortex images by LPT, then the clustered color distribution in cortex image A is transformed to match the cortex image by PCM. As a result, the MA of greenish transparent wine glass B appears to be transferred to that of gold mask image A.

Figure 7.

Improved LPT-PCM color transfer model.

Since the original images A and B have color dissimilarity, it’s a hard to make the color matching only by the single use of basic PCM. While, by just placing LPT before PCM, the feeling of greenish wine glass B is well conveyed to that of gold mask A.

Results in Comparison with Other Methods

The performance of proposed LPT-PCM model is compared with the other methods mentioned in Section 3. Figure 8 shows the results for the same images used in Fig. 7. The lαβ model fails for such images with color dissimilarity. The source colors remain almost unchanged. SVD and Cholesky decomposition reflect the greenish target colors a little bit, but look unnatural.

In the basic PCM model, the black in eyes and the green in mask face seem to have replaced the unnatural look. Mismatches in the directions of PC axes may occur. Meanwhile, LPT-PCM model worked successfully for transferring the color atmosphere of greenish wine glass to that of gold mask.

Figure 8.

Performance of LPT/PCM in comparison with other methods.

Figure 9 shows another example for color transfer between three glass vases with different textures. lαβ had a minimal effect leaving the source colors almost unchanged. Though SVD and Cholesky decomposition showed certain effects, a partial color mixing happened between the source B and target A as shown in B to A color matching. PCM and LPT-PCM looks like a neck and neck. But looking carefully, LPT-PCM gives a little bit better impression than PCM due to conveying the clean textures in the target.

Figure 9.

Performance of LPT/PCM for images with different textures.

Figure 10 is another result for the images with heterogeneous color & texture. The color atmosphere of “greenish wine glass” is attempted to transfer to that of “reddish Porsche”, where only LPT-PCM seems to be successful.

Figure 10.

A result for the images with heterogeneous color & textures.

Problems and Discussion on Countermeasure

Since our LPT-PCM model utilizes the characteristics of visual cortex V1 that concentrates color information in the center of viewpoint, it has higher MA transfer performance than others.

However, it is not all-round, as occasionally we often encounter unexpected or unnatural results depending on the target. Below are some typical problems and examples of countermeasures.

5.1

Dependency on Background Margins and Cropping

Since the color transfer model is based on the color distribution of the source and the target, the background margin also affects the color distribution and has a simple problem of changing results.

Figure 11 shows an example of the resulting color change due to the background margins of the source image. One of the measures to prevent unintended MA caused by a large background margin is to crop in advance. The figure is such an example. LPT-PCM looks insensitive to the margins and robust than PCM.

Figure 11.

Problem of color change due to background margin.

5.2

How LPT-PCM Works for Multi-Cluster Target?

For the sake of simplicity, the basic PCM is designed assuming a single-cluster image. In the case of multi-cluster images, if segmentation is needed for separating the colored objects to each cluster then the object-to-object PCM is performed. however, it is hard to find the corresponding pair of objects particularly in the case of dissimilar color images [14–16]. Hence, the proposed model may be limited to a single cluster image rather than universal.

Figure 12 shows a difficult example of multi-cluster image. However, LPT-PCM resulted in a color transfer function that matches the light green glass placed in the center of the three colored wine glasses. This is because light green occupies the largest area in the striate cortex image mapped to V1 as shown in the dotted circle.

Figure 12.

Performance of LPT/PCM for Multi-Cluster Target.

If the target B is rearranged so that the pink glass is in the center as target C, a pinkish Porsche is obtained as shown in the 3rd. line.

Why LPT? Because LPT mimics the spatial transform function of retina to/from visual cortex called foveation.

Similarity Index

Since MA involves many factors as a perceptual phenomenon, it is difficult to quantitatively evaluate the similarity between the transferred result and the target.

As a simple way, the following similarity index ρ is tested.

(22)

ρ = Cor [F l a t t e n {Cov [t r n L a b]}, F l a t t e n {Cov [d s t L a b]}] .

Where, trnLab and dstLab denote CIELAB color distributions of Transferred result and that of Target to be matched.

Cor [X, Y ] means Correlation Operator between the vectors X and Y and Cov [Z] denotes Covariance of Z.

Here,

F l a t t e n

works to convert 2-D matrix to 1-D array vector.

Figure 13 shows the examples of estimated similarity index ρ.

Figure 13.

Evaluated Similarity Index and Comparison with Other Methods.

Index values generally seem to reflect similarities, but do not always correspond to human visual senses. Verification by more cases and the search for more reliable indicators are future tasks.

Conclusions

This paper discussed the applicability of LPT/PCM MA transfer model. It’s a model that combines LPT and PCM algorithms. Prior to PCM, the retinal images are transformed to striate cortex images by LPT. The key is to make use of color concentration features on the central viewpoint areas by LPT. The performance of basic PCM is significantly enhanced when combined with LPT. This innovative model worked better than others without any a priori information or optical measurement for the material properties. The question is how to evaluate the transformed image to be perceptually acceptable or not. A new similarity index ρ is proposed and showed some reasonable results. Towards future MA research, it’ll be an eternal goal to answer the question “how the brain feels beauty?” asked repeatedly by Semir Zeki in his book, INNER VISION [1].

References

1Semir Zeki, INNER VISION, “An Exploration of Art and the Brain” (Oxford University Press, Oxford, UK, 1999)

2TsumuraN.BabaK.YamamotoS.SambongiM.2015Estimating reflectance property from refocused images and its application to auto material appearance balancingJ. Imaging Sci. Technol.5930501-130501-630501-1–610.2352/J.ImagingSci.Technol.2015.59.3.030501

3http://www.shitsukan.jp/ISST/

4MotoyoshiI.NishidaS.SharanL.AdelsonE. H.2007Image statistics and the perception of surface qualitiesNature447206209206–910.1038/nature05724

5SawayamaM.NishidaS.2015Visual perception of surface wetnessJ. Vis.1593710.1167/15.12.937

6KobikiH.NonakaR.BabaM.2013Specular reflection control technology to increase glossiness of imagesToshiba Rev.68384138–41

7FlemingR. W.2014Visual perception of materials and their propertiesVis. Res.94627562–7510.1016/j.visres.2013.11.004

8MihálikA.Durikovi£R.2009Material appearance transfer between imagesSCCG’09, Proc. Spring Conf., CG.555855–8ACMNew York, NY

9NguyenC. H.RitschelT.MyszkowskiK.EisemannE.SeidelH. P.20123D material style transferComputer Graphics Forum31431438431–810.1111/j.1467-8659.2012.03022.x

10ReinhardE.AdhikhminM.GoochB.ShirleyP.2001Color transfer between imagesProc. IEEE CG Appl.21344034–40IEEEPiscataway, NJ10.1109/38.946629

11RudermanD. L.CroninT. W.ChiaoC. C.1998Statistics of cone responses to natural images: implications for visual codingJ. Opt. Soc. Am. A15203620452036–4510.1364/JOSAA.15.002036

12GuC.LuX.ZhangC.2022Example-based color transfer with Gaussian mixture modelingJ. Pattern Recognit.12910871610.1016/j.patcog.2022.108716

13KoteraH.TominagaS.SchettiniR.TrémeauA.HoriuchiT.2019Material appearance transfer with visual cortex imageComputational Color Imaging. CCIW 2019Lecture Notes in Computer ScienceVol. 11418334348334–48SpringerCham10.1007/978-3-030-13940-7_25

14KoteraH.MorimotoT.SaitoR.1998Object-oriented color matching by image clusteringProc. IS&T/SID CIC6: Sixth Color Imaging Conf.154158154–8IS&TSpringfield, VA

15KoteraH.SuzukiM.ChenH. S.2001Object-to-object color mapping by image segmentationJ. Electron. Imaging10977987977–8710.1117/1.1407263

16KoteraH.HoriuchiT.2004Automatic interchange in scene colors by image segmentationProc. IS&T/SID CIC12: Twelfth Color Imaging Conf.939993–9IS&TSpringfield, VA

17KoteraH.MatsusakiY.HoriuchiT.SaitoR.Automatic color interchange between imagesProc. Congress of the Int’l. Color Association2005Vol. AIC 05101910221019–22

18KoteraH.2006Intelligent Image ProcessingJ. SID.14745754745–54

19PitiéF.KokaramA.2007The linear monge-kantorovitch colour mapping for example-based colour transferProc. IET CVMP.233123–31IETLondon

20SchwartzE. L.1977Spatial mapping in the primate sensory projection: analytic structure and relevance to perceptionBiol. Cybern.25181194181–9410.1007/BF01885636