Diffusion models can see an illusion. Then they forget it before the picture comes out.

How denoising models quietly build human-like illusion representations deep inside the network, and throw the signal away before it ever reaches a pixel.

Denoising diffusion models are trained to remove noise to reconstruct images, without any priors to develop illusion sensitivity or show human-like psychophysics traits.

And yet, if you look inside their intermediate activations while they process a classic color illusion, something interesting shows up. 2 regions that are physically identical but look different to a human because of their surrounding context also look different to the model, at specific layers inside its own representations.

This was not specified to them. It just emerged as a process of training. This is the core finding of the paper we worked on: Denoising models develop human like perceptual illusion representations across architectures.

What we actually did

We took 9 models across 3 architecture families: pixel-space DDPMs, latent diffusion U-Nets, and a diffusion transformer (DiT-XL/2), plus 3 discriminative baselines (ResNet-50, VGG-19, ViT-B/16) trained on classification instead of denoising. We fed them classic color-contrast illusions, images where two regions are pixel identical but look different to a human because of context, and ran a single forward pass, capturing activations at every layer.

To measure whether a model “notices” the illusion, we compared its internal activation on the two regions that look different to a human, and did the same on a matched control image with the illusory context removed. We used standard effect-size statistics (Cohen’s d, Hedge’s g, Glass’s delta) to see where in the network, and how strongly, this differential shows up.

Crucially, no specific probing classifiers or fine-tuning was done. We just looked at what was already there.

What we found

Every denoising model, regardless of architecture, develops a measurable illusion-sensitive signal. It is not spread evenly across the network. It concentrates sharply at specific layers, and is 2 to 4.5 times stronger for the actual illusion than for the matched control image.

The discriminative models trained on classification, with largely similar architectures, in the case of ViT-B/16 versus DiT-XL/2, show a much weaker effect, well below the threshold to be called meaningful. Hence, we come to the conclusion that “Architecture may provide a variable baseline strength, but the denoising objective makes the effect stable and directionally concrete”.

On grayscale brightness illusions, where a validated psychophysical model of human perception (FLODOG) already exists, the model’s activations track it closely, with spearman correlation upto 0.78. Essentially meaning that the model activations closely resemble the outputs of human psychophysics models.

We follow by asking the question, are there representations causal or in more simpler terms, does any of this reach the output? We zeroed out the specific channels that carry the illusion signal and watched it propagate, dropping by 44% just 1 layer downstream, confirming the effect is real and causal inside the network. But when measured on the output, the produced shift was close to zero.

We call this a perceptual phantom: a representation that is fully active and causally involved in how the model processes an image internally, and is then almost completely dissolved before it ever shows up in what the model actually generates.

Why this matters

Most evaluation of what a generative model “knows” happens behaviourally at the output i.e. the predicted pixels or the generated image. This work shows that a model can build a genuinely human like internal representation of something and have it be completely invisible if you only ever look at what it produces. The model computed illusion, used it internally, and then its own later layers overwrote it on the way out.

That has an uncomfortable implication for how we evaluate these systems. If output-level behavior can systematically hide what a model has actually represented internally, then judging a model purely by what it generates may be judging the wrong thing entirely.

What we are not claiming

This is not evidence that denoising models perceive illusions or develop illusion sensitive representations exactly the way humans do, and it is not a claim that all internal representations in these models are phantoms. We show this for one well-defined class of illusion-sensitive channels, established through channel ablation, cross-architecture replication, and injection tests that rule out the more boring explanations (mask geometry, incomplete ablation, off-manifold artifacts). Whether this generalizes to other kinds of internal knowledge is an open question.

The full paper is on arXiv (link below).

ArXiv link: https://arxiv.org/abs/2607.17138

Website link: https://lossfunk.github.io/Denoising-models-illusion-representations/

Code: https://github.com/Lossfunk/Denoising-models-illusion-representations