Two thought experiments probe the same nerve: whether a complete physical or functional account of a mind leaves anything out.
Mary's Room (Frank Jackson, 1982)
Mary is a brilliant scientist who knows every physical fact about color vision — wavelengths, neurons, all of it — but has lived her whole life in a black-and-white room. The day she steps out and sees red for the first time, does she learn something new? If she does, then there are facts about experience that no amount of physical knowledge contains.
Philosophical zombies (David Chalmers)
Imagine a being physically identical to you, behaving identically — but with no inner experience whatsoever. "All dark inside." If such a zombie is even conceivable, experience may not be reducible to physical function.
Applied to AI: a system could know — and do — everything about understanding, and still never have a single moment of it.
Related: The Hard Problem of Consciousness · The Chinese Room · Moral Patienthood