Electronic computers don't internally manipulate zeros, ones, numbers, numerals, characters, symbols or text. They react to, manipulate, store, and emit electronic things.
The concept of information is an albatross around AI's neck. AI can't explain it. The closest is Claude Shannon's quantitative information-theoretic conception, which is quantitative, not semantic. If it's not semantic then it's not information in a sense relevant to human-like knowledge. It would be better to stop using the concept of information.
Knowledge is togetherness in time. The substrate is irrelevant. At least one type of difference must exist. Having only two makes a machine cheap and reliable. To say knowledge is togetherness in time only refers to the sensory side where instances of difference impinge on the system. In these impinging streams of atomic or compound units of difference, in a given stream units arrive one after another. Between streams units can arrive together. That is, within the same relatively short time period.
Arriving one after the other or together are relationships in time. The units are related in time. But storage is removal of time, so to store the units and still retain (in some sense) their temporal relation, time has to be replaced by permanence, or absence of time.
So in storage the units are related by some permanent means of association. One possibility is spatial juxtaposition like billiard balls that have fallen into a pocket. But this is very limiting. Much better is some type of connective element. This means the units linked by adjacency in time when entering the system are not limited by adjacency is space in storage.
Storage is timeless. Time has been removed. What's stored persists. That's what storage means - absence of time. The togetherness in storage might be physical, spatial, a fiber for instance. Or it might be calculated in the sense that one compound item in storage contains the address of (points to) another.
That's the bottom-level primitive idea. The data compression algorithms build tree structures which means there's no repetition within the "forest". Of course the above explains very little and crucially nothing about semantic content. But the initial idea is to try and explain the very basics. That's turned out to be pretty hard. But talking about binary difference and not talking about information I think is a good start.