The Table That Isn't There

You slide your phone over a glass coffee table. The virtual plant pot you were about to buy wobbles, floats six inches above the surface, then drifts sideways like it's bored. You tilt the phone. You try again. Nothing sticks. Put that same phone over a wooden desk and the object snaps down in under two seconds, shadow and all. You didn't do anything differently. The app isn't broken. Glass, from the perspective of a depth-sensing camera, is about as readable as a window in a fog bank.

That's the whole problem, compressed.

What AR Actually Does Before It Places Anything

Before a virtual lamp can sit on a surface, the AR engine has to confirm the surface exists. It does this through visual feature tracking and depth estimation, sometimes with dedicated hardware like a LiDAR scanner, sometimes with pure computer vision on a standard RGB camera.

The visual approach hunts for features: corners, edges, texture gradients, the natural noise a surface carries. A painted concrete floor gives the algorithm thousands of things to lock onto. A pane of glass gives it almost none. Worse, glass reflects whatever is behind the camera, so the features the algorithm does find are lying about their location. The system briefly sees a reflection of the ceiling and registers it as a surface beneath your feet.

Depth sensors hit the same wall, just from a different direction.

Structured-light sensors project a grid of infrared dots and read how that grid deforms over a surface. Glass transmits most of that infrared straight through instead of bouncing it back, so the sensor gets either silence or a confused double-return from whatever sits behind the glass. LiDAR fires laser pulses and times their return. A clean pane of glass reflects roughly 4% of incoming light at perpendicular incidence. That is not enough signal to build geometry from. The system maps the room with a hole exactly where the glass is.

The Worked Example: Two People, One IKEA App

Sara places a virtual plant pot on her reclaimed-wood dining table. The app detects the surface in under two seconds, renders the pot with a convincing shadow, and she can walk around it. Realistic enough to make a purchase decision.

Marco tries the same app on his glass-topped kitchen island. The pot hovers ten centimeters above the surface, refuses to anchor, and when it briefly locks it slides sideways as he moves the phone. He ends up holding the phone at a steep angle so the camera can also see the floor, anchoring the object there, then mentally estimating whether it would fit on the glass. Two workarounds for a feature designed to remove workarounds.

Marco's experience isn't a bug report. It's a structural limitation. The app cannot manufacture geometry data the camera never captured.

The Reflection Problem Makes Everything Worse

Even partial glass detection introduces a second layer of chaos. An AR compositor has to blend virtual objects convincingly with the real scene, which means virtual objects need to cast shadows and respond to the room's lighting.

On an opaque surface, the system makes reasonable guesses about lighting direction from the image itself. On glass, the reflected environment shifts constantly as the camera moves. The lighting estimate destabilizes. Virtual objects end up looking like they were photographed in a different room and composited in, because in a meaningful optical sense, they were. Like a bad green-screen job from a decade ago, except the background is moving.

Water is the same problem with a speed boost. A still water surface reflects polarized light in ways that confuse color channels. Rippling water changes its surface normal thousands of times per second. No real-time tracking system in a consumer device keeps up with that, and honestly, expecting it to is wishful thinking.

What People Usually Get Wrong About This

Here's the assumption almost everyone makes: better hardware solves it. More sensors, higher resolution, throw money at the spec sheet. But the issue isn't sensor precision, it's physics. A sensor that receives no return signal doesn't benefit from being more accurate. LiDAR on a current flagship phone is genuinely more capable than the structured-light system it replaced, and it still struggles with glass for exactly the same fundamental reason.

The more promising paths are algorithmic. Some research systems use polarization cameras, which detect the characteristic polarization signature of reflected light and infer that a surface is specular even without a clean depth return. Others use learned priors: neural networks trained on millions of indoor scenes that predict where glass is likely to be based on context like window frames, table legs, or cabinet edges, then flag those regions for special handling.

Apple's RoomPlan API handles some of this by cross-referencing multiple scan passes and inferring surfaces from surrounding geometry rather than direct returns. It still labels glass surfaces with lower confidence scores and often marks them as walls. Functional, not solved. Those are different things.

The Confidence Score You Never See

Every AR surface detection system generates a confidence score internally. When you place an object and it sticks, the score cleared the threshold. When it floats or drifts, the system is essentially guessing out loud.

So when does AR feel like magic? You're on a textured, opaque, matte surface in decent light. You won the surface lottery without knowing there was a lottery. Glass, polished marble, and water all sit near the bottom of that confidence curve, not because engineers haven't tried, but because those materials treat incoming light as a suggestion rather than an instruction.

AR is extraordinarily good at the problem it was designed for: a textured, physical, reasonably lit world that cooperates with sensors. The pipeline was never built for materials that defeat it at the physics layer. Fixing that with software is like writing code to change how refraction works.

Glass has been defeating sensors since before sensors existed. It's patient.