Well that’s at least an inch of electronics and lens

Surprisingly not. Look at the previous link.

and doesn’t address the focal distance problem.

This is lens related. For thin lenses you sacrifice quality.

You’re putting the cart well before the horse here, automatically swapping from AR to immersive VR is the last thing to solve.

I was addresses a specific question about VR using AR glasses.