Most people won't. Since the brain can't usually distinguish stereo sound from front to back which much detail, it still processes as being in front of you. From left to right, it generally doesn't matter unless it's associated with a video where sounds coming from off screen could be confused. I typically do make sure that I have the the correct monitor in each ear, but it doesn't affect the over quality of the sound or the monitors themselves if the ears are reversed.