Luckily the switches take more effort to depress than say, a window switch or radio button. Plus they're in a good location where they can't easily be bumped.
That would suck though... I like to cruise with the top down and the A/C on... so that plus butt warmers by accident would confuse my body into oblivion. >