I feel like in the right context, a ten foot tall guy could be pretty scary, it would need to be convincing effects and acting.
A ten foot tall guy ordering a sandwich or something wouldn't be scary, but if he was smashing up cars in a parking lot, I think I would be scared and would be well within my rights to be scared. Although watching that in a movie, probably not so much.