Well, dammit, woman, tell him where the boundaries are and don't let him cross.
Family, sort of family, or not family, it doesn't matter. If he's ignorant, then by all means enlighten him, but if he's making you uncomfortable enough so that you're willing to call it sexual harassment, you shouldn't have to put up with it.