Always replace the Thermal paste whenever you remove the cooler. It sounds like you're using a stock intel cooler. You can either try to zip-tie it down(somehow, idk) or just get a third party cooler.
Gotta tell you, I hate the stock intel coolers. I would get something that screws onto the motherboard. Make sure whatever cooler you're wanting to get supports your socket if you do this.