I get the feeling Jimmy was at least partially responsible for Chuck's wife leaving. It's the only thing I can think of that would make Chuck this awful. At least it would be something to make him potentially relatable again. Not that what we got was enough to justify his hatred of Jimmy, but it was enough to understand where he's coming from. Now he's really just punishing Jimmy for fucking him over once after he fucked Jimmy over for years.
It feels like there has to be more to it than what we've been shown. Or at least I hope it is.