I'd give Vice City high marks off the virtue that the soundtrack & atmosphere were top notch, but V was one of the most enjoyable experiences I've had with the controls and missions in the series. I liked IV and the tone they went with, but but I feel like V made more improvements and Liberty City wasn't as lively or memorable in my book as Los Santos.
San Andreas was good, but there were way too many things that turned me off from the game. They added so many things that pissed me off like the plane/driving school missions, the turf wars, and the weight/muscle system, and had even more annoying missions. The newer games had shades of that too with IV's cellphone and V's stock market, but the stuff that made SA hard to want to play again is much more harder to overlook.
It's stuff like that that makes the PS2 games harder to revisit, even Vice City with some of it's annoying missions. IV and V are the only games in the series I would gladly replay the missions from start to finish just off the basis that the controls & scenarios won't make me want to chuck the controller. I just return to VC for the scenery and music.
That being said, Sleeping Dogs and Saints Row II take the cake for me as far as sandbox games I dig the most.