I mean have you seen how the rest of the One Piece world treats most non-human races? (Or maybe you haven't even seen the worst of it, since the block didn't air the Sabaody arc.) Granted as we've just seen the Minks are far more able to defend themselves than a mermaid like Camie, but if you lived in a world where the ruling powers could enslave you on a whim, wouldn't you stay in your hidden (former) paradise of a homeland too? There are a handful of Minks like Bepo and a few others who live outside of Zou, but in Bepo's case he's a member of a powerful pirate crew so no one's gonna fuck with him.
Elephants' backs are usually dirty, right? This is one hell of an elephant, so there's a lot of dirt up there.
See here's the thing with One Piece, and I say this as someone who's been watching it for very nearly 20 years straight. (Christ...) It's not a series you can watch with the mindset of "I wonder how much longer it'll be until X happens." The bigger things will come eventually, but you have to live in the moment. "It's the journey, not the destination" is a massive cliche, but One Piece is the living embodiment of it.