Boxers, maybe.
I used to do boxers/basketball shorts and an undershirt/wifebeater......Unfortunately, where I live, that screams "gang member" to the neighbors when I go get the mail, so I've stuck to boxers and a shirt.
I kinda have to keep it minimal, because heat in the summer is a bitch.
In winter, I stick to lounge pants and a shirt, if it's cold