I recommend you start counting calories. The majority of people who think they have fast metabolisms usually don't eat as much as they think they do. Myfitnesspal is a free app you can use to track calories. I use it.
I only started to workout again. This time, I am giving myself more time to recover. I am only working out 4 or 5 times a week. The last time I regularly worked out I was doing 6 days a week and my body couldn't handle it. Keep in mind that my goals were always weight loss, whereas yours seem to be maintenance.