I've been a swimmer for about 8 years and I still have belly fat...?

I've been a competitive swimmer and on swim teams for about 8 years (i'm 18). I realize that swimming is a great sport because it exercises your whole body. So how come I still have fat on my back, chest, and stomach?? It's kind of sad, especially compared to my fellow swimmers who have such flat stomachs. When I sit down, the fat on my stomach folds over the waistline of my swimsuit and it looks gross, haha. Not only that, but it seems to be working with the rest of my body because, well, I'm not a fat person at all. If you saw me on the street, I would not look fat at all, I would actually look very skinny. My arms and legs, etc. are in good shape and toned well. Is there something I'm doing wrong? I swam about 4-5 times a week, about an hour to an hour and a half each time. Why do I still have so much stomach fat? And I know nutrition has to do with it, and I do try to eat healthy. My mom is a health freak, so I only eat brown rice, whole grains, and healthier meals and some vegetables. Of course, a few times a week (maybe like 1-2 days) if my mom runs out of dinner ideas, we'll have burgers, pizza, etc. But I don't indulge all the time. Please help me out! Is there something I'm not doing right at swim practice? I'd really appreciate any tips and advice!