I agree. Do a 15% water change (don't forget the dechlorinator) every other day for a week, and then a 20-40% water change weekly, in addition to having a good filter. I like a 50gallon filter or two 20 gallon filters for a 10 gallon tank for goldfish. I have one submersible one, and one that hangs over the edge, to also stir the surface of the aquarium (oxygenation).
What kind of filter do you have? A submersible? One that hangs on the edge? An gravel filter?