If it's a grinding sound it could just be the break pads making uneven contact with the rotor disc.

Had that problem with our Rav4 when we still had it, mechanic said it was because they used poor quality brake pads. First thing I noticed is that there was a lot of brake dust even on short trips.

Does it usually happen at slow stops and starts at traffic lights when you're riding the brake a little bit?