Yes it will, until you have erased all those "very high" scores.
Tamsach: I've heard that now, if you win a bracket, you get XP points, helping you to climb levels.
Rumour! Never heard of that.
Tamsach: As well I know that mp won affect the average, but I never noticed changing in the average.
This is only true if you are below Legend tier.
Thus, it's not a mystery nor a bug, only saturation as Jim stated.