andyson: I think you're forgetting that VEM doesn't kick in until it sees your skill level increase/decrease over a certain threshold.
This is where you and I are thinking different. I don't see VEM as some evil lurking in the background waiting to strike-it's there all the time.
The best way I can think of to analogize what I am trying to say would be to think of the equation that a shot produces. It's filled with numbers that are generated by the shooter-aim, swing strength, club, etc. Those would be hard numbers. VEM is included but as a variable that changes based on tables generated by your scores. All of them, not just the one in the game you're playing.
To my way of thinking, VEM is an error factor that magnifies your mistakes. I don't think it takes a perfectly executed shot and destroys it to keep you in line. The low scorers just keep those mistakes to a minimum and correct well when they happen.