To be fair, this exact scenario happens to me all the time on actual courses. I look at the putt from behind the ball, and it looks like it's going to break right. Then I look at it from behind the hole, and it looks like it's going to break the other way.
In fact, I'll go on record as saying that the reason why I hate putting in this game so much is because it is too dang realistic.