I have always just moved it out of the way after calculating speed and then began reading putts. What I find more annoying is that I move it off to the right and there isn’t much room with the greens button over there, it switches me sometimes when I don’t want to switch and during a coin game that can be crucial time.
i say, yes it is annoying but just move it out of the way to read the putt.