I've seen this when my swing meter gets choppy. The response time from clicking on the mouse to it being registered in the game (during the choppy swing meter) ranges anywhere from a fraction of a second to a full second to not registering at all.
If you haven't done so already, try to turn off the putting grid (which will remove the putting grid during the backswing of the putt).