Say the game only bases your average on your top 20 scores. For math purposes, say those are 10 62s and 10 65s. That would make you average 63.5. If you shoot a 64, which is above your average, that 64 will replace one of the 65s that are comprising your average. Thus, your average will go DOWN despite shooting above it.
Does that make it any clearer?