If you setup a game, invite someone and they decline, their name does not get removed from the list. You have to cancel the game and start over..