There's a small possibility your game could get hung when someone disconnects inappropriately (not through the menu). That would break up the remaining players in your group leaving everyone to restart or finish out solo.
Positive effects include possible reduced lag (less connections) and not having to play with the guy hacking it up anymore.
Aside from that, no, it doesn't affect any elements of your game or your score.