The host of the lobby (the player who set it up) has the rights to remove ("boot", "kick", etc.) anyone he wants to from the lobby.
Typically, if you are at a higher tier than the host (Legend vs. Tour Master, Tour Pro vs. Amateur, etc.), then you are probably going to be getting kicked.
Solution? Create your own lobbies.