Sounds like you have the mouse buttons reversed, as that context menu is usually a right click option.
If that seems to be the likely cause, go to Control Panel, Hardware & Sound, Devices & Printers, Mouse, then you should find some options to change the left and right buttons over.
You may also get some help from their website, CLICK.