It won't stay on top of the game, though. If you click on the game screen when you're done, as opposed to hitting the X on calculator, the calculator will minimize to your taskbar.
If you play in full screen and can't see the taskbar while you're playing, hitting the Windows key next to the ALT key will show you that taskbar where you can click on the calculator.