Yep. In and out of the hole. You can hear it rattle around in the cup. It happens in real life on occasion.
Get in the habit of pulling the flag stick on approach, pitch, and chip shots.
I missed a pitch for bird yesterday on Congressional's par 3 #10 because I got lazy and didn't click and pull the stick. The shot hit the stick and away deflected away from the hole.