I keep mining when I release the mouse button

If I'm not mistaken, your computer's OS or the driver's software for the mouse is set to have holding down for a bit of time to "lock" in the pressed position. Check your mouse driver's program and in your computer's control panel (the Mouse and Accessibility sections) to see if a setting is causing it. Using a laptop's touchpad is a common source of the issue as I have also witnessed it myself.

But if it is just like you said to the response of sniffylongnose, then I think your computer may be a bit overworked by Minecraft. Try to lower your graphics settings first and play without mods on a new single player world for a bit to see if it still happens. If that fixes it, or even if it doesn't, then your computer may not be strong enough to run the version of Minecraft you were using. Also remember that mods are a common source of demanding more processing power from your computer.

I have had this problem too. What people don't realize is that Minecraft, despite having very basic-looking graphics, is surprisingly PC intense, especially if you're playing at higher graphic settings and with the widest view range you can get. For non-gaming-dedicated PCs, it can cause stuttering, chunk-loading problems, and problems like this one. If your game has performance issues beyond this one, this could be your problem*. Especially if you're noticing other performance problems along with it.

The easiest thing to do is to turn down your view rendering and graphics settings. This will reduce the resources Minecraft needs to use and help with performance. Using a Vanilla version will also help.

But if it doesn't, you may need to upgrade your computer. There are three things that might be causing an issue here - your Processor, your Video Card, and your RAM. Compare your system to the system requirements that I just got off of a quick google search and see if they're sufficient.

• CPU: Intel Pentium D or AMD Athlon 64 (K8) 2.6 GHz.
• RAM: 2GB.
• GPU(Integrated): Intel HD Graphics or AMD (formerly ATI) Radeon HD Graphics with OpenGL 2.1.
• GPU (Discrete): Nvidia GeForce 9600 GT or AMD Radeon HD 2400 with OpenGL 3.1.
• HDD: At least 200MB for Game Core and Other Files.
• Java 6 Release 45.

System Requirements for Minecraft

*Otherwise, it could be a mouse setting, since most games do not require you to hold a mouse button down for very long periods of time.