This used to happen to me intermittently before I started using a controller, and was indeed one of the reasons I switched in the first place. Always with the left arrow or the shift key, though in my case, simply pressing the affected key again would make it stop, so maybe it's not the same thing. What I did notice was that it never happened in games where I didn't use focus, either PoDD or no-focus runs in something else (that is, not pressing shift solved the problem for the left arrow as well as the shift key itself), so I think the shift key may have been behind it somehow. Something that might work (though I never tried it myself) is remapping your keys to, say, focus with z, shoot with x, etc. I can't promise that'll work but it may be worth a try. Otherwise I've no idea. Though I guess you could always join the dark side and start playing with a controller as well. Might not be ideal for scoring though.
(Why am I awake right now?)
edit: Seppo got in before me, but everything I said should still apply I guess. I'd assumed you'd have already tried pressing the same key again, but yeah, make sure to try that if you didn't. And I don't think this can happen with controllers (as you can probably gather from what I said already). At least, I've been cured ever since I switched.