Yes, backgrounds can cause lag. That's one of the reasons that Cirno's stage is used as the default for some PoFV competitions. Not sure how much of an fps improvement there would be with them removed, but there would be an improvement.
I'll join the op in wanting to know if there's a simple way to disable the background layer(s) in the various games. I don't need an fps improvement, but I'm curious how much of a visibility improvement there would be.