To be quite honest, there is a given percentage of the Touhou fandom that is just excessively creepy and awkward enough that they probably deserve, or at least should see coming from a mile away, the flak they might get for being into the series, especially if they don't play the games enough to show other people they mean business. I mean really. Not talking about the games in this instance, but you see a screen filled with girls of questionable ages doing something stupidly "cute" with colors everywhere and squeaky pikopiko unicorn rainbow barf music being played in the background and all that sorts of crap and people really do have to wonder. It isn't shallow-minded to question it; to label people who won't immediately "understand" you as "shallow-minded" is shallow itself. Touhou has a bajillion characters and no-one's going to get that everything is from one game series, nor that it's even a game to begin with, nor will they accept it just because they know it's based on a game. Besides, if you aren't actually into the games, you have no excuse since 80% of the fandom isn't even about the games. The offline behaviour displayed by a lot of Touhou fans isn't normal, even among anime enthusiasts and friggin whatever. What people will get immediately is that there's probably porn involved at some point, and no matter how much you may try (or not try) to deny you're interested in the porn you can't seriously think you're going to convince anyone. Newsflash, 99% of the world is not going to see things your way, whether you think confusion, misunderstanding, intolerance and judgment towards you is "shallow-minded" or not.
What I seem to be noticing (however little information I have) is that there's no actual explaining going on when you talk to these "shallow-minded people" (even if they are, quotation marks needed), just "i'm not a pedophile really, here look at somebody else's replay of the game so I don't look nearly as strange". Even if you can't negotiate with some of these people, explanations are outright better than denial.
If I play a piano arrangement from a Touhou song, my mom might ask me if it's from "that game". I answer yes, and will actually go load up the game and play her the music as comparison. She's not going to confront me about it anytime soon as I don't act odd enough to warrant it, but even if she did, I'm not going to deny there isn't something really strange about this fandom, and there's no way I'd be able to explain it without going "yes there is a lot of porn, i'm weird, deal with it". She knows I've made friends over the internet and have connections to many countries across the globe. She knows I get mail from weird places, and she knows I play weird games and watch weird shows. I may not know what some of you are talking about with such radical peers/parents, but I don't exactly see anyone complaining about Touhou flak being very open-minded about people that aren't them.