You should do what I do and almost completely avoid Touhou fandom. This is also a fairly good course of action with just about any other fandom too, since most of them are fairly unbearable. Or you could find a relatively small group of people whom you can still talk about your interests to.
Also, stop reading Youtube comments.
It's a really shit manga, though.
I just don't think you can appreciate ZUN's true genius. Invading the moon to steal some sake? Absolutely flawless.
There's more than SSiB though, SaBND and EaLND are a much better and fairly enjoyable read, and show Gensokyo as the carefree, peaceful place it really is.