For the most part, I don't dislike any particular character; most of them rate somewhere from thoroughly likable to forgettable or mildly irritating.
However, I rather dislike the concept of the Lunarians, exemplified in the characters Toyohime and Yorihime. Mostly, it's the whole Mary Sue thing, which has been discussed aplenty elsewhere, but also because they operate on a completely different sense of scale to the earthbound characters, and not in a manner that introduces interesting interactions or somesuch. Rather, the presence of characters that can nuke Gensokyo with a swipe of a fan or the like kinda diminishes the majesty of the setting, making it out to be a kiddy pool, with the moon being a family of idle rich tutting at the boorish behavior of the unsupervised roughhousing within.
It wouldn't really irk me, except that the entire thing seems so needless. It just seems like a tacked on addition to a setting that retroactively degrades everything else, without really adding anything of value. The earth-based Lunarians, I don't have a problem with, since they are neither explicitly or heavily implied to be flawless, nor seem to operate on a different scale (with the possible exception of Eirin, but then she did annoy me initially due to the whole attitude and "wouldn't have actually failed if not for the mistake of someone else" thing, until the end of SSiB (I think? it's been a while. the one with the whole party-at-the-SDM-where-Eirin-panicks chapter) made her more fallible, and thus relatable). The trip to the moon came across not so much as an opportunity to expand on the origins of these characters, which would be fine and dandy, but rather more like fanfiction written by someone whose lack of experience in world building or characterization has caused them to fall back on the old standby of both informing the audience of what their reactions and opinions should be, rather than letting them interpret things for themselves, and trying to get a positive reception on their original characters by explicitly comparing them favorably to existing and established ones.
I don't really mind that sort of writing provided there is some sort of redeeming feature involved; interesting character interactions, interplay, theories, anything that makes it interesting to read or think about the concepts introduced within. Unfortunately, I simply didn't get any of that in their introductory manga, and since it is still canonical, simply ignoring it falls into the category of headcanon, which means I have to recognize that it is indeed a thing in any discussion in which it becomes relevant. So, I dislike that particular aspect of the setting, and the two characters whom the majority of my issues concerning it center around.
Oh, and Suwako's hat is p ridiculous.