I wouldn't say WoW, actually. The proportion of "normal" people to geeks in WoW is actually surprisingly high, whereas the Touhou fanbase in Japan, as far as I can make out from Nico Nico Douga, is pretty much 95% geeks.
Probably more accurate to say it's like D&D, in that a lot of people know what it is, a lot of people have grave misconceptions about what it is, and the number of fans is big but not THAT huge.
I'm pretty sure that if you tell the average Japanese person you're a Touhou fan, the odds are about:
50% - they won't know what you're talking about,
40% - they'll assume you are a deranged potential serial rapist and pedophile who has a 100-strong collection of hump pillows,
7% - they will actually have a more balanced view (and may be an otaku themselves),
3% - they're a Touhou fan too.
Of course the numbers are a bit different if you only consider guys.
I'd guess that at least half of Japanese otaku/geeks, and an infinitesimal proportion of nongeeks, are interested enough in Touhou to be considered at least borderline fans. That makes about 4 million people, roughly speaking (ballpark figure), going by NRI survey data (which places the otaku population at 1.7 million as of 2005) and extrapolating via educated guesswork for borderline otaku, liars, and growth since 2005. This sounds like a shitload of people, and explains the crowds at Reitaisai etc.
However, the population of Japan is 127 million. 4 out of 127 is not a lot.
Data from
http://www.nri.co.jp/english/news/2005/051006.html (disclaimer: I have no idea whether these figures are pulled out of their collective asses).
edit: the 7% didn't linebreak properly. bleh.
