I don't think this has really been gone into much in any translated material. (I've heard it rumored that Bohemian Archive's article about the Hakurei Shrine is also relevant, but there's no translation on that yet, so what can ya do.) All I can think of offhand is two quotes about it.
Reimu, in MoF: "I wonder if my shrine's god is any stronger?" So apparently she believes she has one, but doesn't know much about the specifics. Edit for curse you, rascally Tonepoet!
And Rinnosuke, in Curiosities of Lotus Asia naturally. Sakuya calls the shrine godless, Reimu objects, and he narrates, "I?m probably the only one that knows about the origins of the Hakurei Shrine. Here I thought about teaching it as for restoring Reimu?s honor, but? it?s not really important, so it was rejected. How sad." Which mostly tells us that he's a huge tease.
So in conclusion... I know no more about it now than when I started. Except, I suppose, that it's probably intended to be mysterious, not just an oversight. Either way, it's some nice fertile ground for speculation and wacky theories!