This depends on a few things
Assuming there was nowhere else for them to go: If there were no barrier, then the youkai would have quietly died out due to modernization just as they had been doing. Some of the deities might have hung around for a bit, but given Kanako and Suwako were at the end of their rope, I doubt many would last long. Now, whether magic would still exist is an interesting question.
Is Makai an entirely separate world, or part of Gensokyo? I could see Yukari, at least, using it as a refuge. How well mortal world youkai would do in Makai, that's the question.
Do other separate world refuges, such as Shangri-La, Mu, Avalon, and so forth exist? I could see some Gensokyoan youkai fleeing to those instead, if they could.
But if there's nowhere to go, the youkai are pretty much screwed. Faith has power over them; and if everyone's faith is in the rational, observable, scientific world...well...there you have it.