UFO's my favorite in the series, by far. The artwork has definitely improved, and the backgrounds are great (Stages 5 and 6 are gorgeous).
The UFO system is definitely my favorite scoring system, mostly because it gives you so much diversity in your playthroughs. In the previous games, how you performed was dependent on where ZUN put the bomb/1UP items; in UFO, it's entirely up to you.
One of the best parts of the game IMO is how little memorization is involved. Almost all the danmaku is aimed at you, or moves depending on your position. In the previous games (especially SA), you just memorize where to move and hope you don't get hit. UFO challenges you to really think about where you're moving and to quickly adapt to whatever's thrown at you.
The music tends to range from forgettable to FUCKING AWESOME. Byakuren's theme is my favorite song in the entire series, closely followed by Stage 4 theme. I love that ZUN's trying different things with the music, not just giving us the same-sounding, oriental sounds.
The only song I blatantly dislike is Nue's theme. It's basically a demented U.N. Owen Was Her?. It just...really annoys me for some reason :\
Most of the negative reception I see on the game comes from Lunatic players. Frankly, I don't think I can really blame them, UFO on Lunatic is just insane.