Urban fantasy is fantasy set in a city in our world, specifically. That's the main difference for me between it and straight up epic fantasy or whatever.

Though that said, there are plenty of fantasy novels these days that follow the tropes of the urban fantasy genre while having more traditional fantasy settings.

And you're right, many things calling themselves "urban fantasy" are really thinly disguised paranormal romance. Which is nice if that's what you like to read, but it drives me a little spare, too. ;)