PSP and PSP Vita are both much more powerful than the DS. Bigger screens, more memory, faster processors etc. The PSP is quite old though and I don't think any (or, at least, not many) new games are being produced for it. There's still a good number of older titles for it though.
The PSP Vita is the PSP's replacement and adds touchscreen capabilities, motion-sensing, better online capabilities and other whizzy new features. PSP games will work on the Vita if you can physically get them loaded on to it. Original PSP games came on tiny CD-ROMs. The Vita doesn't have the miniature CD-ROM drive and requires games to come either on memory cards or as downloads. PSP Vita games won't run on a PSP.
Being charitable one could say that the Vita's sales have been disappointing. Being a bit more frank, its utterly bombed on the market. Which is a worry as if the punters aren't buying it then the games producers won't bother making games for it.
To be honest, I'd try to avoid buying either right now. If sales of the Vita don't pick up markedly (which, so far, seems unlikely) then I suspect it's going to be a bit of a dodo.
The handheld game console market is in a bit of a mess all round at the moment. Many customers are moving to smartphones for portable gaming rather than dedicated games machines. If you can buy a good game for your iPhone or Android phone for a couple of quid, who's going to pay £35 for one on a DS or Vita?