Partially repeating other comments but some slight adjustments...
You pay for a Minecraft username/account - this lets you download the game to multiple machines (many PC's at least, not sure about the xbox edition and if you can cross-over 'tween xbox and PC). You should only really be playing from one PC at a time (don't share your username/password around).
The game lets you have single player worlds stored on the PC that you're playing on - in these worlds you are basically alone and free to do your stuff all by yourself.
The game also allows you to connect to servers run by other people - here you're in a shared world. There's no in-game talking (text only chat (again, I'm thinking PC here, xbox is probably different as niceguy2 notes)).
Unlike other online games you can download the game server software and run it yourself - while 'proper' hosting (rackspace, as already mentioned, is good - but never cheap) is good for lots of players connecting, it's perfectly possible to run it on a PC at home over a decent cable/DSL connection.
Game servers (whoever is running them) only really connect back to the main minecraft servers to authenticate users as they connect - no constant connection is needed.
We run the server on my PC (where DS1(7) is also playing), DS2(5) on DP's laptop elsewhere in the house (hence on the local network, not over the cable connection) while their grandpa (who is also quite obsessed) connects to the same server from the other side of the country.
Servers can be configured with a whitelist of accounts that are allowed to use it - this could (as in our case) be a very short list of names.
You're never going to get 10's of people running to a minecraft server over a residential connection, but for a few it's fine.
I'm quite willing to assist with any Minecraft-ish bits if it's of any use to anyone.