When you say company, I'm assuming that you just mean setting up a business, rather than an actual 'company'.
You only need to register the name with companies house if you want to be a limited company. This isn't really worth doing unless you are going to be turning over thousands, not least because it involves having to file additional tax returns which you would most likely need to hire an accountant for. You may however wish to register the name as a dormant company to protect it if you start to become successful. You would still have to file accounts but they are much simpler for a dormant company and you could continue to trade as a sole trader.
To set up as a sole trader you need to:
Come up with a company name - try and make this as original as possible - check Google for other businesses with similar names. Also check that the web domain name is available. and register both the .co.uk and .com even if you do not intend to set up a website (speaks from bitter experience) Your business name is your most important and valuable asset - sounds corny I know but very true. A good eye catching, memorable logo is good too.
Set up a sole trader bank account - you will need a business name to do that.
Register with HMRC for tax. You will need to decide what period you are going to run your accounts from - most people choose Apr - Mar or Jan - Dec. You will need to do your first tax return by Jan 2013, either for the period from now till end of Dec or now till end of March.
The self assessment is relatively simple but if you are not confident you will need to find an accountant - look for a sole trader accountant who specialises in sole traders rather than a firm and go on recommendation if you can. Talk to other local sole traders to find out who they use.
Keep all receipts for purchases made for the business. If you buy things from car boot sales or other places where you may not automatically get a receipt ask for one or self-receipt (note down date, items and cost of each item, location bought). Include petrol for going to suppliers, craft fairs etc.
Keep records of all sales (Date, item, price).
Remember when doing your tax return to work out a percentage for the use of your home for running the business (space, heat, light, phone, broadband etc). This is where an accountant comes is useful.
Good luck - I remember when we started our business, it is a very exciting time.