I was worried about exposing my DC to more chemicals than necessary . I bought organic sheets and organic cotton waterproof mattress protectors for DD's Moses basket and cot - all of which were lovely, and didn't cost that much more than ordinary versions. When DS came along, I bought him an organic moses basket and some new organic sheets.
The sheets are still going strong (some have been in use now for nine years), and the Moses basket was sold on at a good price. The only thing which didn't last a long time were the mattress protectors, which ended up, after lots of washing, not being waterproof but we still got a reasonable amount of use from them - 2-3 years if I remember correctly - before this happened).
DD had a ordinary mattress due to the cost of organic mattresses. By the time DS came along, I had discovered a natural (lambswool and coir) cotbed mattress which came with a zipped waterproof cover. Again, it wasn't that expensive and I think that it has been a really good buy quality-wise. It is still going strong after more than 5 years and DS will outgrow it long before it wears out.
I would really recommend buying a cotbed (ie one that converts into a toddler bed) rather than a cot if you are going down the natural mattress route as you will get much more use out of the mattress and the bedding. My DS (now 5) is big for his age but is still comfy in his cotbed and we should get at least another 3-6 months out of it.
The Little Green Sheep company was where we bought most of our stuff. The natural cot bed mattress came from the Cot Mattress Company and was much cheaper than organic mattresses I priced elsewhere.
Some people will probably think you are being unduly fussy but it is your baby and if you have the money and think it is worthwhile, do it and don't worry about it. Organic stuff tends to be made from ethically grown cotton which is another bonus.