Ssmile - with regards to building up speed, yes a lot of it is dependent on fitness and strength, which you can either build up through following a set interval training program, or just incorporating one into your normal riding. For example, if you have a long straight stretch, try doing mini sprints- 20 pedal revolutions or a couple of lamp posts, then ease off until you catch your breath, then go again.
As others have said, a good pair of slick tires do help, as does working on your technique. Cadence (number of pedal revolutions per minute) and pedalling technique will both help with speed.
Do you use a cadence sensor? If so you should be aiming for a cadence of 90 - 100 revolutions per minute. If you don't use one then you can gauge it roughly by effort. I find that about 120 revs is about as quick as I can sustain on the flat, and that really feels like my legs are going like the clappers. Get in an easy gear, see how fast you can get your legs going. Ease off a bit from that, and you should find a rhythm that suits you. Don't be tempted into thinking that the bigger gear you are in, the quicker you will go. If you are mashing round a big gear then you will use a lot of energy not very efficiently, and your legs will get tired before you build up speed.
The other thing is to think about your pedalling technique - do you ride clipped in? If so, don't just push down on the pedal but think of pulling through on the bottom of the stroke as well. Imagine you are scraping your foot back on the floor to get something off your sole. You will engage more muscles and for longer, making your stroke much more efficient.
I hope that helps! I haven't been riding for all that long, so this is all stuff that I am still working on.