It means that once you accept Jesus as your saviour, your name goes in the book - you've given a pass into heaven.
That happens once only, the moment you become a Christian.
After that, you will continue to sin, because humans aren't perfect, and will still be tempted. You can't go to hell though. You only have to accept him as your saviour once and that's it.
That is the Christian teaching.
I think it's a little misleading to say a child is born with sin. It's not wrong, just misleading. It's more accurate to say humans are sinners. It's inevitable. We will all sin in the eyes of God, before being Christian and after. So yes, a child too will naturally be a sinner, but that's not the same as the child having actually already sinned per se.
The theory of why Jesus died so that mankind's sins can be forgiven is very complex, and I do not fully understand it (perhaps because it doesn't actually make sense). But in theory, Jesus took on all the sins of mankind - past, present and future, but in order to have our sins forgiven, we have to ask him into our lives and have our slate wiped clean), It's a bit strange to say the least, but that's the basic premise.
One of the common challenges that non Christians put forwards is 'I may as well sin all my life, and just ask forgiveness on my death bed'. In theory IF you truly asked for forgiveness on your deathbed, you'd receive it, but you can't fake it. In the same way (and this is very perverse) IF you became a Christian at 18 then went on to become a serial killer, you'd still get into heaven!