Could it be to do with the seams? They need to be snipped with scissors so that the stitch line is smooth, especially if theres lots of curves. Always press your sewn lines too before you turn it right way out.
On the little hole thats left I do a running stitch to use as a guide to follow when closing it up. This is helpful on a curve as it stops it distorting.
As I understand it, you are talking about the final step, over-sewing the seam?
Right, couple of basics, firstly think about where it is least likely to be seen, eg the crotch or the inside seam of a leg and make sure that the hole is as small as you can make it. Now when you sew it up, make sure that the thread measures from your finger tips to your elbow (approx 18"), use the finest needle that will go through the fabric, then fold the seams in and do a little back stitch on the inner side of the seam. You are now going to use a hemming stitch to join the seams together, try to make your stitched as small as possible, same size and equally spaced - on this tutorial, this lady is making them big so you can see what to do.
The more you practice with hand sewing the better it gets... honest!
I think you might mean a 'ladder stitch' - (great tutorial here which has helped me recently). I use it if I'm doing a cushion cover or something, and have sewed up three sides fully (while inside out), then a bit of the fourth side, turn rightway round and then 'ladder' stitch them together as it's totally invisible.