Newborns are programmed to feed frequently, to get your supply up. If you're worried he's just comfort sucking, check whether he's actually drinking - can you hear him swallowing, is his throat moving etc? If you Google something along the lines of signs of drinking breastfeeding, you might be able to get a better idea. Good sites are LLL (as mentioned) & Kellymom.
If he is just comfort sucking, it could be he is 'slipping off' slightly, so not stimulating your milk, but making you sore. Please don't introduce a dummy at this early stage, as it will affect your supply & could affect the latch, causing more problems & even end your bfing journey. They usually recommend waiting 4-6wks, but I introduced a dummy at about 2-3wks with my DD & it was fine.
If you are really, really sore & cream, letting your nips get fresh air, using cool packs &/or soothing things (tommee tippee make some I think) doesn't help, then you could try a tiny bit of formula. But again, at this stage, introducing a bottle could impact on your supply & his latch, & if he has too much, the larger volume of formula taken, compared to the colostrum/early milk, might stretch his stomach & result in a greater hunger.
It is hard, the first 2-3wks are a massive learning curve for you both, but every feed you provide now really sets you up for continued BFing. Hth.