I wouldn't worry too much about confusing a 2.3 year old - my children were interested in letters from much younger than that, and every time they pointed at a letter, I'd say "that's an a and a says AH", or whatever, just as I'd say "that's a ball" or "that's a car". I'd quite often also add something else, like "apple begins with an a - a says AH for apple...", or even come up with a whole list of words beginning with that letter. I favoured talking about the name of the letter and the sound at the same time because most children learn the alphabet song at pre-school, regardless of whether they recognise any of their letters, so I thought it would help that they knew the names of the letters as well as their sounds, so that they could connect the song to the symbols.
My children were obsessed with jigsaws and peg puzzles, and we were given alphabet puzzles of both varieties, hence the endless opportunities to talk about letters from a very early age... It certainly worked with mine - they know their alphabet forwards and backwards and the eldest rather anally knows what number a letter of the alphabet is (eg u is the 21st letter in the alphabet...).
As for whether they learnt lower case or capitals first, I think they probably learnt their lower case letters first, because the peg puzzle was in lower case... I also think it's probably more useful, because you see more letters in lower case day-to-day. The jigsaw had capital letter and lower case next to each other, so I'd point out that both letters were actually the same. (Which all makes it sound very complicated, but all I can say is that it didn't confuse them at all, and for them it was all just part of Mummy's usual witterings about the world, rather than a formal learning process, so gradually it sank in, without anyone noticing when or exactly how).