Musetta - if you can at least sometimes get to a session here and there, you should be ok. You will pick it up more slowly, but nearly every place I've been starts every beginners lesson from scratch, to go over it for the ones who have been before and to introduce newbies. Each time it will be from scratch for a different basic step, if that makes sense.
The more informal Lindy groups tend to be the ones that are Lindy/swing only, rather than part of a dance school. For example, the best ones round here are Manchester Lindy, which meets in a huge Masonic hall, Mersey Swing which meets in a nightclub function room and Lancaster Lindy Circle , which used to meet above a pub and now meets in a village hall. It's a lovely community, we ran into people from all three of those at Hullzapoppin, and we get to whichever one we can on nights we have free.
There are some exceptions - Mersey swing sometimes uses a room in a dance school and I did a few lessons run by an outside swing teacher in another school - but that is a good rule of thumb. Any teacher who is funny about you going to other places (and especially other socials) is rubbish.
The swing family of dances has some basic differences to ballroom and Latin - lots if people do all three, but don't be fooled into thinking they are the same. Someone once said that ballroom is graceful, Latin is sexy and swing is joyful 
It is very hard to Lindy without smiling...