Andrew, just get it done. I started Masters racing last year and immediately regretted not starting years ago. It's an absolute blast! Over here (in the UK) you get seeded by start time so you tend to swim with people of equal ability. And so far all the swimmers I've talked to at these meets have been really possitive and just lovely people.
So just go for it - and good luck!