Whispering at Family Gatherings

I often leave family gatherings with a bad feeling after observing two or more family members whispering to each other.  I've been thinking about this for several days and even looked up the word whispering in an online dictionary.  The verb form means to speak softly, or to speak quietly and privately, as by way of gossip, slander, or intrigue.   It also means to say or tell privately or secretly.  No wonder I have a bad feeling after seeing this happen in my family. 
I personally think private whispered conversations are rude when sitting at a table, or in a room full of family members.  I'm not talking about the occasional whispered words between spouses or a parent and child when it's necessary to convey a short message.  I'm talking about two people sitting next to each other and speaking quietly to each other for a period of time.  Sometimes, I can't help but think it is done intentionally to hurt other people by excluding them.   I hope it isn't intentional but only thoughtless. 

I have discovered that whispering at family gatherings isn't only bothering me, but other family members as well.  I've been contemplating whether or not I can do anything to stop this kind of hurtful behavior when our family gets together.  So far, I haven't come up with a solution.  I'd love to hear my readers ideas.  Thanks for reading and I'll watch for your comments.