Don't want to spend the money on a kayak just yet, but you're interested in trying one out? Well, of course there's the option of renting one at a lake or beach, or you can purchase an inflatable kayak for less than the $300-$500 the "real" one costs.
My friends used an inflatable kayak on a small body of water in Key West once, and it looked like fun. It held up well too, but I can't say that I wasn't a little nervous when they were out of my line of sight (in case something popped the kayak.)
My advice would be to definitely buy an inflatable kayak if you're interested in kayaking but not ready to invest a lot of money on one until you're sure it's something you'll do often. But, make sure you can be easily seen or heard by a friend or family member, and make sure if you plan to venture beyond that point to leave a travel "plan," like you would on any other boat trip.