A number of my closest friends are people that I have been paddling with for many years and the only time when I see them is during paddling and there's a whole comraderie thing that goes along with the sport.   In fact most of my friends are people that I paddle, ski, hunt or go birding with, so the events become more than just a personal activity (or maybe that's just an excuse.)    But, for this reason, paddling doesn't seem quite as selfish  as some other activities such as running which is usually done alone (and then after a long run I don't have any energy to socialize the rest of the day.) 

Like a coupla other posts said, the downside of paddling for many of us is the long drives which means longer times away from home  compared to other activities.