I've owned the following:

1998 Subaru Impreza Wagon
2004 Subaru Legacy Wagon
2005 Subaru Impreza Wagon (Current)
2008 Toyota Matrix (Current)

Subaru Pros:  Pretty good gas mileage for AWD.  Great handeling.  Great power (w/ standard at least).   Great acceleration.  
Subaru Cons:  The catalytic converter on my 2004 had to be replaced twice (under warranty).  I got rid of it when it needed to be replaced the third time (@120K miles).  The catalytic converter on the 2005 also had to be replaced at 30K.  Service was generally expensive, with $500+ every 15K miles.  Not designed for tall people.  Generally people over 6' seemed uncomfortable.  Newer models do not have key locks on the passenger or hatch.

Legacy/Outback Pro:  Great amount of space.  2005 and newer - the seats fold completely flat.  Comfy. Managed to fit 5 people w/ boating gear for a day trip to a river about 2 hours away.  Managed to fit 4 people w/ boating gear & camping gear for a weekend trip.  Ground clearance.  Heard it is OK to sleep in, I never tried it.
Legacy/Outback Con:  Depending on engine, not as much fun to drive as the Impreza.

Impreza Pro:  FUN to drive!
Impreza Con:  Not a huge amount of space (in comparison to the Legacy/Outback or Matrix)

Matrix Pro:  Great gas mileage.  All plastic cargo area is great.  Lots of space.  I feel like it has much more space than the Impreza in my side by side comparison.  Great tie-downs in the cargo area.  Also available in AWD for some years.  Actual key locks on the hatch and the passenger side.  Front seat folds down flat - fairly easy for someone to sleep in the car.  Toyota reliability and maintenance.  Feels a lot like a small SUV.
Matrix con:  Roof rack not standard.  Not a lot of power.  Slow acceleration.  Can't get AWD and standard.  Antenna on the top of the car.  Requires a non-waterproof electronic key to operate car.

I loved my Legacy, but I couldn't afford the maintenance and cats required at around 120K miles.  It was also very dependenable.  The Matrix seems very nice and very functional, but I still can't say that I like it as much as I liked my Legacy.  It looks like the ground clearance of the Matrix is VERY close to the clearance of the Impreza RS. 

Hope this helps.  -JK