Brakes.  Disc.  Get them.  They are the only true brake anymore.  forget those damn rim-clinchers, they suck by comparison.

Suspension.  Full.  Get it.  Not the power-robbing, life-sucking, overweight, overpriced failure-prone hell it used to be.  Look into full-suspension rides with SPV or other comparable technologies to prevent your pedal force from driving your suspension- and also realize they've got lockout options- meaning you can have the option of riding rigid.  Don't let your snobby hardtail "purist" buddies tell you their way is better.
You can take your full-suspension bike on the street.  You won't want to take your rigid bike on the trail.  Get the squishy, you'll have more fun and more fun == more exercise.

Tires.  Big ones.  Get them.  Big tires provide more rolling resistance in a straight line, but that's never the limit to your speed unless you're going in a straight line.  In all other cases, big tires make you a faster rider.

Clip-in pedals.  They are awesome.  there's a learning curve, but there was also a learning curve to ride a bike, too. 

Look for reviews on XC mountain bikes- you don't need to pay top-dollar for a freeride or DH rig.  I have a Giant VT2, which is a little on the plush side of the XC range, and it's the most fun bike I've ever owned.  Brands to look at:  Jamis, Giant, Trek, Fischer, Specialized.   You should be able to find something in the $1k-$1400 range that weighs ~30 lbs, has 5" front and rear, with 2nd-generation or lockout-available rear suspension at least.  Used, you can probably pay half that.