Why have a barometric altimeter?
NOTE: A much more current version of this post can be found here.
Quite simply, the primary reason you would want a barometric altimeter in a handheld GPS is for more accurate elevation readings. This is especially useful for bikers, hikers and atheletes in training. You can use the resulting data to track your progress and maintain an accurate record of your trips. A barometric altimeter can also help you accurately place your location on a topo map.
Furthermore, you can use the altimeter to forecast weather trends. A falling barometer can indicate worsening weather. See the barometer section of this review for some really cool things you can do with your GPS!
GPS-based altitude error is generally worse than latitude/longitude error. A good explanation of why barometric altimeters are better than GPS-based elevation data comes from GPSInformation.net. Here is another good resource — an electronic compass and barometric altimeter FAQ from Garmin.
Finally, a barometric altimeter is often paired with an electronic compass in higher end handheld GPS receivers.