<p>If a hacker has access to the hashes in my DB, he has access to the rest of the information in the DB anyways. So why would he bother trying to decrypt the passwords? Should I be storing the passwords on a different server to the rest of my data? That is the only scenario in which I can envision it being useful. </p>

Sometimes a hacker doesn't get full access to your DB.  Sometimes they find a little SQL injection hole or other weakness that someone didn't code correctly, and so they can only do simple things at first like print out database cells one at a time.  If they can print out a real password all of  a sudden things get much worse.
Things happen: backup tapes are lost, accidentally thrown away, or stolen. A retired system wasn't wiped properly. A breach elsewhere leads to accidental exposure of a database. If a hacker gets access to a snapshot like this he can learn a lot about your system.  But if the passwords are still hashed he can't also use the system to do something malicious, like log in as a different user and start changing things.
I've heard that most hacks are an inside job.  Better to remove the ability even for people you trust to log in as others.
It's not about just you. Users tend to share passwords across systems. Maybe some day (God forbid) you have a breach that has nothing to do with passwords, but in the course of that breach your authentication tables will be one of the attacker's targets. If you store passwords in plain-text, you have also just compromised user accounts at many other services, and your very bad day just got quite a lot worse.
If you think this kind of thing doesn't happen, go talk to the guys at reddit.