<p>We all hear a lot about scaling issues in Rails.</p>

<p>I was just curious what the actual costs in handling a HTTP request is in the Rails framework.  Meaning, what has to happen for each and every request which comes in?  Is there class parsing?  Configuration?  Database Connection establishment?</p>

That actually depends a lot on which web server you're using, and which configuration you're using, not to mention the application design itself. Configuration and design issues involved include:
Whether you're using fastcgi, old-school cgi, or some other request handling mechanism (affects whether you're going to have to rerun all of the app initialization code per request or not)
Whether you're using memcache (or an alternate caching strategy) or not (affects cost of database requests)
Whether you're using additional load balancing techniques or not
Which session persistence strategy you're using (if needed)
Whether you're using "development" mode or not, which causes code files to be reloaded whenever they're changed (as I recall; maybe it's just per-request) or not
Like most web app frameworks, there are solutions for connection pooling, caching, and process management. There are a whole bunch of ways to manage database access; the usual, default ones are not necessarily the highest performance, but it's not rocket science to adjust that strategy.
Someone who has dug into the internals more deeply can probably speak in more excruciating detail, but most apps use either FastCGI on Apache or an alternate more rails-friendly web server, which means that you only have app setup once per process.