<p>What is the maximum number of threads you can create in a C# application?  And what happens when you reach this limit?  Is an exception of some kind thrown?</p>

There is no inherent limit. The maximum number of threads is determined by the amount of physical resources available. See this
article by Raymond Chen
for specifics.
If you need to ask what the maximum number of threads is, you are probably doing something wrong.
[
Update
: Just out of interest: .NET Thread Pool default numbers of threads:
1023 in Framework 4.0 (32-bit environment)
32767 in Framework 4.0 (64-bit environment)
250 per core in Framework 3.5
25 per core in Framework 2.0
(These numbers may vary depending upon the hardware and OS)]