<p>In some my project I notice that during executing unit tests under VSTS2008 its VSTestHost's memory consuming grows. As I have very many tests in my solution it leads to OutOfMemroyException eventually.
That looks very strange for me as I was sure that MSTest creates a new AppDomain for each unit test. Otherwise how would it reset static fields?
But if AppDomain is being created for each test than memory shouldn't leak. But it does.</p>

<p>So the question is: Should VS create AppDomain for each test class or not? If yes than how can I check that it does it. 
I tried tracing through ProcessExpolorer and Performance snap-in. A value of "Total appdomain unloaded" is always 0 during test run.</p>

I don't think the unit test engine creates a new AppDomain for each test.  Since creating an AppDomain is a relatively expensive operation, doing so for each test would slow down execution of unit tests considerably!
Visual Studio 2008 uses a seperate executable called vstesthost.exe to run unit tests.  VS communicates with vstesthost.exe (how it does this I don't know) to tell it what tests to run.  vstesthost.exe returns the execution results to VS which displays those results.
If you are getting OutOfMemoryExceptions when running your unit tests I would say that's a strong indicator that your code under test is actually not cleaning things up.  Are you sure that you aren't retaining handles to unmanaged objects/memory?  I would recommend running your unit tests under a Performance Analysis (you can do that by finding the unit test under the "Test View", right-clicking on it, and selecting "Create Performance Session").  This might shed some light at least on your object allocations.