<p>First off, how do I know if my html file is running on localhost in Xampp?
Is there a tutorial on how to manage files/directories and get that all working under htdocs?
Is there a good tutorial on how to setup includes?</p>

<p>I want to use "includes" in Xampp with my html.
Can I use both html includes AND php includes?
Do I have to put shtml?
Can I use shtml, html, htm, and php includes?
Do they have to be in an includes directory that is a subdirectory right under htdocs?
Can I reference includes in some other subdirectory?
My site will have over 100 pages, and I am trying to do "experiments" with different versions until I am happy. So, I have subdirectories for the various drop down menus. Unfortunately, I don't seem to be able to get this working in xampp.
Having trouble getting my javascript menus from Vista Buttons to show up, now that I moved my main directory for my site to the htdocs directory.</p>

To answer your first question, can you access
http://localhost/
? If so, then xampp is running correctly and there is nothing more that you need to do.
EDIT:
If you want to ensure that things are working correctly, place foo.html under /xampp/htdocs and then access it in a browser at
http://localhost/foo.html
. If you can't get this page then xampp still isn't running correctly.
In answer to your second question, all Includes whether in the HTML or within PHP are server side. If you have xampp installed correctly, then you can begin working in PHP immediately and can use the include functions within PHP (please note that even if a page only contains HTML, if you're using PHP it must still have a .php extension).
If you would rather use SSI in your .html pages, then
this link
might provide some help on how to set up and debug SSI in xampp.
Additionally, if you are not already doing so I would recommend that you work in Firefox with Firebug installed. Firebug will handle most of your JavaScript debugging needs.
As a final note, if you've moved directories and the like, the first thing I would check would be the src attributes on your script tags.