<p>A product that I am working on collects several thousand readings a day and stores them as 64k binary files on a NTFS partition (Windows XP).  After a year in production there is over 300000 files in a single directory and the number keeps growing.  This has made accessing the parent/ancestor directories from windows explorer very time consuming.  </p>

<p>I have tried turning off the indexing service but that made no difference.  I have also contemplated moving the file content into a database/zip files/tarballs but it is beneficial for us to access the files individually; basically, the files are still needed for research purposes and the researchers are not willing to deal with anything else.  </p>

<p>Is there a way to optimize NTFS or Windows so that it can work with all these small files?  </p>

If you can calculate names of files, you might be able to sort them into folders by date, so that each folder only have files for a particular date.  You might also want to create month and year hierarchies.
Also, could you move files older than say, a year, to a different (but still accessible) location?
Finally, and again, this requires you to be able to calculate names, you'll find that directly accessing a file is much faster than trying to open it via explorer.  For example, saying
notepad.exe "P:\ath\to\your\filen.ame"
from the command line should actually be pretty quick, assuming you know the path of the file you need without having to get a directory listing.