<p>I have written a program in C to parse large XML files and then create files with insert statements.  Some other process would ingest the files into a MySQL database. 
This data will serve as a indexing service so that users can find documents easily.</p>

<p>I have chosen InnoDB for the ability of row-level locking.  The C program will be generating any where from 500 to 5 million insert statements on a given invocation.</p>

<p>What is the best way to get all this data into the database as quickly as possible?  The other thing to note is that the DB is on a separate server.  Is it worth moving the files over to that server to speed up inserts?</p>

<p>EDIT:  This table won't really be updated, but rows will be deleted.    </p>

I'd do at least these things according to
this link
:
Move the files there and connect over the unix socket
Generate, instead of the INSERTS, a
LOAD DATA INFILE
file
Disabling indexes during the loading