<p>Sometimes, when we're doing small changes to our web apps, e.g. bug fixes, we don't build a whole new WAR-file each time, but merely replace just the affected class files in the exploded web app directory under <code>WEB-INF/classes</code> and restart the app.</p>

<p>Is that okay?</p>

I'd say that probably isn't a best practice, because of versioning: how do you know which version of the application you've got deployed? If you deploy a .war file your build process can take care of updating a build number (from source control, or separately, whatever - as long as each build has a different number it's OK).
If you're using continuous integration (and it's definitely a good idea to be) then your build process should be kicking out an 'artifact' (war file) each time you make changes in source code. also perhaps tagging the code in version control with the build number.
So, when you deploy your web app, you know exactly which version is running and which source code makes up that version.
Making small incremental changes by updating individual .class files I'd say is probably not a good idea for anything other than local developer testing.