<p>Like it is the case with jpeg images for example?
What's about mp3 or ogg vorbis audio files?</p>

It all depends on the definition of "loading" and "saving".
If the program in question actually decodes mp3/ogg upon loading and encodes in back when saving (this could be detected by noticeable time it takes to save a file), then yes - you could experience degradation in quality. Especially if the encoding uses different settings (lower bitrate, etc).
On the other hand, a loseless mp3 splitter, for example, could take mp3 and split it into several smaller mp3's without re-encoding them.
So you need to consult a manual of the software in question, there is no generic answer.