craigix said:
I think I'd rather the sound played slightly slower (ie it slowed down with the game) than it stuttering.
It probably isn't possible for N64 emulation, at least not with how it's currently being done, except to continually resample the output at a varying rate. However, that would result in completely pitch shifted output, not "slower" audio like you're imagining, and the resampling would probably sound uneven like playing a record that's gone all wobbly.
WARNING: confusing explanation ahead
When you see the slow down happening in other emulators it's because they're emulating the audio in
real time and everything else in
emulated time. This requires completely decoupling the emulation of the audio from the emulation of everything else. In other words, the two things are now asynchronous instead of synchronous, and are connected by a FIFO.
So let's say there's some period of time, the "synchronization period." It could be one frame (50Hz, 60Hz, whatever), or it could be when the audio's buffer runs out.
During this period the emulator will emulate X machine cycles where X is a completely fixed number. Like, if the period is 10ms and the machine is 100MHz it might be emulating 1 million cycles. Whatever the number is, however accurate or inaccurate, it'll be constant regardless of how fast or slow the emulation is; if it took you 5ms to emulate that then the emulator will then throttle and wait for 10ms to pass so it looks like it's running at the right speed. But if it took 20ms then the emulator will just feel half as fast as it should be and there's no way around this.
The audio, on the other hand, will emulate a number of of output samples based on how much real time has passed since the last time audio was emulated. This is usually based on the audio output rate. So if you're outputting at 44.1KHz you're going to be generating 441 samples every time 10ms has passed on the machine you're emulating. This will happen regardless of whether or not you were able to emulate all your machine cycles during this time.
(the actual synchronization of the machine and audio emulation might not line up, ie it might synchronize machine emulation every frame and audio emulation every buffer period, but the aggregate synchronization over longer periods of time will match what I'm saying)
So you end up having too few emulated cycles for the audio cycles.
How this affects you depends on what the audio chip's role in generating music is, vs what the CPU's role is.
For platforms like NES and SNES the CPU's role is usually to tell the audio chip to start playing an instrument at some pitch, then stop playing it. The audio chip then takes care of playing it with the right waveform, pitch, and in this case even effects like looping, envelope, sweep, reverb, etc.
What this means is that if you aren't emulating the CPU fast enough you'll still end up with notes that play at the correct pitch and have the correct timbre, but they'll start and stop at slowed down intervals. So the overall tempo will sound slower.
On platforms like N64 and GBA the CPU's role is much more low level; it generates audio sample by sample by writing to a buffer. In N64 this might be abstracted by HLE, but from what I understand not in a way that really allows proper decoupling, since the games still need to be able to stream audio. Zilmar has said that a synchronous approach is necessary, and given that even emulators like N64 had synchronous audio I tend to believe it.
Usually the asynchronous audio emulators just capture the audio state at the beginning of the emulation interval, so it's not even really a FIFO between machine emulation and audio emulation. For platforms like NES/SNES this usually (not always) sounds OK, because the fidelity of when instruments key on and off isn't that important. But if you're streaming audio it's very important, and this will completely wreck the output. You can actually buffer every audio state change instead, then linearly spread them throughout the audio emulation period. You can even cycle timestamp the audio state changes then scale their emulation proportionately to when they should happen during the interval.
The end result with that is that you'll have pitch shifted audio, although it'll sound really bad if you don't filter it. The thing is, buffering every sample write like this (especially timestamping them) usually has much more overhead than emulating synchronously directly out of audio buffers. You also have to pay additional overhead for interpolating the events and filtering the result. Furthermore, if the audio generation occurs by writing to plain old system memory (that is then, say, DMA'd out to audio buffers internally) you won't be able to capture audio events very easily because you won't know when the game is writing to an audio buffer, at least not without adding a lot of additional overhead to the emulation.
You can also resample the audio AFTER it's emulated to match the amount of time that it's passed, basically stretching out the samples you emulate. That way you end up with it filling the entire time instead of filling a bit then stopping, hence the stuttering.
The problem with either of these interpolation approaches is that if things aren't fullspeed you probably won't have a constant framerate (unless you force one, but since that'll tend to make things much slower you don't really want this), so you won't have a constant resample rate, hence wobbly record effect.
Since the goal of an emulator is usually fullspeed emulation considerations that improve quality of sub-realtime emulation aren't usually considered, especially if they end up making the emulation even slower. Just not something people often want to invest time in, but for emulators where not tolerating slow speeds is practically a mission statement you sometimes see it (cough, desmume has this option)
The current dev for N900 has gotten married and moved to Spain.. so we've run out of developers
