Pandora Developing a Guitar Simulator Application for the Pandora


Matthias_H

Member
Joined
May 21, 2009
Messages
194
Dear all,


Quite a while ago I suggested the creation of a guitar simulation for the Pandora. The idea is to have the photo of guitar strings as GUI, and the user could use the stylus (or their fingers) to pluck or strum the strings. The actual pitch per string (fingering) could be controlled through key combinations, preprogrammed song structures, etc. Once the basic functionality is up and running, we can think about dedicated modes for different musical styles (Folk, Jazz, Campfire, Flamenco, Latin strummed, Latin plucked, ...), a network competition mode, guitar lessons, effects, more guitar models, etc.





I am not a programmer (though most of the code I write somehow gets its job done) but there should be plenty of these around here. So if you're interested, feel free to join the discussion. On the other hand, I know quite a bit about physics and music and guitars. Given the skeleton of an audio engine / UI, I could contribute to the functionality to make everything behave like it should. Also, I have the equipment to record a multisample, (and I could even contribute a proper animation of vibrating guitar strings :) ).


Here's what I think are the two main TECHNICAL challenges that the engine would have to meet.


(1) Polyphonic low-latency playback of audio samples (at least 12 stereo voices). For decent quality, a pool of about 150 individual samples will be recorded (each a few seconds long), totalling at about 80MB of 44.1kHz/16bit/stereo sample data which, as I figure, would all fit into the RAM at once. The engine must be able to access all the data randomly in an instant, and play back samples at variable volume and pitch (50% - 150% of original speed). It would also be good to have an antialiasing filter for transposed playback.


Edit: Solved (see below :) )

Code:
// fire a sample

void playSample(

                int voice,                                 // which playback channel?

                short *data, int length,                   // address and size of the audio sample

                float volume, float pitch, int offset = 0  // playback parameters

               ); 


// change the pitch during playback

void changePitch(

                 int voice, 

                 float pitch

               );


// ramp down the volume of a sample

void dampenSample(

                  int voice,

                  float decay

                 );




(2) GUI with touchpad handling. Essentially, the stylus position needs to be tracked at high rate. Whenever a string is crossed, the following function is fired.




Code:
void pluckString(

                 int number,               // which string was plucked?

                 float stylusVelocity,     // something along the lines of "multiples of screen height per second"

                 float stringPosition      // bridge: 0.f; end of fretboard: 1.f

                ) {


  int velocity = stylusVelocity * SCALE_VELOCITY * NUMSAMPLES_VELOCITY;

  int pitch = getStringPitch(number); // manage the fingering somehow

  int note =  getStringNote(number);  // ...

  int stringPos = stringPosition * NUMSAMPLES_STRINGPOS;


  float volume = getVolume(stylusVelocity, stringPosition); // some scaling function


  int sampleIndex = getSampleIndex(number, velocity, note, stringPos);


  playSample(number, samples[sampleIndex], sampleLengths[sampleIndex], volume, pitch, 0);


}


Quite a few people seem to have gotten their Pandoras already (I haven't), so if there is any programmer out there who could hack together a basic engine, that'd be awesome. The very moment I receive my Pandora, I'm in :)


Cheers,


Matthias
 
Last edited by a moderator:

Matthias_H

Member
Joined
May 21, 2009
Messages
194
I just found out about the MikMod sound library currently maintained by Raphael Assénat (http://mikmod.raphnet.net). It appears to offer everything we need, with the voice management/mixing, transposed playback, pitch manipulation, WAV file loading, etc. already included.



Code:
Sample_Load

SAMPLE* Sample_Load(CHAR* filename)

Description
    This function loads a sample.
Parameters
filename
    The sample filename.
Result
    A pointer to a SAMPLE structure, or NULL if an error has occurred.


Voice_Play

void Voice_Play(SBYTE voice, SAMPLE* sample, ULONG start)

Description
    Start a new sample in the specified voice.
Parameters
voice
    The number of the voice to be processed (starting from zero).
sample
    The sample to play.
start
    The starting position (in samples).
Notes
    The sample will be played at the volume, panning and frequency settings of the voice, regardless or the sample characteristics.
    The sample played this way gets the same "critical" status as the sample which was previously played on this voice.
See also
    Sample_Play, Voice_SetFrequency, Voice_SetPanning, Voice_SetVolume. 
[/html]

etc.
 

whynodd

Member
Joined
Sep 20, 2008
Messages
262
Hey Matthias_H,


A Guitar Simulation is a very good idea. It should be possible to create realistic sounds with such a concept. Of course, the sound depends on the number of samples. Do you want to sample each String at each fret position? Hehe, and the fret sounds.


Concerning input: The touchscreen should work good for strumming, but picking 3 strings at once is not possible (no multitouch).


Pandoras keyboard to input chords similar to an accordion? For this, you have to work out a good system that is not too complicated and allows many chord combinations + modifications with max 2 fingers. Or you prepare different sets of chords (depending on song/style/general pitch).


And there are nubs...


Edit: Quick Idea


Numbers: Select chord set


Character Buttons: Select chord


ABXY, LEFT/RIGHT/UP/DOWN: Modifications, minor, mayor..


hmm, doubletap: Switch between sus and non-sus. Pull-off/Hammer-on is fired, if button is kept pressed and released. Short taps: no Pull-off/Hammer-on.


hmmmmmmmm...


Select: Switch between "chord mode" and "fretboard mode". In fretboard mode, Pandoras keyboard represents the fretboard, then you can tap around or pick a few strings.
 
Last edited by a moderator:

lulzfish

Pandora Defense Squad
Joined
Jan 14, 2009
Messages
3,502
Website
troyanonymous.homelinux.com
The touchscreen should work good for strumming, but picking 3 strings at once is not possible (no multitouch).
I think it would be done with a swiping motion, sort of like on a regular guitar. I didn't play guitar much, but I don't think you have to pick all 3 strings of a chord at once.


The issue I'm wondering about is how to do all the different frets for 6 strings at once, though I suppose there's only a hand's width of frets at a time, but sometimes those move around during a song.
 

Matthias_H

Member
Joined
May 21, 2009
Messages
194
A Guitar Simulation is a very good idea. It should be possible to create realistic sounds with such a concept. Of course, the sound depends on the number of samples. Do you want to sample each String at each fret position? Hehe, and the fret sounds.
Minor thirds might work: open string, 2nd fret, 5th fret, 8th, 11th. Transposing a sample up/down by a semitone should still sound acceptable.

Concerning input: The touchscreen should work good for strumming, but picking 3 strings at once is not possible (no multitouch).
True, as far as finger picking is concerned (which would also need a different set of samples).


Plec picking and chord strumming should work pretty much the same, though.

Pandoras keyboard to input chords similar to an accordion? For this, you have to work out a good system that is not too complicated and allows many chord combinations + modifications with max 2 fingers. Or you prepare different sets of chords (depending on song/style/general pitch).
And there are nubs...
I guess a good approach would be to offer a set of basic chords (arranged around a harmonic center), then extensions/alterations in different key layers that can be switched using the gaming controls. The software should also offer a mode where fingerings can be freely programmed and mapped to controls.


There are lots of other ideas that might require some thought:

  • Graceful handling of chord changes/fretboard slides while strings are vibrating, maybe by providing a second set of samples. If possible, I'd tend to avoid multi-note samples (pull-off, hammer-on) so as not to blow up the memory requirement and sampling efforts.
  • Different inversions (fretboard positions) of the same chord could be mapped either to gaming controls, or the strumming position on the screen.
  • ...



hmm, doubletap: Switch between sus and non-sus.
No way. I refuse to offer this kind of functionality ;-)
 

Matthias_H

Member
Joined
May 21, 2009
Messages
194
So, I just hacked a really crude attempt based on MikMod (OSS driver) and Qt4 (v4.4.3) and using a bunch of classical guitar samples from the OLPC project. No chords yet or anything, just experimenting with audio output and mouse events. Please find (and destroy/improve) it here. (I wrote the code on a 64-bit Debian machine. If anyone could try to compile it on a Pandora, that'd be awesome.)


Issues (among many others):

  • The latency is really bad. Decreasing the OSS buffer size didn't really help too much.
  • The mouse polling rate is rather slow. Maybe upsampling/interpolation of the mouse movement could help.



But at least it's a start. Going to bed now.
 
Last edited by a moderator:
Top