Audiobus: Use your music apps together.

What is Audiobus? — Audiobus is an award-winning music app for iPhone and iPad which lets you use your other music apps together. Chain effects on your favourite synth, run the output of apps or Audio Units into an app like GarageBand or Loopy, or select a different audio interface output for each app. Route MIDI between apps — drive a synth from a MIDI sequencer, or add an arpeggiator to your MIDI keyboard — or sync with your external MIDI gear. And control your entire setup from a MIDI controller.

Download on the App Store

Audiobus is the app that makes the rest of your setup better.

Capture notes of a chord and sort

dcollettdcollett
in General App Discussion

Hi, everyone. I'm new to Mozaic, and I hope some of you can help me.

Here's what I want to do: When a user plays a chord (2, 3, 4, 5 or more notes), how can I capture all the MIDI note numbers so that I can perform a function on each of the notes? Is there a way to put the notes into an array, or ???

Also, is there any way to sort the notes (by MIDI number) in Mozaic?

I'll very much appreciate any help you can give! Thanks!

Comments

  • uncledaveuncledave

    Hi. Yes there is. The tricky part is the first bit. Mozaic is event-based, so it runs once for each event, like Note On. So "play a chord" is not a fundamental concept. You'll need to detect when notes are played relatively close together to identify them as a chord.

    One way to get the notes in numerical order would be to create an array, clear positions 0 to 127 (there's a function), store a flag in the position of each note number, then scan the array for non-zero entries.

    You should take a look at the patchstorage Mozaic for relevant scripts already written. For example, MULTI-DIVISI by @_ki includes chord detection and a whole lot more.

  • McDMcD

    I took a run at making a MIDI recorder app and here's what I learned:

    I use an index that is incremented as each note comes in. This index then is used to store NOTE data in (but all MIDI types could also be saved):

    Channel = [0]
    Command_Type = [0] // This saves Note On vs Note off
    Note = [0] // MIDI Note Number
    Velocity = [0] // Volume of input - MIDI spec's allow Velocity = 0 to be treated as a Note Off
    TimeCode = [0] // Save the time of event and you can use this data to re-play with sub milli-second accuracy

    Using an index like 26, you get a matched set of data for that Note Event. Saving into Byte1, Byte2, Byte3, Byte3 keeps the data in a format close to the MIDI data. Then you can parse Byte1 for the combined Channel + Command information later.

    For Chord detection I find is useful to use the Modulo 12 function (Note % 12) to identify if it's a C, C#, D, D#, E, F, F#, G, G#, A, A#, B then you can detect a C Major despite the chord voicing or inversion at work.

    Since all notes are number you can transpose by adding or subtracting an internal. Inversions are like +/- 12 for any note in the chord.

    I was considering logging all the major chord types but the if statement is really, really long.

    To re-play notes with rhythm you must output all the notes at once but add a "delay" value on the SendMIDI Statements that you get from calculating the time of the current note - subtracting the first note's timestamp.

    You CAN have Note On and Off in the same array or use the Command data to store 2 Arrays for On/Off.

    Checking for groups of notes within time stamp ranges helps know they arrived as a chord and could be treated a "one logical event". The time stamp range will be in the 1-2 millisecond range. When Mozaic sends chord out they are sent as a quick list of individual Notes just as they arrived.

    Mozaic places a limit of 256 SendMIDI at a single time. But you can get multiple seconds of very complex music posted in 256 events. The trick is to time them in batches if you want to
    make a recorded that saves for minutes of input. You also have to detect the index getting above 1024 and deal with a 2nd set of arrays to keep capturing.

    Have fun... I know I have been having fun solving these event driven problems with solutions.

  • McDMcD

    Also, identify the note’s position in a scale is useful for sliding motifs or changing modes.

    Using arrays for scale types:

    Major = [0,2,4,5,7,9,11]
    Then you add the root note for a specific key.

    Transposing just shifts root numbers but changing scale or chord types does not impact every note equally.

  • wimwim

    As @uncledave mentioned, capturing chords can be difficult due to timing issues. Since not all the note-on events are guaranteed to arrive at the same time, you have to introduce some amount of delay to be sure they've all arrived. If you're doing operations on those notes, then this delay will translate into latency before that operation can complete.

    You also have to keep in mind that each note is actually two events, a note-on and a note-off. If you're altering note-on events, you have to do some tracking of the note-off's as well so that these are altered in ways that don't result in stuck notes (a note-on with no corresponding note-off).

    Further complications can arise if you have any overlapping notes between chords.

    Storing notes into an array isn't hard, but can get a little complicated if you don't know in advance how many chords and notes you need to store. I've found that scanning through large arrays can be time consuming and introduce glitches if you're not careful.

    Sorting ... there are no built-in sorting routines in Mozaic. You'll need to write your own.

    ;tldr - dealing with incoming chords in Mozaic is one of the more difficult tasks you could try to take on. Sorry to be so discouraging, but as someone who has spent many hours working the problem and discussing it with others, I though I should forewarn you that it's not a straightforward as I'm sure it seems it should be.

  • espiegel123espiegel123

    @dcollett said:
    Hi, everyone. I'm new to Mozaic, and I hope some of you can help me.

    Here's what I want to do: When a user plays a chord (2, 3, 4, 5 or more notes), how can I capture all the MIDI note numbers so that I can perform a function on each of the notes? Is there a way to put the notes into an array, or ???

    Also, is there any way to sort the notes (by MIDI number) in Mozaic?

    I'll very much appreciate any help you can give! Thanks!

    The situation is not as dire as it may sound from the responses that you have had.

    The situation can be simplified in a number of ways.

    Does this need to follow natural playing — I.e. does it need to automatically detect that a chord is being from a stream of notes that might include non-chords OR is it allowable to use something like a sustain pedal or other CC or a special note to start/stop the detection?

    Do the notes need sorting or do you just need to categorize them by note name (or really note number mod 12)?

  • McDMcD

    What do you want to do with the chords? It may be that doing it on a note by note basis sounds like you intend. Mozaic will get a note and you can do something as a response.
    Getting a chord is just getting and doing on multiple events.

    The idea you have to wait doesn't really fit the event model. You start doing immediately...
    even if the doing is just copy this note data to these arrays. I find counting to some group of notes just creates a lot of extra work. If you want to shift a chord, transpose it, turn it into an arp sequence or whatever you can probably start on getting the first note and if there's no more notes you just have Mozaic idling for more input to do something with.

    If you like we can provide comments on your script as you go forward but I think wrestling with Mozaic and getting something that works is more fun than taking apart and engine and putting it back together. You'll be inventing a new engine here.

  • dcollettdcollett

    Hi, everyone. Thank you all so much for your willingness to help. Because I'm new to Mozaic, my initial goal is quite simple: If I play an n-note chord, Mozaic "captures" all the notes (in an array or something???) so that I can modify each note, sort the notes into a different order (for inversions, perhaps), etc.

    I mention sorting them because, for example, the 3 notes in a triad may be pressed in a slightly different order, so I want to sort them in ascending order (in MIDI numbers). At this point, I don't need to distinguish between a C3, C4, C5 note. Just a C (although if I can get the MIDI #, that would be great).

    Thanks again for your willingness to help a novice 😺

  • espiegel123espiegel123

    @dcollett said:
    Hi, everyone. Thank you all so much for your willingness to help. Because I'm new to Mozaic, my initial goal is quite simple: If I play an n-note chord, Mozaic "captures" all the notes (in an array or something???) so that I can modify each note, sort the notes into a different order (for inversions, perhaps), etc.

    I mention sorting them because, for example, the 3 notes in a triad may be pressed in a slightly different order, so I want to sort them in ascending order (in MIDI numbers). At this point, I don't need to distinguish between a C3, C4, C5 note. Just a C (although if I can get the MIDI #, that would be great).

    Thanks again for your willingness to help a novice 😺

    Are you only going to be playing chords?

  • uncledaveuncledave

    Will the chords always be in root form, never inverted?

  • wimwim
    edited July 2022

    Some other questions that would help determine the scope of the effort:

    • Will the chords always have the same number of notes?
    • How precise will the timing of the notes be? Will they all hit very close to the same time, or might they be be staggered somewhat)
    • Will the chords ever overlap?
    • Will the inversions have to happen real-time as you're playing live?
    • Do you need to be able to store and recall these chords? For instance, play a certain number of chords, then recall and trigger them again with different inversions? If so, how many chords? Just one, or a series of them.
    • Would you be able to write your own sorting routines for the arrays?
    • Can how you want to interact with the chords be carried out within the constraints of the Mozaic user interface. Mozaic has 16 pads, 21 knobs, 9 faders, and an XY pad to work with. Labeling of controls and pads is limited to fixed strings and numbers.

    Each of those questions has a big effect on how feasible this would be.

    Part of the difficulty in answering your question about Mozaic is you've only described a chunk of what you're trying to do. The answer to whether you can capture chords, store them in an array, and sort that array, is a pretty simple "yes". But I assume you want to do something with the chords after that happens.

    If you just want to hold down a chord, then show the notes of the various inversions while it's held down, that's one thing. If you want to play free-form, detecting chords on the fly, while tapping pads to pick different inversions to alter your playing real-time, that's a very different challenge.

  • dcollettdcollett

    Hi, everyone. Thanks again for your help. Here are some answers to hopefully explain better what I'm trying to do:

    • Initially, this is for analyzing chords

    • The chords can be between 2 - 7 notes

    • They will be hit as precisely at the same time as most players can 😺

    • No chord overlaps (that is, the user plays a chord, lifts fingers, plays another chord)

    • Inversions don't need to happen as you're playing live

    • I'd like to store the notes of the chord in order to analyze the separate notes, play different inversions, change the notes, etc., but only for that single chord. Once the user plays the next chord, I don't need to save any notes of the previous chord (in other words, an array (?) that holds up to 7 notes, if that's possible

    • If Mozaic doesn't already have a way to sort items, then I could figure out a way to sort them (maybe into another array?)

    Again, initially, my goal is very basic (well, it seems basic to me). I'd like to be able to play a 2-7 note chord, figure out what notes are in the chord, then print the notes in the order played, figure out what chord the user played, arrange the notes in ascending order (close position), then play the chord in close position or in any inversion I want (by adding/subtracting 12).

    The reason I mention sorting is so that the notes would be from lowest to highest rather than exactly as they played them. For example, if the user pressed the E a fraction before pressing the C and G, the order would be E,C,G.

    If I could do this basic first step, it would get me on my way. Then I could try to figure out something more interesting to do with the chord.

    Thanks again for your help!

    David

  • espiegel123espiegel123

    @dcollett : that should be straightforward. There isn’t built in sorting but sorting like this is pretty straightforward BUT since you just want the notes you probably don’t need sorting. Here is rough pseudo code that I think does what you want. I think if you refine the pseudo code to what you want, the Mozaic code will be straight to write.

    Set chordInProgress to false
    Set chordTimeout to 500 ms

    Listen to midi

    When a note comes in,

    • set chordInProgress to true
    • Clear old note arrays
    • startCollectingNotes
    • When timeout time has passed with no notes, set chordInProgress to false
    • When a note comes in, set lastNoteTime to systemTime

    To start collecting notes:

    • start timing loop , more on this later
    • Add note number mod 12 to next slot in notetypeArray
    • add noteNumber to next slot of notesPlayed

    Timing loop, set time interval to something like 60 milliseconds. In the timing loop, check systemTime minus lastNoteTime. If that is greater than chordTimeout then process your chord

    The notetypeArray will have just 12 slots: 0..11 where 0 is C, 1 is C# etc

    There will be a 1 in any slot if that note was in the chord

    NotesPlayed will have the notes in the order received.

    A simple sort can be done be having a notesArray where you set a slot to 1 if the note was received. Then iterate through the array and pull out any notes where the value is 1. Computationally inefficient but trivial to code and for this efficiency probably doesn’t matter.

  • dcollettdcollett

    espiegel123,

    Thanks SO much for taking the time to help. Also, thanks to everyone here for your input. I appreciate it.

    I'll try to implement your algorithm to see if I can get it working. (If not, I'll be back 😺 )

    At any rate, I'll let you know if it works.

    David

  • wimwim
    edited July 2022

    That does all sound doable. To make timing easier, you can just keep track of the notes played and consider the array filled when all notes have been released.

    Here's a real quick example of just capturing the chord into an array, then displaying the notes on Mozaic's pads. There's obviously a lot more that you'd need to do, but this should give a little bit of an idea of one simple capture method.

    @OnLoad
  FillArray chord, -1
  notecount = 0
  currentnote = 0
  ShowLayout 2
@End

@OnMidiNote
  if MIDICommand = 0x90 //note-on
    chord[notecount] = MIDINote
    Inc notecount
  elseif MIDICommand = 0x80 //note-off
    Dec notecount
  endif
  
  if notecount = 0
    Call @DisplayChord
  endif
  
  @End

@DisplayChord
  currentnote = 0
  while chord[currentNote] <> -1 and currentNote < 16
    LabelPad currentnote, (NoteName chord[currentnote],YES)
    Inc currentNote
  endwhile

  // Wipe the array for the next chord
  // Obviously you'd want to do more than this with the chord in practice
  FillArray chord, -1
@End
  • dcollettdcollett

    wim, thanks so much! This example code will help get me running, I hope. Then I can start to experiment with the notes. I thought this would be a good way to learn how to manipulate the notes of a chord.

  • wimwim

    @dcollett said:
    wim, thanks so much! This example code will help get me running, I hope. Then I can start to experiment with the notes. I thought this would be a good way to learn how to manipulate the notes of a chord.

    You may want to check out some of the scripts on patchstorage.com for examples of how to sort and manipulate chords. I did a bit of that in The Chordulator. You might find some ideas there. Another that looks closer to what you're doing is Chordpeggio, though I haven't looked at any more than just the description for that one. I know there are other, more extensive scripts, dealing with chords on Patchstorage.

    Good luck!

  • McDMcD

    @wim said:
    Here's a real quick example of just capturing the chord into an array, then displaying the notes on Mozaic's pads.

    Nice implementation of the specs. I haven’t seen that NoteName function used before this.

  • dcollettdcollett

    Thanks again Wim, espiegel123, McD, and uncledave! I'll keep you posted if I'm successful (or not). Best to you all.

  • McDMcD

    @dcollett said:
    Thanks again Wim, espiegel123, McD, and uncledave! I'll keep you posted if I'm successful (or not). Best to you all.

    Just remember each note generates 2 events (On and Off) at a minimum. But Moziac deals with them 1 event at a time without ever stopping until you stop the app. You defined a chord as all the notes that are pressed until all the fingers leave the keyboard... or
    the On's = Off's. It could be 2 notes up to 16 max (for the display limit)... it could even be 27
    but it would not display past 16.

Sign In or Register to comment.