Audiobus: Use your music apps together.

What is Audiobus?Audiobus is an award-winning music app for iPhone and iPad which lets you use your other music apps together. Chain effects on your favourite synth, run the output of apps or Audio Units into an app like GarageBand or Loopy, or select a different audio interface output for each app. Route MIDI between apps — drive a synth from a MIDI sequencer, or add an arpeggiator to your MIDI keyboard — or sync with your external MIDI gear. And control your entire setup from a MIDI controller.

Download on the App Store

Audiobus is the app that makes the rest of your setup better.

New Midi app MusiKraken

12467

Comments

  • McDMcD
    edited May 2021

    I got a gift card for my birthday and this has been on my list for a while... I'm starting with the "Open Mouth" insert foot pattern. I'm sending notes right into AUM just by opening and closing my mouth open in front of the camera... good times!

    IMG_2C4345289FB2-1.jpeg

    One path for Notes and the other for CC events... not sure the CC's are doing much yet. I tried MIDI learn on the SWAM side and it mapped vibrato to
    "give me some tongue" but I'm not feelin' it yet.

    https://forum.audiob.us/uploads/editor/xv/9wdod7zosbuz.jpeg

    NOTE TO SELF: point the camera at the cats as a test for animal musical intelligence.

    Thanks @Snarp for the interesting take on a MIDI controller app.

    This is my first experiment: Musickraken "Mouth Open" controller into the $20 SWAM Cello. I can't play the cello but I can lick one with my mind:

    I'm getting an Ornette Colman vibe... pure oral/aural interplay.

    Now people will beg me for a video... I make crap music... not crap videos.
    NOTE TO SELF: Google for ASMR "tongue" pyrotechnix freaks... could be a
    cash generating opportunity making crap videos.

    For $10 this will be more fun than a barrel of rabid monkeys and won't require permits or vet fees.

  • @McD said:
    For $10 this will be more fun than a barrel of rabid monkeys and won't require permits or vet fees.

    :D sounds interesting! Now add a strange sounding title to this and post it in a contemporary classical music forum, maybe with the comment "I made this with my mouth". Instant success?

    You do not need two Mouth Tracking modules, only one module with two connections would be more efficient.

    I mostly use CC 1 (Modulation) and CC 11 (Expression) for SWAM instruments, as they are premapped to useful parameters, but you can of course map it to whatever you want. If you host the instrument directly in MusiKraken, you can see the parameter changes directly while the AudioUnit view is visible, but currently you cannot interact with the MusiKraken-GUI simultaneously (I will probably change that soon).

  • @Snarp said:
    You do not need two Mouth Tracking modules, only one module with two connections would be more efficient.

    We have just met... I never optimize for efficiency. I optimize for mayhem. I created two tacks for notes and cc's. Can I combine both on 1 track? Because mayhem could ensue.

    I love the elegance of your design. I could figure out how to make something without the need for a manual or attention to prompts. That's a good design... it encourages one to experiment and learn by hearing results.

    I haven't looked yet... does it AUv3 or IAA? I tried split screen which is what I do with Piano Motifs into AUM. Didn't seem to support that approach to a more efficient User Interface.

  • @McD said:
    We have just met... I never optimize for efficiency. I optimize for mayhem. I created two tacks for notes and cc's. Can I combine both on 1 track? Because mayhem could ensue.

    I haven't looked yet... does it AUv3 or IAA? I tried split screen which is what I do with Piano Motifs into AUM. Didn't seem to support that approach to a more efficient User Interface.

    Try controlling multiple instruments simultaneously with the mouth, the hands and the body. Perfect opportunities for mayhem!
    MusiKraken so far only supports AUv3, no IAA. It would be easy to add IAA as well, but theoretically this is deprecated by Apple, that is why I didn't want to spend any time on it if they could remove it with any update. But it's been deprecated now for a while already, maybe I should add it anyways...

    Split screen so far is only for the keyboard, the chords pad and the video image. I will probably change the AUv3 to a floating window in a future version to support all possible strange AUv3 GUIs...

  • @Snarp said:
    MusiKraken so far only supports AUv3

    Good answer. IAA is so 2010's.

    Split screen so far is only for the keyboard, the chords pad and the video image.

    Good to know.

    I will probably change the AUv3 to a floating window in a future version to support all possible strange AUv3 GUIs...

    I'm intrigued. You seem committed to selling more copies and that's great. Let's make some demos and create a community of live artists in the spirit of Laurie Anderson and Andy Kaufman.

    Who's with me! (expect total silence). Not your fault.

  • @Toastedghost said:

    @Snarp said:

    @Toastedghost said:
    Had a fiddle with Musikraken’s new body tracker. Took Snarp’s advice abd did not go crazy. The app will still work even when seated.

    :) Nice!

    The body detection probably works much better if you move a bit further away from the camera. Which might be difficult with the headphones? The body pose tracker works best if at least half of your body is constantly visible on the camera frame.

    For this distance, I would use the hand tracker instead.

    I figured so just experimenting. My wife is school nurse and works from home. Domestic violence as risen 25% during the pandemic so she is always online. Thus I must wear headphones and that tethers me to the ipad. Would love a pair of bluetoothed headphones but pennies are tight right now, partly due to the movesense sensor I bought to use with holon.ist. My latest crazy idea is would this work with puppets and maybe with TV? I will continue to experiment. Cheers

    These earbuds are £20 right now, with the £10 discount on Amazon. Ah well, I hear you say, £20 is still too much. How about £5? Once you buy these they have an insert that gives a £15 Amazon voucher for a review. Not a glowing one, just your honest opinion. You would have to be an Amazon customer who needs other stuff from them in order to get your £15 back in a useful way. Or you could, perhaps, sell the code on. Apologies if you're not from the UK.

  • @ashh said:

    @Toastedghost said:

    @Snarp said:

    @Toastedghost said:
    Had a fiddle with Musikraken’s new body tracker. Took Snarp’s advice abd did not go crazy. The app will still work even when seated.

    :) Nice!

    The body detection probably works much better if you move a bit further away from the camera. Which might be difficult with the headphones? The body pose tracker works best if at least half of your body is constantly visible on the camera frame.

    For this distance, I would use the hand tracker instead.

    I figured so just experimenting. My wife is school nurse and works from home. Domestic violence as risen 25% during the pandemic so she is always online. Thus I must wear headphones and that tethers me to the ipad. Would love a pair of bluetoothed headphones but pennies are tight right now, partly due to the movesense sensor I bought to use with holon.ist. My latest crazy idea is would this work with puppets and maybe with TV? I will continue to experiment. Cheers

    These earbuds are £20 right now, with the £10 discount on Amazon. Ah well, I hear you say, £20 is still too much. How about £5? Once you buy these they have an insert that gives a £15 Amazon voucher for a review. Not a glowing one, just your honest opinion. You would have to be an Amazon customer who needs other stuff from them in order to get your £15 back in a useful way. Or you could, perhaps, sell the code on. Apologies if you're not from the UK.

    Poping up @ £29 on my Amazon. Cheers all the same

  • For some reason Simple Synth the test midi synth has stopped working. Not that I need it and all other connections work fine.

  • @Toastedghost said:
    For some reason Simple Synth the test midi synth has stopped working. Not that I need it and all other connections work fine.

    Ups, thanks for the info! I did a small change as MusiKraken seems to have cut off the sounds of other apps in the last version, seems like this has killed Simple Synth (and I haven't tested it, because I also never use it. I should add on the better synth more...). I will fix it as soon as possible...

  • edited May 2021

    @Toastedghost said:

    @ashh said:

    @Toastedghost said:

    @Snarp said:

    @Toastedghost said:
    Had a fiddle with Musikraken’s new body tracker. Took Snarp’s advice abd did not go crazy. The app will still work even when seated.

    :) Nice!

    The body detection probably works much better if you move a bit further away from the camera. Which might be difficult with the headphones? The body pose tracker works best if at least half of your body is constantly visible on the camera frame.

    For this distance, I would use the hand tracker instead.

    I figured so just experimenting. My wife is school nurse and works from home. Domestic violence as risen 25% during the pandemic so she is always online. Thus I must wear headphones and that tethers me to the ipad. Would love a pair of bluetoothed headphones but pennies are tight right now, partly due to the movesense sensor I bought to use with holon.ist. My latest crazy idea is would this work with puppets and maybe with TV? I will continue to experiment. Cheers

    These earbuds are £20 right now, with the £10 discount on Amazon. Ah well, I hear you say, £20 is still too much. How about £5? Once you buy these they have an insert that gives a £15 Amazon voucher for a review. Not a glowing one, just your honest opinion. You would have to be an Amazon customer who needs other stuff from them in order to get your £15 back in a useful way. Or you could, perhaps, sell the code on. Apologies if you're not from the UK.

    Poping up @ £29 on my Amazon. Cheers all the same

    Yep. The £10 discount on Amazon (little tick box on the actual item's page, just under the image of the item) makes them £20 and then the review rebate insert takes them down to £5.

  • @ashh said:

    @Toastedghost said:

    @ashh said:

    @Toastedghost said:

    @Snarp said:

    @Toastedghost said:
    Had a fiddle with Musikraken’s new body tracker. Took Snarp’s advice abd did not go crazy. The app will still work even when seated.

    :) Nice!

    The body detection probably works much better if you move a bit further away from the camera. Which might be difficult with the headphones? The body pose tracker works best if at least half of your body is constantly visible on the camera frame.

    For this distance, I would use the hand tracker instead.

    I figured so just experimenting. My wife is school nurse and works from home. Domestic violence as risen 25% during the pandemic so she is always online. Thus I must wear headphones and that tethers me to the ipad. Would love a pair of bluetoothed headphones but pennies are tight right now, partly due to the movesense sensor I bought to use with holon.ist. My latest crazy idea is would this work with puppets and maybe with TV? I will continue to experiment. Cheers

    These earbuds are £20 right now, with the £10 discount on Amazon. Ah well, I hear you say, £20 is still too much. How about £5? Once you buy these they have an insert that gives a £15 Amazon voucher for a review. Not a glowing one, just your honest opinion. You would have to be an Amazon customer who needs other stuff from them in order to get your £15 back in a useful way. Or you could, perhaps, sell the code on. Apologies if you're not from the UK.

    Poping up @ £29 on my Amazon. Cheers all the same

    Yep. The £10 discount on Amazon (little tick box on the actual item's page, just under the image of the item) makes them £20 and then the review rebate insert takes them down to £5.

    Thanks

  • My latest foolings With Musikraken this time using Spacecraft and the hand tracker

  • @Snarp, thanks for this... it sort of replaces those $200-300 expression rings, no? Or do I not understand?

    Also, can you record this stuff as midi tracks in DAWs like Cubasis? I’m not sure I can grapple with this... unless the true depth camera will pick up grappling... but I can bug @McD about it all I want.
    Anyway, this is amazing stuff to a septuagenarian!

  • “Septuagenarian”

    We Don’t call you names.

  • @LinearLineman said:
    @Snarp, thanks for this... it sort of replaces those $200-300 expression rings, no? Or do I not understand?

    Also, can you record this stuff as midi tracks in DAWs like Cubasis? I’m not sure I can grapple with this... unless the true depth camera will pick up grappling... but I can bug @McD about it all I want.
    Anyway, this is amazing stuff to a septuagenarian!

    Thank you for your comments. Musikraken can convert hands, face or body into either Midi or CC. All of which can be recorded. The only downside is that Musikraken works on its own screen, the controls can be mapped to basically any mappable control with any instrument that has the capability to be tracked. With Musikraken mointor screen AUM instruments can be seen and worked with (in most cases, Spacecraft does not like to open in’kraken). The problem is you can not view AUM and ‘Kraken at the same time.
    I do have an alternative tracker/sensor app called Holon.ist this is equiped with so many sensors I loose count. It will track eg the sun, moon, air pressure, gravity, location etc. It is insane what it will track. I bought a Movesense sensor, the size of a chocolate gold coin and weighs about the same, this works the same as thoughs expensive rings. Cost me approx £50. This frees me from the ipad and works through bluetooth. Because it works with so many sensors, including your heart rate, it is more complex to setup than ‘kraken. It works best in AUM but nesting inside mirack. Mirack recognises the midi signals better than other apps. Inside mirack the vast amounts of incoming data can then be tamed to something more managable. Synths in mirack can be rewired to accept holon.ist signals and play as though they were attached to a keyboard. Here is a demo of a dancer using holon.ist with the Movesense sensor.
    I am disabled myself and in my 60’s and work with a disabled group. Some members can only neogiate life through an electronic wheelchair. How wonderful it would be if they could play music without a keyboard, by hand gestures or just opening and closing their mouths.

  • edited May 2021

    A little more musical musing Musikaken’s hand gestures. No fancy filters just my ugly mug. Got to work out how I could just show hands without face. Hands as maybe puppets?

  • @Toastedghost, is it possible for MusicKraken to monitor any input while the app is not open... like the camera following "Mouth Open" status while AUM is dominant. Failing that could MusicKraken support the split-screen that Piano Motifs supports so I can open it and
    have it on the right edge of the screen.

    Re-packaging MusicKraken as an AUv3 would be useful but I'm not sure if an AUv3 can access the camera.

    Aha! It runs on my iPhone. Let me try that as the sensor and the iPad as the intended target.
    Bluetooth MIDI?

  • @McD said:
    @Toastedghost, is it possible for MusicKraken to monitor any input while the app is not open... like the camera following "Mouth Open" status while AUM is dominant. Failing that could MusicKraken support the split-screen that Piano Motifs supports so I can open it and
    have it on the right edge of the screen.

    Re-packaging MusicKraken as an AUv3 would be useful but I'm not sure if an AUv3 can access the camera.

    Aha! It runs on my iPhone. Let me try that as the sensor and the iPad as the intended target.
    Bluetooth MIDI?

    @McD MusiKraken currently doesn't work while it is in the background and cannot be loaded into an AUv3 host. I will probably add this feature soon, but you will not be able to use any of the camera features, or sensors (except maybe the microphone, but haven't tested this yet), because I don't think the app is allowed to access these (at least not while it is in the background. I haven't tried hosting it into AUM yet, but I guess the restrictions will be similar).
    But you will still be able to use the keyboard, the chords pad, the touchpad and all the effects, so I think now it would make sense to at least try out how well it will work...

    And yes, you can combine the iPhone and an iPad. Or even combine it with an Android device. You can connect either via Bluetooth (create a MIDI Output module, connect via Bluetooth, and then select the Bluetooth device from the list) or via Wifi (CoreMIDI Network module. Push the plus sign to select and connect with another device).

  • McDMcD
    edited May 2021

    @Snarp said:
    @McD MusiKraken currently doesn't work while it is in the background

    I expected as much but wanted to ask.

    But you will still be able to use the keyboard, the chords pad, the touchpad and all the effects, so I think now it would make sense to at least try out how well it will work...

    I have a lot of apps that provide that... the trick is a hands free sensor for CC's.

    And yes, you can combine the iPhone and an iPad. Or even combine it with an Android device. You can connect either via Bluetooth (create a MIDI Output module, connect via Bluetooth, and then select the Bluetooth device from the list) or via Wifi (CoreMIDI Network module. Push the plus sign to select and connect with another device).

    That's the ticket... I will try linking "Mouth Open"on the iPhone to SWAM Cello's vibrato depth on my iPad.

    Aha! It works on IOS 11. I have an iPhone 5S I might use just for this purpose
    and a really old iPad.

  • @Toastedghost said:
    A little more musical musing Musikaken’s hand gestures. No fancy filters just my ugly mug. Got to work out how I could just show hands without face. Hands as maybe puppets?

    I thought about adding a 3D model of a Kraken that is overlayed over the body instead of these red dots... but then dismissed it... Maybe one day, when I'm bored...

  • @McD said:
    Aha! It works on IOS 11. I have an iPhone 5S I might use just for this purpose
    and a really old iPad.

    Mouth detection will probably be very slow on your iPhone 5S. And hand tracking needs iOS 14+. So this probably won't work...

  • edited May 2021

    @McD said:
    @Toastedghost, is it possible for MusicKraken to monitor any input while the app is not open... like the camera following "Mouth Open" status while AUM is dominant. Failing that could MusicKraken support the split-screen that Piano Motifs supports so I can open it and
    have it on the right edge of the screen.

    Re-packaging MusicKraken as an AUv3 would be useful but I'm not sure if an AUv3 can access the camera.

    Aha! It runs on my iPhone. Let me try that as the sensor and the iPad as the intended target.
    Bluetooth MIDI?

    Nope, Musikraken must be open to track and just tried to split screen AUM and ‘kraken to no avail.
    AUv3 would be great and like wise unsure if camera will work in that situation.
    This is where holon.ist wins over because it continues to work when AUM is the preffered screen. The downside of holon.ist is it work best with mirack since the data needs to be slewed in order to have something playable.

    Yes if you are lucky enough to have an iphone and ipad then that would surely be the ideal combination. Whilst one is tracking the data the other can be manipulating and recording.

  • @Snarp said:

    @Toastedghost said:
    A little more musical musing Musikaken’s hand gestures. No fancy filters just my ugly mug. Got to work out how I could just show hands without face. Hands as maybe puppets?

    I thought about adding a 3D model of a Kraken that is overlayed over the body instead of these red dots... but then dismissed it... Maybe one day, when I'm bored...

    I do have an Octopus puppet that I might try, might be cool.

  • @McD said:
    That's the ticket... I will try linking "Mouth Open"on the iPhone to SWAM Cello's vibrato depth on my iPad.

    It works great... I have an oral AfterTouch sensor using the iPhone's camera and a Bluetooth
    connection to the iPad running AUM. It works great after being "MIDI Learned" to the vibrato depth parameter in SWAM Cello... look Ma', no hands. Definitely pays for itself with this simple use case for $10. Hardware solutions like "rings" and such are pretty pricey.

    Aha! It works on IOS 11. I have an iPhone 5S I might use just for this purpose
    and a really old iPad.

    Now I need to power up these older devices to test.

  • @Snarp said:

    @Toastedghost said:
    A little more musical musing Musikaken’s hand gestures. No fancy filters just my ugly mug. Got to work out how I could just show hands without face. Hands as maybe puppets?

    I thought about adding a 3D model of a Kraken that is overlayed over the body instead of these red dots... but then dismissed it... Maybe one day, when I'm bored...

    I think this needs to be higher up the to do list.

  • @ashh said:

    @Snarp said:

    @Toastedghost said:
    A little more musical musing Musikaken’s hand gestures. No fancy filters just my ugly mug. Got to work out how I could just show hands without face. Hands as maybe puppets?

    I thought about adding a 3D model of a Kraken that is overlayed over the body instead of these red dots... but then dismissed it... Maybe one day, when I'm bored...

    I think this needs to be higher up the to do list.

    A dancing Kraken would be fun for sure.

  • McDMcD
    edited May 2021

    @Snarp
    Do you think Apple will allow access to the camera in an AUv3 app?

    I'm getting the most excited by the potential for hands-free CC, AfterTouch and Note events
    driven by camera events. But I need the camera to be active while I'm using my hands in AUM. If it's NOT allowed I'll just keep using the 2nd device for camera sensor use.

    Another idea I think you would be able to entertain is opening a video and sending a correlated MIDI stream that reacts to events in the video... like when there's a "cut" from
    scene to scene in he video or a rapid or rhythmic movement in he video (probably harder to detect in milliseconds). But sending out what I would use as "accents" I could slip into my
    generative music scripts to trigger chords would be a great app or extension of MusiKraken.

    Can you add a face feature that triggers a "send" when I:

    1. nod my head down
    2. tilt my head up
    3. turn to the right
    4. turn to the left

    I'd like to use your app as a trigger for page turning music. This could effectively use used wherever I currently use the BlueBoard which has 4 buttons. Each of the gestures has a On and Off phase like a button press.

    Another use would be targets I could touch visually in a "virtual box" shown in the camera view. Ideally, being hands free I'd move my eyes towards 4 or more areas of the virtual box. Even look right, look left would be helpful.

    Thanks for making this controller app.

  • edited May 2021

    @Snarp said:
    It would be easy to add IAA as well, but theoretically this is deprecated by Apple, that is why I didn't want to spend any time on it if they could remove it with any update. But it's been deprecated now for a while already, maybe I should add it anyways...

    I don’t know if I would find any value in an IAA version - I spend most of my time hoping that IAA only apps will be updated to AUv3.

    As it is being deprecated, I don’t know I should even invest time in IAA when there’s an AUv3 option.

  • @ashh said:

    @Snarp said:
    I thought about adding a 3D model of a Kraken that is overlayed over the body instead of these red dots... but then dismissed it... Maybe one day, when I'm bored...

    I think this needs to be higher up the to do list.

    It probably wouldn't even be very difficult to add a moving 3D model to the current app. Maybe after a few beers....

  • @McD said:
    @Snarp
    Do you think Apple will allow access to the camera in an AUv3 app?

    Probably not, but I will try it out....

    Can you add a face feature that triggers a "send" when I:

    1. nod my head down
    2. tilt my head up
    3. turn to the right
    4. turn to the left

    If your device has a TrueDepth front camera (newer iPhones and iPad Pros), you can already do that. You can even track your tongue. See here for an example:

    The reason why I haven't added this to the face tracking without the ARKit and called it mouth tracking instead is that the Vision framework that I am using for this only reports the rotation of the face in 45 degree steps, which isn't that much fun when using with MIDI. But maybe I can compute better rotation values myself by using the face landmarks, I will try this out and see if I can improve the 2D face tracking... And maybe try to find other things in the vision framework that I could misuse for music making (but not sure how much sense it makes to add things like text or object detection :D )

Sign In or Register to comment.