Resources for iOS audio apps development

I started playing a bit with iOS development and I am wondering if there's any community or "must-have" resources for iOS audio development specifically - forums, tutorials, articles... Even if you know about some very good resources about iOS development or audio (and MIDI) programming in general, it would be interesting for me.
There's a lot of tutorials on iOS/swift all over the web but of course, 99% is not usually worth it. I am experienced (I dare to say "professional") developer for 15 years, currently doing web frontend, so I am not really interested in the basics like what is variable, function, loop,... how and why do unit testing, what is build process, what are design patterns etc... I would rather focus on specifics of Swift and Xcode and of course audio/MID processing.
I have checked some of the Audiobus documentation and started playing around with AudioKit, which also have some interesting tutorials, so I am definitely going to dig into these.
Please, feed me with some goodness :smile:

Comments

  • maybe JUCE framework is also an option?

  • Do you want the good news or the bad news first? Just kidding, there is no good news.

    If you know what you want to work on audio wise, it might be helpful in guiding you to useful resources.

    I personally don't like JUCE, but that is certainly just my opinion. I prefer to work with native libraries in almost all situations.

    AudioKit is absolutely a good place to look. I haven't developed anything with it myself, but I have looked at the demos and samples and read the docs. It could be very useful if it fits with what you want to develop.

    If you want to develop AU's, Apple has sample code that is a starting point. It's at,

    https://developer.apple.com/documentation/audiotoolbox/creating_custom_audio_effects and
    https://developer.apple.com/documentation/audiotoolbox/incorporating_audio_effects_and_instruments

    These are the new betas for the OS's that'll be released this fall.

    As some general things to look at, for audio work on iOS, you are probably going to need to know C++, Objective-C/C++, and Swift. Understanding plain old C is also very useful.

    Apple's Accelerate framework is very useful. I use it in everything I do and it can make many things faster and easier to do.
    https://developer.apple.com/documentation/accelerate

    If you are new to DSP, this online book is a useful starting point,
    http://dspguide.com

    There is a ton of information on algorithms and DSP for audio and music here,
    https://ccrma.stanford.edu/~jos/

    For MIDI stuff, Apple's Core MIDI framework and the docs for the midi spec are the onlything I ever really used,
    https://developer.apple.com/documentation/coremidi
    https://www.midi.org

    There is actually lots of information on the web for working in audio, but wading through it and figuring out where to start is not an easy task. A bit more information about what you want to do might be really helpful in pointing you at useful resources. Also, knowing about your math and engineering background could be helpful.

  • Adding to the Midi section: http://audeonic.boards.net
    These guys have a nice framework/library (used by a lot of 'big selling' developers for midi tasks, so there must be a reason... and saves you from reinventing the wheel)

  • There’s also the Audiobus Developer site.

  • Avoid global variables in AUv3 plugins.

  • This could be interesting...
    https://www.facebook.com/olilarkinplugins/posts/411372009467342?xts%5B0%5D=68.ARCwrTsgPuyKrBcW8ckAbHuK3m6ee_cMFYBE_tQCiOdB1hd2eBtwOWpeeeJu_9DriwX5SJNwqPhPJlCFa_DewGwZAmHHNpA4YTEjrexLB19OA8CLcZHbHgqdqcupTpsgXd8OY9LS5ZVw_2X2KLjmdOEqoPI1FdTm5-47yV8pMyeDZxULqQ2ai4JY_LIdZSmC1kpbwaHWl3cZ_UKvW5UGwnvWvpnQU5jqTa-j42jSD41IF6dFjbC7g2M1oPGOGPA4ZcmT99DEZjQRSeNR_Im8MWUSOGV1dHfPC44c60ezxnL291HcsTvM_3dw3Dfm&tn=C-R

  • If you want to make good AUv3 plugins, especially for audio processing, C++ and JUCE framework i probably best way to go
    https://juce.com/learn

  • @TrevorLlewellyn said:
    maybe JUCE framework is also an option?

    Thanks, I've heard many times about JUCE and it looks very powerful, but not sure if it wouldn't be overwhelming for the start. Also, I would like to focus only on iOS (and eventually MacOS if Marzipan proves like a way to go), to keep it simple, so I wouldn't use JUCE's biggest advantage - multiplatform support.

    @NeonSilicon said:
    Do you want the good news or the bad news first? Just kidding, there is no good news.

    If you know what you want to work on audio wise, it might be helpful in guiding you to useful resources.

    I personally don't like JUCE, but that is certainly just my opinion. I prefer to work with native libraries in almost all situations.

    AudioKit is absolutely a good place to look. I haven't developed anything with it myself, but I have looked at the demos and samples and read the docs. It could be very useful if it fits with what you want to develop.

    If you want to develop AU's, Apple has sample code that is a starting point. It's at,

    https://developer.apple.com/documentation/audiotoolbox/creating_custom_audio_effects and
    https://developer.apple.com/documentation/audiotoolbox/incorporating_audio_effects_and_instruments

    These are the new betas for the OS's that'll be released this fall.

    As some general things to look at, for audio work on iOS, you are probably going to need to know C++, Objective-C/C++, and Swift. Understanding plain old C is also very useful.

    Apple's Accelerate framework is very useful. I use it in everything I do and it can make many things faster and easier to do.
    https://developer.apple.com/documentation/accelerate

    If you are new to DSP, this online book is a useful starting point,
    http://dspguide.com

    There is a ton of information on algorithms and DSP for audio and music here,
    https://ccrma.stanford.edu/~jos/

    For MIDI stuff, Apple's Core MIDI framework and the docs for the midi spec are the onlything I ever really used,
    https://developer.apple.com/documentation/coremidi
    https://www.midi.org

    There is actually lots of information on the web for working in audio, but wading through it and figuring out where to start is not an easy task. A bit more information about what you want to do might be really helpful in pointing you at useful resources. Also, knowing about your math and engineering background could be helpful.

    Thanks a lot for the exhaustive list, this is what I was looking for! Yes, audio is obviously one of the toughest area for programming, but also the most rewarding ;)

    My current plan for an "MVP" app is to create a simple looper grid (X = channels, Y = scenes, cells are playable/stoppable buttons), ideally able to trigger both audio and MIDI. That would already require quite a lot of things to embrace and implement - transport, tempo, time stretching, mixer, MIDI (only recording from external source, still I believe it's a huge challenge) and of course, the whole GUI, layout, etc... I would like to first make it Audiobus compatible (and of course Core MIDI compatible), IAA is probably not a way to go now.

    I am not interested in building a synth or sound effect , this is why I would like to start with using AudioKit (maybe other framework/s) as much as possible. I don't want to have control over everything - rather the opposite. Just be able to get a working app the easiest (but still reasonable) way possible.

    As for engineering background, I haven't been programming in C nor C++, I have tried objective C long time ago to make a simple iOS app (around iOS 5-6?) but don't remember anything :smiley:
    But I have started learning Swift, it looks very similar to TypeScript I use on a daily basis and I would love to stick to it as much as possible as I like it much more than Objective C ;)

    Thanks a lot again, I have at least some starting points and already quite a lot of material to study!

  • Keep in mind that for audio work you'll need your code to be "realtime safe". This means in practice that any DSP/MIDI stuff that happens on the realtime-thread needs to be written in C (or carefully checked C++) code. High-level languages like Swift, Objective-C or the convenient parts of C++ (hello vectors!) are essentially a big no-no.

    Our host Michael has written a must-read blog post on this:
    http://atastypixel.com/blog/four-common-mistakes-in-audio-development/

  • Jonatan Liljedahl (Kymatica) has some great pointers as well:
    http://devnotes.kymatica.com

  • edited July 5

    I would definitely agree with your starting point of using AudioKit. For one thing, it's going to allow you to do your project in Swift and maintain the critical path that @brambos points out of keeping the audio thread realtime safe (and this really is the most important thing in an audio setting).

    Since you do want the possibility of putting your application on macOS eventually, SwiftUI would probably be the easiest thing to use to do that. SwiftUI also looks to me like a great way to do UI development in general. The only problem with that is that it is still beta (for a few months) and I don't have any feel for how it is going to mix with AudioKit. I'd try to see if anyone in the AudioKit world could give you pointers on this and start learning SwiftUI if it looks like it is going to be possible in the near term. If SwiftUI doesn't look like it's going to work well with AudioKit any time soon, then UIKit will still get you where you want to go.

    The other issue with using SwiftUI now is that you'd have to install the betas of macOS and iOS on your devices.

    I'd suggest that you do still want to learn C and C++ because eventually you are likely going to need to dig in a bit deeper and extend or alter the functionality of something in AudioKit. I think it's best to actually learn straight C first. I do all of my DSP stuff in a very small subset of C++ (basically C with a bit of C++ polymorphism) and understanding C and how it works is still the most important aspect of this. I think you'll be able to get a long way using AudioKit without getting into C/C++ (maybe all the way), but it's still good to be versed in C even if all you need it for is understanding how something is working when you need to read the AudioKit code.

  • @TheOriginalPaulB said:
    Avoid global variables in AUv3 plugins.

    There can be a time and place for global AUv3 variable functionality. I’m glad Mozaic supports this as I plan to create some presets that will use global variables to integrate multiple instances of Mozaic together so they function in unison.

  • edited July 5

    @InfoCheck said:

    @TheOriginalPaulB said:
    Avoid global variables in AUv3 plugins.

    There can be a time and place for global AUv3 variable functionality. I’m glad Mozaic supports this as I plan to create some presets that will use global variables to integrate multiple instances of Mozaic together so they function in unison.

    Lol. It was a tongue in cheek reference to the Spectrum dev’s lament in another thread...

  • @TheOriginalPaulB said:
    Avoid global variables in AUv3 plugins.

    :lol:

  • While I was reading your docs for Mozaic, I realized you should write a book for iOS music app development!

    @brambos said:
    Keep in mind that for audio work you'll need your code to be "realtime safe". This means in practice that any DSP/MIDI stuff that happens on the realtime-thread needs to be written in C (or carefully checked C++) code. High-level languages like Swift, Objective-C or the convenient parts of C++ (hello vectors!) are essentially a big no-no.

    Our host Michael has written a must-read blog post on this:
    http://atastypixel.com/blog/four-common-mistakes-in-audio-development/

  • @brambos said:
    Keep in mind that for audio work you'll need your code to be "realtime safe". This means in practice that any DSP/MIDI stuff that happens on the realtime-thread needs to be written in C (or carefully checked C++) code. High-level languages like Swift, Objective-C or the convenient parts of C++ (hello vectors!) are essentially a big no-no.

    Our host Michael has written a must-read blog post on this:
    http://atastypixel.com/blog/four-common-mistakes-in-audio-development/

    Thanks a lot, this was very interesting read, I am still slowly revealing all of these specifics of audio development and such articles helps a lot!
    And yes, as @Antkn33 said, it would be extremely valuable, if you'd share some of your wisdom about making iOS apps. I am still amazed about your skills in both UX and development. Your apps are super solid, Rozeta suite was AFAIK the first AU MIDI plugin out there and all you apps look and feel amazing.

    @NeonSilicon said:
    I would definitely agree with your starting point of using AudioKit. For one thing, it's going to allow you to do your project in Swift and maintain the critical path that @brambos points out of keeping the audio thread realtime safe (and this really is the most important thing in an audio setting).

    Since you do want the possibility of putting your application on macOS eventually, SwiftUI would probably be the easiest thing to use to do that. SwiftUI also looks to me like a great way to do UI development in general. The only problem with that is that it is still beta (for a few months) and I don't have any feel for how it is going to mix with AudioKit. I'd try to see if anyone in the AudioKit world could give you pointers on this and start learning SwiftUI if it looks like it is going to be possible in the near term. If SwiftUI doesn't look like it's going to work well with AudioKit any time soon, then UIKit will still get you where you want to go.

    The other issue with using SwiftUI now is that you'd have to install the betas of macOS and iOS on your devices.

    I'd suggest that you do still want to learn C and C++ because eventually you are likely going to need to dig in a bit deeper and extend or alter the functionality of something in AudioKit. I think it's best to actually learn straight C first. I do all of my DSP stuff in a very small subset of C++ (basically C with a bit of C++ polymorphism) and understanding C and how it works is still the most important aspect of this. I think you'll be able to get a long way using AudioKit without getting into C/C++ (maybe all the way), but it's still good to be versed in C even if all you need it for is understanding how something is working when you need to read the AudioKit code.

    Thanks for your words, actually this is exactly my thinking right now - I have already went trough the Apple's tutorial for SwiftUI and it's awesome, it reminds me a lot React from javascript world I use in my day job. I have already tried to use it in my test app together with AudioKit but as you say - it's not currently possible, SwiftUI requires Swift 5.1, AudioKit is currently running on 5.0, but it looks like guys are aware and they are going to update it. Still not sure if they'll mix together well, so currently I am just working on my view layer in UIKit.

    Now I only need a lot of time, that's usually the toughest part :grimace:

Sign In or Register to comment.