How to remotely control *any* iOS app!

2»

Comments

  • I have just filed a bug report to Apple including steps to reproduce the issue.

  • @Janosax said:

    @rs2000 said:

    @rs2000 said:

    @Janosax said:
    Too bad, a nice use would have been to launch Gadget scenes by midi... It could be interesting to know which iOS versions are not broken.

    iOS 11.4.1 definitely works.

    This is worth a bug report @ Apple me thinks.

    Yes that’s worth it for sure. The issue with Apple bug report is their lack of communication. I’ve reported to them the GarageBand undo bug some months ago, they asked me for some song files and then no news. It has been fixed lately, so that’s cool, but the user need to test from time to time after an update to know it. However, I will look to report this one too.

    Just a question: as you assign a recipe to « launch recipe », does that mean only this recipe will works? Or you can have several recipes?

    I recommend to give the recipe the same name as your app under control.
    One recipe for each app feels like the most logical way to handle different recipes.

  • Will have a go with a bt keyboard, confirmed to not work for me with a midi device.

  • edited February 11

    @aplourde said:

    @rs2000 said:

    @rs2000 said:

    @Janosax said:
    Too bad, a nice use would have been to launch Gadget scenes by midi... It could be interesting to know which iOS versions are not broken.

    iOS 11.4.1 definitely works.

    This is worth a bug report @ Apple me thinks.

    @rs2000 Thanks for this - another very cool technique to play with!

    I'm on iOS 12.1 and I can get this to work with a Bluetooth keyboard, but not a MIDI keyboard. I can assign the Switches, but I can't trigger the Recipes; gonna keep playing with this.

    The idea is especially appealing for apps that have gestural controls, but no motion recording (SpaceCraft, Shoom, etc.) The end result isn't as clean as true motion recording as you have to pantomime the gestures on the blank recording screen (I wish you could load a screenshot...) and sometimes apps don't quite respond correctly. Still this is very fun and very cool.

    Like I said before, you can use an overhead pen and paint on the iDevice screen, you can remove it easily with a small piece of paper towel dipped into alcohol.
    A bit of practice certainly doesn't hurt :)

    For Genome MIDI, I've recorded a set of gestures that each actually does five things:
    tap "back" to go back to the session/pattern view
    tap "+" to create a new empty pattern on a new track
    tap on the new pattern to select it
    tap the "stylus" to edit the pattern in the piano roll
    tap "REC" to enable recording in the piano roll
    This is basically a MIDI looper that can be controlled (6 tracks in my case) by a footswitch.

    Genome will loop the pattern while recording and you can add notes the way you want in each cycle (overdub recording).

  • @rs2000 said:

    @aplourde said:

    @rs2000 said:

    @rs2000 said:

    @Janosax said:
    Too bad, a nice use would have been to launch Gadget scenes by midi... It could be interesting to know which iOS versions are not broken.

    iOS 11.4.1 definitely works.

    This is worth a bug report @ Apple me thinks.

    @rs2000 Thanks for this - another very cool technique to play with!

    I'm on iOS 12.1 and I can get this to work with a Bluetooth keyboard, but not a MIDI keyboard. I can assign the Switches, but I can't trigger the Recipes; gonna keep playing with this.

    The idea is especially appealing for apps that have gestural controls, but no motion recording (SpaceCraft, Shoom, etc.) The end result isn't as clean as true motion recording as you have to pantomime the gestures on the blank recording screen (I wish you could load a screenshot...) and sometimes apps don't quite respond correctly. Still this is very fun and very cool.

    Like I said before, you can use an overhead pen and paint on the iDevice screen, you can remove it easily with a small piece of paper towel dipped into alcohol.

    I wouldn't go crazy with this! The iPad has an oleophobic coating that will be degraded by alcohol. Occasional use is probably fine, but I wouldn't treat it like an overhead projector in a school!

    A bit of practice certainly doesn't hurt :)

    No doubt, but trying to do a musical performance blind and without audio is a challenge!

    I just tried playing through a piece of paper and it works. I'm going to try printing out the screen 1:1 and taping that on when recording gestures. This wouldn't work for screen automation where you're going to different screens, but I really want this to work with Shoom (It works, I'm just flying blind and deaf...)

    For Genome MIDI, I've recorded a set of gestures that each actually does five things:
    tap "back" to go back to the session/pattern view
    tap "+" to create a new empty pattern on a new track
    tap on the new pattern to select it
    tap the "stylus" to edit the pattern in the piano roll
    tap "REC" to enable recording in the piano roll
    This is basically a MIDI looper that can be controlled (6 tracks in my case) by a footswitch.

    Genome will loop the pattern while recording and you can add notes the way you want in each cycle (overdub recording).

    Very cool.

  • I had another look at this, working better for me with a Bluetooth keyboard but still buggy, the gesture recording doesn’t playback the gesture properly in the right place.
    I can see the potential for it more though now, the keyboard works for highlighting and selecting focus points, so yeah could be useful if they fixed it. :smile:

  • OK. I'm getting this set up and working but I can't figure out how to stop the scanning? In Loopy HD it keeps scanning sets of four loops, and changing focus with the blue box. It never scans the whole screen.

  • also, I'm pretty hyped about my standard logitech wireless keyboard working (at all) on my iPad. Sure enough, works great. I thought that only bluetooth keyboards work with iOS so I have never even tried using my full-size keyboard with the iPad.

  • @Hmtx said:
    OK. I'm getting this set up and working but I can't figure out how to stop the scanning? In Loopy HD it keeps scanning sets of four loops, and changing focus with the blue box. It never scans the whole screen.

    Set "Scanning Style" (below the Switches and Recipes items) to Manual to stop the scanning.

  • edited February 11

    Yeah, scanning style is set to manual. But that only stops the scanning. The focus just stays on the first item in each app.

  • edited February 11

    @Hmtx said:
    Yeah, scanning style is set to manual. But that only stops the scanning. The focus just stays on the first item in each app.

    If you still see scanning frames or lines then something is wrong.
    Using a wireless or Bluetooth keyboard definitely allows me to directly tap/toggle Loopy's Play button and each loop circle, even on an iPhone with iOS 12. Loopy looks no different with Switch Control enabled, everything happens "silently".

  • @Hmtx said:
    Yeah, scanning style is set to manual. But that only stops the scanning. The focus just stays on the first item in each app.

    It should disappear after the Auto Hide time....

  • edited February 11

    @Hmtx it seems you have to assign your recipe to « Launch recipe » to avoid that behavior:

  • edited February 11

    @Janosax you go it! Yeah, for some reason I didn't really understand that. So, it has to be set up to "launch recipe" when switch control is enabled. And put all your switches into that recipe. Success!

    I can now connect my Logitech keyboard and draw a smiley face in Notes with a press of numpad 3 and a frownie face with a press of numpad 1. Now on to more useful things.

  • edited February 11

    @Hmtx said:
    @Janosax you go it! Yeah, for some reason I didn't really understand that. So, it has to be set up to "launch recipe" when switch control is enabled. And put all your switches into that recipe. Success!

    My text is too long, isn't it? :D
    Congrats and have fun!

  • @rs2000 said:
    My text is too long, isn't it? :D
    Congrats and have fun!

    or my attention span is not long enough :|

    thanks for this B)

  • edited February 11

    Oh crap > @aplourde said:

    @rs2000 said:

    @rs2000 said:

    @Janosax said:
    Too bad, a nice use would have been to launch Gadget scenes by midi... It could be interesting to know which iOS versions are not broken.

    iOS 11.4.1 definitely works.

    This is worth a bug report @ Apple me thinks.

    @rs2000 Thanks for this - another very cool technique to play with!

    I'm on iOS 12.1 and I can get this to work with a Bluetooth keyboard, but not a MIDI keyboard. I can assign the Switches, but I can't trigger the Recipes; gonna keep playing with this.

    The idea is especially appealing for apps that have gestural controls, but no motion recording (SpaceCraft, Shoom, etc.) The end result isn't as clean as true motion recording as you have to pantomime the gestures on the blank recording screen (I wish you could load a screenshot...) and sometimes apps don't quite respond correctly. Still this is very fun and very cool.

    Oh man, automations triggered by a BT Numeric keypad?! 🤯

  • @Dr_M said:
    Oh crap > @aplourde said:

    @rs2000 said:

    @rs2000 said:

    @Janosax said:
    Too bad, a nice use would have been to launch Gadget scenes by midi... It could be interesting to know which iOS versions are not broken.

    iOS 11.4.1 definitely works.

    This is worth a bug report @ Apple me thinks.

    @rs2000 Thanks for this - another very cool technique to play with!

    I'm on iOS 12.1 and I can get this to work with a Bluetooth keyboard, but not a MIDI keyboard. I can assign the Switches, but I can't trigger the Recipes; gonna keep playing with this.

    The idea is especially appealing for apps that have gestural controls, but no motion recording (SpaceCraft, Shoom, etc.) The end result isn't as clean as true motion recording as you have to pantomime the gestures on the blank recording screen (I wish you could load a screenshot...) and sometimes apps don't quite respond correctly. Still this is very fun and very cool.

    Oh man, automations triggered by a BT Numerical Keyboard?! 🤯

    💡👌🏼🌈☀️🍾🥂

  • @rs2000 THANK YOU FOR BREACHING THIS SUBJECT

  • @Dr_M said:
    @rs2000 THANK YOU FOR BREACHING THIS SUBJECT

    You're welcome :)
    I just found it too exciting to keep it for myself o:)

  • this is just mind-boggling. If I have a loopy track that I want to load up in Sector and get glitchy... done with a tap of a button, maybe even mid-song?

  • @rs2000 said:

    @Hmtx said:
    Yeah, scanning style is set to manual. But that only stops the scanning. The focus just stays on the first item in each app.

    If you still see scanning frames or lines then something is wrong.
    Using a wireless or Bluetooth keyboard definitely allows me to directly tap/toggle Loopy's Play button and each loop circle, even on an iPhone with iOS 12. Loopy looks no different with Switch Control enabled, everything happens "silently".

    Thanks so much for documenting this, @rs2000. I tried setting it up for Gadget transport controls before they opened it up to Midi CC's, but never figured out how to get the scanning lines to turn off.

  • Video pls!

  • I've just created a set of remote control gestures for my navigation system:
    Zoom in, zoom out, move up/down/left/right, re-center and follow GPS position (This is what I need on the motorcycle for spontaneous routing decisions).
    Now I'm waiting for this $4 BT game controller to arrive (hopefully it will work):

Sign In or Register to comment.