Hi Mehrdad & Evan,
I’m writing to give you a quick update on the status of running MWorks on iOS devices, as well as to request some input on its further development.
Current Status
At present, nearly all of MWServer is ported to and running on iOS. This includes all existing stimulus types, actions, and experimental-flow-control components (state machines, blocks, trials, etc.) The only parts missing are I/O devices, as all the existing ones rely on hardware and/or software that isn’t compatible with iOS. Also, I haven’t yet added support for touch input, but that should be straightforward and hopefully will be done sometime this week. Experiments are loaded and controlled from a Mac via the existing MWClient application, just as they are with the macOS version of MWServer.
The port is designed to be compatible with all iOS devices that are currently for sale, including iPads, iPhones, and iPods touch. More specifically, the device needs to support 64-bit instructions and OpenGL ES 3 (meaning the SoC must be an Apple A7 or later), and it must be running iOS 10.2 or later (though I will likely bump that to 10.3 once that OS version comes out of beta). During development, my test device has been an iPad mini 2, which is the cheapest iPad currently available and just meets the minimum required hardware specs. However, I’m finding it to be a bit underpowered – specifically, it regularly drops frames when displaying dynamic stimuli – so I’d probably recommend something newer for production use.
Next Steps
At this point, I see two significant questions that need answering: How should we support external I/O, and how should we handle storage of MWorks event data?
I/O Devices
The only built-in, wired I/O interface that’s directly accessible to MWorks is the headphone jack, which can provide two channels of analog output. While this may be useful and shouldn’t be dismissed out of hand, it is obviously limited. Also, Apple has already removed the headphone jack from the iPhone 7 and presumably will do the same with all future iOS devices, so we can’t rely on it long term.
The only other (and soon to be the only) wired interface to an iOS device is the Lightning port. Access to this port is strictly controlled by Apple, but there are existing external hardware products that could be useful for our purposes. For example, Apple’s Lightning to USB 3 Camera Adapter expressly supports USB Ethernet adaptors, which would allow us to connect to Ethernet-capable I/O devices. Also, a company named Redpark sells Lightning to serial cables, which would give us access to wired serial I/O.
As for wireless connections, both WiFi and Bluetooth are available and could be used to access I/O hardware. I think the most appealing options in this space are devices like the Redbear Blend Micro and the Arduino 101, which are low cost, have built-in support for Bluetooth, and provide a variety of analog and digital I/O ports. It should be straightforward to extend MWorks’ existing Firmata interface to support these devices. If you want a more traditional DAQ, the Labjack T7 has built-in WiFi, although getting MWorks to talk to it from iOS would take some work.
What are your thoughts here? Do you have a sense of what hardware you may want to incorporate in to your iOS-based experiments? (I assume that, if nothing else, you’ll need to control a juice pump.)
Event Storage
At the moment, MWServer on iOS writes event (i.e. .mwk) files to local storage, just like MWServer on macOS. This presents a few problems:
- Internal storage is limited on iOS devices, and there’s no way to add external storage or otherwise expand it.
- At present, there’s no way to get event files off of an iOS device.
Certainly issue 2 is solvable. In fact, Evan has already requested that I add support for client-side retrieval of event files, and the solution there would be usable on both iOS and macOS. However, I’m not sure we have any options for addressing issue 1.
A better solution may be to stream MWorks events (via WiFi) from an iOS device to a dedicated external system (e.g. a Mac or PC) with ample storage space. Since event streaming is already an integral part of MWorks’ architecture, this shouldn’t present any major challenges. However, it would require every MWorks-using lab to maintain one or more “MWorks data servers”, which would be annoying but probably workable.
We could also support both of the above options, i.e. on-device and off-device event storage.
How do you feel about these two approaches? Do you have other ideas about how to handle event storage?
Thanks in advance for your time and input.
Chris