-We haven’t been able to run our HoldAndDetectConstant19.mwel on the updated version of MWorks due to the way our machines are configured. This should be resolved by next week.
-If we plug something into the iMac audio jack mid experiment and then unplug it, then sounds are normal, even after the experiment is playing. When the audio automatically snaps back to HDMI output, it is normal. A caveat here is that I cannot hear internal MWorks sounds after unplugging the jack. I don’t know if this is a volume issue, because I can still hear non-MWorks sounds.
Again,
Play FindTheCircle → distorted HDMI sounds
Plug in to iMac jack
Unplug iMac jack
Normal sounds out of HDMI monitor
Also, now, if the sounds are distorted (because i haven’t plugged into the jack), and I stop the experiment, sounds return to normal. Previously I had to entirely quit MWorks for normal sounds. Let me know if you want to chat again.
This all is sounding like some kind of internal issue in the macOS audio frameworks that MWorks uses. This is frustrating: One of the main reasons for the audio revamp in MWorks 0.12 was to move from deprecated API’s to the current, recommended ones, and it’s very unfortunate that the recommend API’s don’t seem to work correctly with your hardware. Maybe HDMI audio doesn’t get a lot of attention from Apple’s engineers?
In any case, I doubt there are any changes I can make to MWorks to fix this issue. The debugging info you’ve shared shows that everything is set up correctly, and it all works as expected except for (sometimes?) on the HDMI output.
Have you tried connecting an audio cable to the monitor? If that works consistently, then I still think it’s probably the best workaround.
I am currently working on the aux jack solution but I haven’t quite got it working yet, even though of course it should just be plugging the cord in, I have encountered volume trouble. I will send you specific notes when I have spent more time on this. We are also working to update some of our older iMacs in the next couple of months, and it is possible newer machines work better with HDMI audio.
I hope you are having a good week. Huge progress here! We now have a simple solution:
Launch MWorks (23.10.18)
Load Experiment
Unplug and replug from the thunderbolt port on the iMac
Now all sounds internal and external to MWorks are normal.
One question, is there a way to “unplug and replug” the thunderbolt ports within the code that loads the experiment? I think it would have to be executed at the end of this code.
That’s very interesting. So switching output devices somehow resolves the issue. I wonder if a software-only switch would also work. Can you try the following?
Launch MWorks.
Load experiment.
Without disconnecting any cables, switch from HDMI audio to built-in speakers and then back to HDMI audio.
Does that also result in normal sounds? If so, then there might be hope for fixing this in MWorks after all.
Here are the two main iterations that do/don’t work:
Audio midi on built-in output
Launch mworks app
Load experiment
Audio midi on HDMI
Sounds are good
Audio midi on HDMI
Launch mworks app
Load experiment
Audio midi on built-in output
Audio midi on HDMI
Sounds are bad and cannot be switched to good through audio midi changes.
Ultimately, if we start of HDMI through the software, we cannot return to normal once the experiment is loaded. Also, Start/Stop seems irrelevant. Connor Phillips (cc’d) and I tried a bunch of different iterations of Launch App, Load Experiment, Start/Stop, and switching audio, so let us know if you want more details there.
As a first check of my theory, could you do some quick sound tests using MWorks 0.12.1? There’s a change that went in to 0.12.2 that I think may be involved in your issue. If sounds work normally in 0.12.1, that would confirm that I’m on the right track.
You are correct. Sounds are normal with this version.
Aha! So this is MWorks’ fault after all! (Hooray?)
The relevant change in 0.12.2 was made to reduce the start-up latency of sound playback. The fact that you can (sometimes) resolve your audio issue by switching output devices was a crucial hint, because it made me realize that the start-up latency reduction code only ran on the initial output device.
Note that this doesn’t really explain the problem yet, since it happens even when you load your experiment with the HDMI output active and don’t switch outputs. But it’s given me some ideas for changes that might help. And if all else fails, we can always add an option to disable the latency reduction code.
Thanks so much for all your debugging work! I’ll let you know when I have a potential fix for you to test.
Can you please try some sound tests using the current MWorks nightly build?
The nightly contains a fix for the issue of running the start-up latency reduction code only on the initial audio output device. Now, MWorks monitors output device changes and will run that code for any new device.
Also, I fixed another bug I discovered, which is somewhat similar to yours. Specifically, if I loaded an experiment using one audio output device (e.g. headphones), ran the experiment, switched to another output device (e.g. built-in speakers), and ran the experiment again, sounds didn’t play at all in the second run. Why the change I made fixed it is not clear. The issue was somehow triggered by the previously mentioned latency reduction code, and it was somehow fixed by calling a nominally unnecessary audio API method before running that code. I discovered the fix just by guesswork and experimentation, but without being able to look in to the internals of Apple’s audio frameworks, I can’t say why it works.
Anyway, I have modest hopes that these changes may fix your problem, too. But there’s also a good chance that they won’t. In that case, let me know, and we’ll try something else.
FYI, I’m going to be out on vacation all of next week, so I won’t be able to help much with this until I’m back the following week.
We tested today’s version of MWorks with FindTheCircle. Loading the experiment did not break external nor internal MWorks sounds. Playing the experiment broke the external MWorks sounds but I don’t think it broke the internal MWorks sounds (less familiar with these sounds but they sound quite normal.). Looks like we are getting closer!
Would you be free to Zoom when you are back from vacation and have your next iteration so we can troubleshoot in real time?
Loading the experiment did not break external nor internal MWorks sounds. Playing the experiment broke the external MWorks sounds but I don’t think it broke the internal MWorks sounds
What do you mean by “external” and “internal”? Do you mean the display speakers vs. the built-in speakers? Or do you mean sounds played outside of MWorks (e.g. in QuickTime Player) vs. sounds played by MWorks?
running an MWorks experiment causes sounds played outside of MWorks to break.
Is that correct? If so, I have a few more questions:
Does running an MWorks experiment break outside sound playback on all audio output devices or just the HDMI audio?
Does closing the experiment or quitting MWServer make outside sound playback return to normal? (Based on your past tests, I assume it must.)
While it’s certainly good that the changes in the nightly build fix sound playback within MWorks, we still don’t have any insight into why they fix it. The fact that sound playback outside of MWorks is broken makes me think, again, that this comes down to an internal issue in the macOS audio frameworks.
As for what we try next, your tests with MWorks 0.12.1 suggest that adding an option to disable the start-up latency reduction code should eliminate the issue. I’m not really happy about adding a global MWorks configuration setting just to work around an obscure bug in Apple’s audio frameworks, but it seems like that may be the only software-based solution.
On the hardware side, were you ever able to get audio output working by connecting an audio cable from the Mac’s headphone output to the input on the display? If the issue is specific to HDMI audio, then that still seems like a good solution.
I’m happy to meet via Zoom to chat about all this, if that seems easier/faster.
I think chatting on Zoom would be best for me, but for a quick response, we never were able to get the audio jack solution working, and we tried a bunch of things there. We are able to use the HDMI output by switching the audio settings through audio MIDI which has been a game changer, (though a bit odd, as you mentioned).
My schedule is relatively flexible this week and next for Zoom especially Thursday 9am-2pm and 3pm-4pm, or anytime Tuesday/Thursday next week.
I’ve created a new test build of MWorks. It’s identical to the current nightly build, except that it reports the transport type of the audio output device.
When you have a chance, can you download and install the build, load any experiment with sounds (FindTheCircle is fine), and then switch back and forth between the built-in speakers and HDMI audio? Each time you change output devices, there should be a message (or two) in the MWServer console window that looks something like this:
transportType: 'bltn'
Please look for these messages and let me know what they say.
The point of this is to determine what transport type the system reports for the HDMI audio output. Probably it will be 'hdmi', but since the display is connected via an adaptor, it might present as something else (e.g. 'dprt' for DisplayPort).
Since the issue you’re having seems to be specific to HDMI, I’m thinking that MWorks could check the transport type and only run the start-up latency reduction code for non-HDMI devices. But before I make that change, I want to confirm that the OS does, in fact, see your display as an HDMI device.
I’m a postbac that works with Nina (space suit guy in one of our meetings). Sorry for the delayed reply, our experiment schedule has ramped up a lot recently. We tested the build you sent, and as you expected it does output “transportType: ‘hdmi’” it also appropriately outputs ‘bltn’ when I switch. FYI it displays the message twice every switching instance after the initial start-up. Thank you for your continued help with this!