Loading new images into mworks without restarting experiment

Support
1

Hi Chris,

We’re trying to move forward with neural control projects and want to achieve something like closed-loop control. This means that we will try to generate images based on neural data and stream them into mworks as fast as possible, within the same recording session. Rob will be implementing this.

My current understanding is that even with the load setting in mworks set to explicit and we unload and load every trial, there’s some read-in of everything at the beginning?

Ideally we would set something up where say 100 images exist at the beginning (x reps), image 101-200 are generated using these. Then these are added to the experiment so that when the monkey reaches the end of the initial images, indexing continues into 101 that we have pushed onto the mworks PC into the right folder at that point. Perhaps with an additional catch that if it finds 101 not to be existing yet (compute delays or so), it gets more trials from 101-200 until 101 is found to exist…

Is this currently within the capabilities of mworks?

I think the catch isn’t the hard part, I am just not sure if we can add fresh images in any way. Could we perhaps cheat with placeholder images of the same name?

Thanks!

Alina

2

Hi Alina & Rob,

It is possible to generate images dynamically and load them in to MWorks on the fly.

I’m attaching an example experiment that demonstrates this. I wrote it for a user who wanted to generate and present white noise images on demand, but the method should work for you, too. The key points are the macro update_dynamic_white_noise in experiment.mwel and the Python script create_whitenoise_images.py, which is loaded as a Python file resource by the experiment. You can ignore all the stuff involving MATLAB, which is there only because the user in question wanted to use MATLAB to create the images.

Hopefully the example will make sense, but please let me know if you have questions.

Cheers,
Chris

3

Attachment: experiment1.zip (6.3 KB)

4

Hi Chris,

This looks promising at first glance, thanks for that!

I assume we can make a python file that then looks for new images written and outputs these to be loaded or unloaded.

5

I assume we can make a python file that then looks for new images written and outputs these to be loaded or unloaded.

Sure, that’s one option. The Python script could also control the generation of the new images, based on the state of the experiment.

The key takeaway from the example I sent is how to load a completely dynamic set of images. How you generate and manage the images is up to you.

Chris