Request for Assistance with MWorks

Support
1

Hello Chris,

Good morning.

I’m Apaala, a new postdoc in Yasmine El-Shamayleh’s lab at Columbia. I’m developing a new behavioral task in MWorks and was hoping to get your advice on the best way to structure and load a relatively large stimulus set in MWEL.

Task Basics

On each trial, the task presents a single image (.jpg) from one of two main stimulus categories, A or B. Each category contains multiple stimulus variants, and for a given session we would like to be able to select an arbitrary subset of those variants and sample images only from the selected groups. The number of variants and the number of images within each variant may differ.

Current Data Structure

Right now, I am not using a hierarchical/tree-style directory structure for the task images. I currently have a single parent folder containing multiple stimulus folders at the same level, for example:

Task_images/
├── A/
│   ├── A_1.jpg
│   ├── A_2.jpg
│   └── ...
├── B/
│   ├── B_1.jpg
│   ├── B_2.jpg
│   └── ...
├── B2/
│   ├── B2_1.jpg
│   ├── B2_2.jpg
│   └── ...
├── B3/
│   ├── B3_1.jpg
│   ├── B3_2.jpg
│   └── ...
└── ...

So at the moment, A, B, B2, B3, etc. are all separate folders under the same parent directory.

For a given session, I would ideally like to be able to activate an arbitrary subset of these folders.

Current MWEL Implementation

Right now, I define a separate stimulus_group for each folder and use a range_replicator to instantiate the individual image_file stimuli:

%define total_images_in_folder = 14

stimulus_group A_Images {
    range_replicator (
        variable = A_counter
        from = 1
        to = total_images_in_folder
        step = 1
    ) {
        image_file A_image_${A_counter} (
            path = 'Task_images/A/A_${A_counter}.jpg'
            x_size = stimulus_display_size
            y_size = stimulus_display_size
            x_position = stimulus_x
            y_position = stimulus_y
            announce_load = false
        )
    }
}

I currently have 14 images in each folder and repeat this pattern for A, B, B2, B3, etc.

At runtime, I separately control which image indices are active through GUI variables. The task selects the stimulus folder/category and the image index independently, and those selections are combined at the end to index the appropriate stimulus_group.

This works, but it requires me to manually duplicate the MWEL structure for every folder, and the number of images in each folder is currently hard-coded.

Questions

1). Would a tree/hierarchical data structure be a better way to organize the image folders?
For example, would it make more sense to reorganize the stimulus library as something like:

Task_images/

├── A/

│   ├── A1/
│   ├── A2/
│   └── ...
└── B/
    ├── B1/
    ├── B2/
    └── ...

and then represent the stimulus selection as something like category → variant → image? Or is there another data structure/pattern in MWEL that would scale better than either of these approaches?

2). Can the MWEL dynamically determine the contents of a directory?

Currently, range_replicator requires me to specify total_images_in_folder. Is there a supported way to enumerate files in a directory, or otherwise determine the number of available images at load/parse time? Ideally, the folders could contain different numbers of images without requiring a separate hard-coded count for each one.

3). Is there a cleaner way to select active stimulus folders at runtime?

Ideally, I would like the GUI or session configuration to specify something equivalent to:
active_groups = [A, B2, B5], and have the trial logic sample only from the images belonging to those groups. Is there an MWEL/MWorks pattern you would recommend for this instead of maintaining many separate stimulus_groups and selection branches?

4). Can stimulus categories be represented differently in the Eye Monitor?

Is there a way to assign different visual properties to stimulus announcements in the MWorks Eye Monitor based on the group/category? For example, could stimuli from category A versus category B be displayed with different colors or markers in the Eye Monitor.

My main concern is scalability and maintainability. The current implementation works, but as the number of stimulus folders increases, I want to avoid duplicating a large amount of MWEL code if there is a more appropriate MWorks-native way to structure it.

Thanks so much in advance for your help!

Apaala

5

Hi Apaala,

Would a tree/hierarchical data structure be a better way to organize the image folders?

Unfortunately, MWorks doesn’t support creating nested stimulus groups using replicators. You can organize your files in a tree if you like, but you’ll still be restricted to a single level of stimulus groups in the experiment.

Can the MWEL dynamically determine the contents of a directory?

Yes, you can do this with a list replicator and the filenames directive. For example:

stimulus_group A_Images {
    list_replicator (
        variable = A_path
        values = 'filenames(Task_images/A/*.jpg)'
        ) {
        image_file ${A_path} (
            path = ${A_path}
            x_size = stimulus_display_size
            y_size = stimulus_display_size
            x_position = stimulus_x
            y_position = stimulus_y
            announce_load = false
            )
    }
}

You can get the number of images in the directory using the filenames function:

var num_A_Images = size(filenames('Task_images/A/*.jpg'))

Is there a cleaner way to select active stimulus folders at runtime?

This is a little tricky. Let me think about it and put together an example.

Can stimulus categories be represented differently in the Eye Monitor?

No, that isn’t possible at present.

Cheers,
Chris

6

Hi Apaala,

I’ve attached an example experiment that demonstrates a more flexible way to select active stimulus variants.

The example defines two image categories, “colors” and “characters”. The first has variants “rgb” and “cmyk”, and the second has variants “letters” and “numbers”. The images are stored in a hierarchical directory structure, but the example could support a single-level structure with minimal modifications. Each variant directory contains a distinct number of images.

When the experiment runs, it first discovers all available images and stores their paths in variable image_paths. It then constructs a list of active images based on the value of variable active_image_variants. This value must be a list, each element of which is a list containing a category name followed by a variant name. The idea is that you would set this via MWClient’s variables window before running the experiment. For example, to use only letters and RGB colors, you would set active_image_variants to

[['characters', 'letters'], ['colors', 'rgb']]

Finally, the experiment displays each image from the chosen variants in random order.

To add additional variants, all you need to do is put the images in a new subdirectory, then add the name of the subdirectory to the appropriate category list in variable image_variants.

If you have questions or run in to any issues, please let me know.

Cheers,
Chris

image_variants.zip (87.8 KB)

7

Hello Chris,

Thank you very much for putting together the example experiment and for your suggestions on how to organize and select the stimulus variants. This is very helpful, especially as we expand the number of stimulus variants and need a more flexible way to select different subsets across sessions.

We’ll work on implementing this structure into our task and adapting the example to our stimulus organization. Once we’ve incorporated it and tested everything, we’ll follow up with you and let you know how it works on our end.

Thank you again for taking the time to put this together and for your help!

Best,
Apaala

8

Hi Chris -

Just wanted to echo Apaala and say thank you!

With respect to visually distinguishing stimulus X/Y categories in the Eye Monitor window (e.g. with a different color or shape) - I realize this isnt possible at the moment, but it does seem to be increasingly important for efficiently training animals on tasks involving making decisions about categories of things.

Could you kindly expand on what it would take to make that happen?

Thank you,
Yasmine

9

Hi Yasmine,

With respect to visually distinguishing stimulus X/Y categories in the Eye Monitor window (e.g. with a different color or shape) - I realize this isnt possible at the moment, but it does seem to be increasingly important for efficiently training animals on tasks involving making decisions about categories of things.

Could you kindly expand on what it would take to make that happen?

Yes, adding something like this has been on my to-do list for a while now.

Here’s an idea: We add a parameter to all stimuli called schematic_color. (I’m open to other suggestions for the name.) This wouldn’t affect how the stimulus is presented on the stimulus display, but the Eye Window would look for it and, if found, use it as the color of the stimulus’ outline.

I like this, because you would be specifying the schematic color in your experiment code, where it has meaning and you can easily distinguish between the different categories of stimuli. What do think?

Cheers,
Chris

10

Hi Chris -

Yes - I really like your idea of adding a parameter for “schematic_color”!

Just two questions and a suggestion please:

(1) Will schematic_color be defined as an RGB value?

(2) Will there be any limit to the number of potential schematic_colors we could use in a given experiment? Currently, we would need a minimum of two (for category A and B) - but I can imagine needing to expand that down the line.

Suggestion: Would it be easy to also specify line thickness for the stimulus drawn in the Eye Window? That way, we could also represent the “correct” stimulus on each trial so that it is easily visualized. If I’m not mistaken, the way we get around this currently is by drawing an extra invisible stimulus on top (to achieve a double circle/square representation)

Much appreciated!
Yasmine

11

Hi Yasmine,

Will schematic_color be defined as an RGB value?

Yes, just like regular stimulus colors.

Will there be any limit to the number of potential schematic_colors we could use in a given experiment?

Since they would be given as RGB values, you could use as many different colors as your graphics hardware and display support (i.e. millions).

Would it be easy to also specify line thickness for the stimulus drawn in the Eye Window? That way, we could also represent the “correct” stimulus on each trial so that it is easily visualized.

If we want to add that, too, then maybe what we need is a more general schematic parameter that takes a dictionary specifying multiple attributes. For example:

image_file image (
    schematic = {
        'color': [1, 0.5, 0.2],
        'line_width': 2
        }
    ...
    )

That way, we could expand the options in the future without the need to add more stimulus parameters. I can imagine other potential schematic attributes, e.g. “fill” to say whether the schematic should be an outline or a filled shape, “visible” to control whether the schematic is drawn at all.

How does that sound?

Chris

12

Hi Chris -

Indeed - this more general approach sounds even better (more future proof)!

Thank you for making this happen!
Yasmine