In response to a problem reported by Najib (namely, MWServer crashing when trying to load an experiment with a large number of image stimuli), I’ve been investigating MWorks’ limitations with respect to image loading. Here’s a summary of what I’ve found.
First problem: Scarab send buffer limit
Although Najib didn’t have this problem, the first potential limitation is in the Scarab networking code.
When loading an experiment, MWorks sends all required files, including images, over a TCP connection from the client to the server. Internally, Scarab buffers all the data before sending. The buffer size is fixed at 100,000,000 bytes (95.4 MB). If the total on-disk size of all the files in an experiment exceeds this limit, then experiment loading will fail. (Unhelpfully, no error will be reported, and the client will just hang forever in the loading phase.)
As a concrete example, I wrote an experiment that loads a set of 969×684 PNG images, each 789 KB on disk. If I try to use 123 such images (94.8 MB total), the experiment loads fine. If I try to use 124 images (95.6 MB total), then the client hangs indefinitely at “Loading Experiment…”.
This issue was originally discovered by Elias and still needs to be addressed.
Second problem: memory limits (RAM and VRAM)
Assuming you get past the first problem, the next potential issue is running out of memory.
When an image is loaded, it’s first converted into RGBA format, with each pixel stored as a 32-bit float (so 16 bytes total per pixel). The image is then converted into a mipmapped OpenGL texture. The memory used by each texture isn’t exactly 16 bytes per pixel, but you can use that as a rough guide. Based on some experimentation, it seems that the textures are initially stored in main memory (RAM) and are transferred to video memory (VRAM) when first displayed.
To get some real figures, I ran an experiment that displays 100 of the previously-described 969×684 images. The maximum amount of RAM used by MWServer was 3.95 GB (out of 8GB total). The main and mirror MWorks displays ran off separate video cards, using about 550 MB of VRAM (out of 1GB total) on each. Although this experiment did not exceed the limits of RAM and VRAM, it would be easy to construct one that did (say, by loading 300 images instead of 100).
Also, in the test quoted above, the server ran as a 64-bit process. When running as a 32-bit process, the server crashes while loading the experiment due to a memory allocation failure, roughly around the time it hits 2GB of virtual memory. Unfortunately, users of the ITC-18 are required to run MWServer in 32-bit mode, as the ITC-18 drivers are available only as 32-bit binaries.
Potential solutions
The Scarab buffer problem is an implementation issue that doesn’t reflect any fundamental limit. It can and should be addressed.
On the other hand, memory limits are fundamental. By running MWServer in 64-bit mode and optimizing MWorks’ image loading code, we can (potentially) increase the maximum number of images that can be loaded at once. However, we should also give users more explicit control over the loading and unloading of image data, so that they can optimize memory usage for their experiment.
Probably the right way to do this is to add “Load Stimulus” and “Unload Stimulus” actions. Combined with deferred loading (already supported), this would allow users to load and unload images as needed.