# Making movies out of tasks

**URL:** <https://mworks.discourse.group/t/making-movies-out-of-tasks/127>\
**Category:** Support\
**Created:** [April 29, 2021, 12:52pm UTC](https://mworks.discourse.group/t/making-movies-out-of-tasks/127 "2021-04-29T12:52:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [April 29, 2021, 12:52pm UTC](https://mworks.discourse.group/t/making-movies-out-of-tasks/127/1 "2021-04-29T12:52:03Z")

</div>

Hi Mahdi,

Support for capturing stimulus display frames is now in the MWorks [nightly build](https://mworks.github.io/downloads/).

You can enable frame capture by including a [Stimulus Display device](https://mworks.github.io/documentation/latest/components/stimulus_display.html) in your experiment and setting its [capture\_format](https://mworks.github.io/documentation/latest/components/stimulus_display.html#capture-format) parameter to a supported value. For example:

```auto
var capture_enabled = false

stimulus_display (
    capture_format = PNG
    capture_height_pixels = 400 // Set to zero or omit for full display resolution
    capture_enabled = capture_enabled
    )

```

In general, PNG is probably the best format, as it uses lossless compression.

If the parameter [capture\_height\_pixels](https://mworks.github.io/documentation/latest/components/stimulus_display.html#capture-height-pixels) is omitted or set to zero, frames are captured at the full resolution of the display. Otherwise, frames are scaled down to the specified height, with the width chosen to match the display’s aspect ratio.

By default, _every_ display update will be captured. However, if you set the parameter [capture\_enabled](https://mworks.github.io/documentation/latest/components/stimulus_display.html#capture-enabled) to the name of a variable, then frames will be captured only when the variable’s value is true/1. This lets you capture only the frames you care about (e.g. while a trial is active).

Each captured frame is assigned to the system variable [#stimDisplayCapture](https://mworks.github.io/documentation/latest/reference/sysvars.html#stimdisplaycapture-var). The value of the variable is binary data ([bytes](https://docs.python.org/3/library/stdtypes.html#bytes-objects) in Python, [uint8](https://www.mathworks.com/help/matlab/ref/uint8.html) array in MATLAB). The data is an image file in the chosen format (e.g. PNG). If you write the data to a file, you should be able to open it in Preview or any standard image viewer. It’s possible to preview the captured images while your experiment is running by using MWClient’s Image Viewer window with “Image data variable” set to `#stimDisplayCapture`. (Note that the image viewer wasn’t designed to be updated every frame and can be quite laggy in this role. I will attempt to make it more performant in the future.)

Once you’ve run your experiment and have an event file full of captured frames, the next step is to extract and combine them into a video, with added eye traces and other desired adornments. I will work on an example of this and try to get it to you soon. You use MATLAB, right? If so, the [VideoWriter](https://www.mathworks.com/help/matlab/ref/videowriter.html) class looks like a useful tool for this job, so I’ll probably try that first.

If you have any questions, please let me know!

Cheers,  
Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [June 2, 2021, 1:13pm UTC](https://mworks.discourse.group/t/making-movies-out-of-tasks/127/2 "2021-06-02T13:13:21Z")

</div>

Hi Mahdi,

Some updates on the task movie stuff:

First, I made a few improvements to the stimulus display frame capture mechanism itself. MWorks now limits the number of “in flight” frames, i.e. frames that have been captured but are still in the process of being converted to image files and written to the event file. This prevents MWServer from running out of memory due to an ever-growing backlog of in-flight frames. Also, I eliminated a potential bottleneck in writing the captured frames to the event file. These changes are in the current [nightly build](https://s3.amazonaws.com/mworks-downloads/nightly/MWorks-NIGHTLY.dmg).

Second, I now have an example of how to extract the captured frames from the event file and turn them in to a movie using MATLAB. As I suggested previously, the example uses [VideoWriter](https://www.mathworks.com/help/matlab/ref/videowriter.html) to create the movie. Basically, for each frame of the video, it loads and displays the appropriate stimulus display frame, renders any additional data/info using standard MATLAB plotting commands, and writes the completed figure to the video.

The attached ZIP file includes an example experiment and a MATLAB script for creating movies from event files generated by the experiment. The experiment is a very lightly-modified version of the standard [RSVPDemo](https://github.com/mworks/mworks/tree/master/examples/Examples/RSVPDemo) experiment. The movie-creation script adds the elapsed time (as text) and an eye trace over each stimulus display frame. The ZIP file also includes an example movie generated by the script.

Using the example script as a starting point, you should be able to generate movies using any data available in your event file. Since you create each frame using standard MATLAB commands, you can make use of all your existing MATLAB plotting skills (which I’m sure are much greater than mine).

I hope this is helpful. If you have any questions or need more info, please let me know!

Cheers,  
Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [June 2, 2021, 1:13pm UTC](https://mworks.discourse.group/t/making-movies-out-of-tasks/127/3 "2021-06-02T13:13:21Z")

</div>

Attachment: [task\_movie.zip](https://mworks.discourse.group/uploads/short-url/eDXB66jrxvQrmQml5cgkr08yIOS.zip) (1.78 MB)

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [June 8, 2021, 8:24pm UTC](https://mworks.discourse.group/t/making-movies-out-of-tasks/127/4 "2021-06-08T20:24:50Z")

</div>

Hi Chris!

This is great, thank you so much!! I will take a look and try to run the  
code!

I will share this with all of Jazlab, I am sure they will find this very  
helpful!

Mahdi

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [July 19, 2022, 7:06pm UTC](https://mworks.discourse.group/t/making-movies-out-of-tasks/127/5 "2022-07-19T19:06:43Z")

</div>