# Extracting #stimDisplayUpdate events **URL:** \ **Category:** Support\ **Created:** [August 28, 2024, 6:15pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027 "2024-08-28T18:15:14Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [August 28, 2024, 6:15pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/1 "2024-08-28T18:15:14Z")
Hi Chris, So trying to get around this, I’m still not able to find the image filename or its hash. Below is a piece of code where I try to filter an “stimulus\_presneted” event (code 71) and get its associated ‘#stimDisplayUpdate’ (code 7) which should reflect the data I’m looking for. Kindly advise. Here is the [mwk2 file](https://www.dropbox.com/scl/fi/tpj3j0w5ggs90cr8ob0ut/apollo_normalizersV3_240809_121750.mwk2?rlkey=re5ivuj5s72b2vmr4rivrg06b&dl=0) just in case. Thank you! Guy ![image](https://canada1.discourse-cdn.com/flex031/uploads/mworks/original/1X/a0b18a53698d72f9ce23c1e5f0404ee0df380104.png) ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 28, 2024, 6:23pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/2 "2024-08-28T18:23:09Z")
---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 28, 2024, 6:40pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/3 "2024-08-28T18:40:00Z")
---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 28, 2024, 6:47pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/4 "2024-08-28T18:47:32Z")
Hi Guy, To extract all the events for `stimulus_presented` and `#stimDisplayUpdate`: ```auto with MWKFile(filename) as fp: sp_events = fp.get_events(codes=['stimulus_presented']) sdu_events = fp.get_events(codes=['#stimDisplayUpdate']) ``` You now have two sequences of `Event` objects, each of which has properties `code`, `time`, and `data`: ```auto >>> evt = sp_events[0] >>> evt.code, evt.time, evt.data (71, 3375706433, -1) ``` You’ll need to use the `time` property to associate `stimulus_presented` and `#stimDisplayUpdate` events. Note that the times will _not_ be equal. Instead, for a `stimulus_presented` event with time _t_, you probably want the `#stimDisplayUpdate` event whose time is closest to _t_. You might want to talk with Yoon or someone else with experience analyzing MWorks event files, as they may have better suggestions. Once you find the `#stimDisplayUpdate` events you want, you can get the image filename and hash from the `data` property: ```auto >>> evt = sdu_events[100] >>> img_params = [d for d in evt.data if d.get('type') == 'image'] >>> for info in img_params: ... print(info['filename']) ... print(info['file_hash']) ... /var/folders/_6/d6xsx0bj6vggxrmhsg_zxvrh0000gn/T/MWorks/Experiment Cache/_Users_rig1_Desktop_Ani_RSVP-normalizers-v3_normalizersV3.mwel/tmp/images/NormalizerV3_15.png 2e94f82e721a32eab22f10bdbd78eba3fd4052e1 ``` Chris ---
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [August 28, 2024, 7:00pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/5 "2024-08-28T19:00:23Z")
Helpful, thanks! _you probably want the `#stimDisplayUpdate` event whose time is closest to t ._ Can I trust that it would _robustly_ be just the following (preceding) `#stimDisplayUpdate` event? Guy ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 28, 2024, 7:18pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/6 "2024-08-28T19:18:24Z")
> Can I trust that it would _robustly_ be just the following (preceding) `#stimDisplayUpdate` event? No, not really. Stimulus display updates happen on their own thread, on their own schedule. If the assignment to `stimulus_presented` happens _before_ the [update\_stimulus\_display](https://mworks.github.io/documentation/latest/components/update_stimulus_display.html) (aka `update_display`) action, then the corresponding `#stimDisplayUpdate` event will definitely happen later. However, if the assignment happens after `update_display`, then the `#stimDisplayUpdate` event will _still_ almost certainly be later. If you have a way to convert a `stimulus_presented` value in to an image filename, then you could find the corresponding SDU event by matching the filename (or rather, matching the end of the filename). For example, if a `stimulus_presented` value of 15 corresponds to file `NormalizerV3_15.png`, then that could be a simpler way to find the SDU events you want. Chris ---
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [August 28, 2024, 7:50pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/7 "2024-08-28T19:50:47Z")
Wouldn’t this approach amount to fooling ourselves? I could just as well implement such a check using the stimulus\_presented value alone on the user end. If we cannot make a _robust association_ between stimulus\_presented and the hash then there is no add to implementing hash based checks at all, no? ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 28, 2024, 9:41pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/8 "2024-08-28T21:41:30Z")
> Wouldn’t this approach amount to fooling ourselves? > I could just as well implement such a check using the stimulus\_presented value alone on the user end. Yes, of course you’re right. Dumb suggestion on my part. But why are you trying to associate `#stimDisplayUpdate` with `stimulus_presented`? If you want determine which images were presented and when, all that information is recorded, _robustly_, in `#stimDisplayUpdate` alone. Why look at `stimulus_presented` at all? Chris ---
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [August 29, 2024, 2:41pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/9 "2024-08-29T14:41:32Z")
Looking at our preprocessing code, it appears that the `stimulus_presented` was traditionally used as the field to lock onto. Isn’t the `#stimDisplayUpdate` just a GUI event whose timing may not accurately reflect the actual stimulus presentation for neural recordings processing purposes? And that the `stimulus_presented` is the precise backend event for those purposes? ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 29, 2024, 3:05pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/10 "2024-08-29T15:05:00Z")
You have that exactly backward 🙂. The time stamp on a `#stimDisplayUpdate` event is the _most accurate_ estimate MWorks has of when the physical display actually updates. It has to be an estimate, because the actual update happens _after_ the SDU event is generated. For more info, see [Understanding Display Updates](https://mworks.github.io/documentation/latest/guide/designing.html#understanding-display-updates) in the [MWorks manual](https://mworks.github.io/documentation/latest/). On the other hand, `stimulus_presented` gets set when the experiment author chooses to set it. Probably the experiment will set it immediately before or after the [update\_display](https://mworks.github.io/documentation/latest/components/update_stimulus_display.html) action, but as the docs note, that is _not_ when the display update takes place. Hence, the time stamp on `stimulus_presented` events is going to be arbitrary and unreliable for determining when a stimulus presentation actually happens. If you want to know when the display really does update (as opposed to when it’s _predicted_ to update), you have to measure it physically with a photodiode. The photodiode signal would typically be fed in to the neural recording system. In the MWK2 file you shared with me, I see that the `#stimDisplayUpdate` events include a stimulus named “photodiode\_image”, which is presumably positioned under the photodiode on the display. I don’t know if the photodiode signal gets used in the data analysis pipeline, but I assume it’s stored and available somewhere. Cheers, Chris ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 29, 2024, 3:20pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/11 "2024-08-29T15:20:09Z")
Also, in case it wasn’t clear already, I have never seen any of the DiCarlo Lab’s data processing code, so I’m only making guesses about what it does. My job is developing and maintaining MWorks, so I’m just trying to help you understand what MWorks does. That’s why I recommended checking with Yoon or someone else who _is_ familiar with the lab’s post-MWorks data processing. Chris ---
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [August 29, 2024, 3:20pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/12 "2024-08-29T15:20:46Z")
Thank you! Photodiode signal is used and is part of the preprocessing pipeline. I’m not certain if in a way that remedies inaccuracies of time reports in `stimulus_presented` or that this error propagates and corrupts our PSTHs (btw do you have order of magnitude of the time mismatch between `stimulus_presented` and `#stimDisplayUpdate`?). In any case, am I safe to completely revise the pipeline such that `stimulus_presented` is discarded and just using `#stimDisplayUpdate` time reports and info instead? ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [August 29, 2024, 3:37pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/13 "2024-08-29T15:37:33Z")
> Photodiode signal is used and is part of the preprocessing pipeline. I’m not certain if in a way that remedies inaccuracies of time reports in `stimulus_presented` or that this error propagates and corrupts our PSTHs I assume it’s used to get accurate stimulus-presentation times, but I don’t know. > btw do you have order of magnitude of the time mismatch between `stimulus_presented` and `#stimDisplayUpdate`? If `stimulus_presented` is set immediately after `update_display`, then the actual display update should normally happen within one display refresh cycle (so about 17ms for a 60Hz display). It _may_ actually happen several refresh cycles later (probably a maximum of 3), if the graphics hardware is well ahead in the drawing loop. > In any case, am I safe to completely revise the pipeline such that `stimulus_presented` is discarded and just using `#stimDisplayUpdate` time reports and info instead? I’d say so, unless you want to hold on to `stimulus_presented` just for the additional redundancy. Chris ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [September 5, 2024, 1:51pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/14 "2024-09-05T13:51:42Z")
Hi Guy, I took a look at the code you shared (`spike_tools/mwk_ultra.py`). It looks like the assignment of times to each stimulus presentation is handled well. Both `stimulus_presented_df['time']` and `correct_fixation_df['time']` are set to the value of `photodiode_on`. This variable contains the times (on the _Intan clock_) of the peaks in the photodiode signal, which is the most robust measure of actual stimulus presentation time. Also, most of the usage of `stimulus_presented` is fine, since it’s just determining a general ordering of stimulus presentations (i.e. the order of a given stimulus in its trial) and isn’t concerned with specific stimulus identity or the real-world time of the presentation. To get the relevant `#stimDisplayUpdate` (SDU) events for specific stimulus identification, you need to extract events where an image was presented. Here’s some Python code to do this: ```auto with MWKFile(filename) as fp: sdu_events = [] file_paths = [] file_hashes = [] for e in fp.get_events_iter(codes=['#stimDisplayUpdate']): for d in e.data: if d.get('type') == 'image': sdu_events.append(e) file_paths.append(d['filename']) file_hashes.append(d['file_hash']) break ``` The number of SDU events extracted this way should equal the number of `stimulus_presented` events whose value is not -1. More specifically, borrowing some code from the section “Extract stimulus presentation order and fixation information”: ```auto stimulus_presented_df = data[data.name == 'stimulus_presented'].reset_index(drop=True) correct_fixation_df = data[data.name == 'correct_fixation'].reset_index(drop=True) stimulus_presented_df = stimulus_presented_df[:len(correct_fixation_df)] # If you have one extra stimulus event but not fixation, use this assert len(stimulus_presented_df) == len(correct_fixation_df) # Drop `empty` data (i.e. -1) before the experiment actually began and after it had already ended correct_fixation_df = correct_fixation_df[stimulus_presented_df.data != -1].reset_index(drop=True) stimulus_presented_df = stimulus_presented_df[stimulus_presented_df.data != -1].reset_index(drop=True) sdu_events = sdu_events[:len(correct_fixation_df)] assert len(sdu_events) == len(stimulus_presented_df) ``` One area where I think it would be better to use SDU instead of `stimulus_presented` is in the “Get eye data” section. Since the point there is to find the eye data coincident with the stimulus presentation, SDU’s more accurate timestamp makes it the better choice. Finally, in the “Save output” section, I think it would be easiest to save `file_paths` and `file_hashes`, rather than the raw SDU events. There’s really no reason to save `stimulus_presented`, except perhaps as a redundant means of image identification. I hope that helps! Cheers, Chris ---
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [September 6, 2024, 5:25pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/15 "2024-09-06T17:25:10Z")
Very helpful Chris! Thanks for looking into this. 1. So I can just create an `sdu_df` as you suggested as a substitute of the `stimulus_presented_df` everywhere in the code seamlessly? 2. As to eye data, we don’t seem to use this in practice anyways (its flag is set to `0`). Guy ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [September 9, 2024, 2:14pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/16 "2024-09-09T14:14:07Z")
Hi Guy, > So I can just create an `sdu_df` as you suggested as a substitute of the `stimulus_presented_df` everywhere in the code seamlessly? No, I would keep using `stimulus_presented_df` where it’s currently used (except in the “Get eye data” section, as I mentioned). For lining things up with other MWorks variables (e.g. `trial_start_line`, `correct_fixation`), it’s a better choice than SDU. What `stimulus_presented` is _not_ good for is establishing stimulus identity (use SDU for that) or stimulus presentation time (use the photodiode peaks for that). But certainly do extract the file paths and file hashes from SDU and include those in the output. Chris ---
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [September 9, 2024, 4:17pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/17 "2024-09-09T16:17:04Z")
OK, great. I think I have successfully implemented this. Just for clarification, what do you mean by _“For lining things up with other MWorks variables (e.g. `trial_start_line` , `correct_fixation` ), it’s a better choice than SDU”_. If I got this correctly, once timing is in any case overridden by photodiode data, then the SDU and `stimulus_presented` whose value is not -1 series of events are _completely equivalent_ and I am safe to use any one of them arbitrarily (maybe up to eye data as you suggested), isn’t it so? This held up in a specific .mwk2 analysis I made. ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [September 10, 2024, 12:59pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/18 "2024-09-10T12:59:58Z")
> Just for clarification, what do you mean by _“For lining things up with other MWorks variables (e.g. `trial_start_line` , `correct_fixation` ), it’s a better choice than SDU”_ . The issue here is one I mentioned previously: Stimulus display updates (and their associated SDU events) happen on a separate thread from the main experiment logic. This means that you don’t know precisely when the SDU event associated with a particular display update is going to occur relative to variable assignments performed in the experiment code. For example, suppose you have four MWorks variables, `a`, `b`, `c`, and `d`, and your experiment includes the following code: ```auto a = 1 b = 2 update_display () c = 3 d = 4 ``` Because the four variable assignments are made on the same thread (the one executing the experiment logic), you know that, in the event file, the events corresponding to those assignments are going to appear in exactly the order given in the MWEL code (1, 2, 3, 4). However, although the display update is triggered between the assignments to `b` and `c`, the corresponding SDU event is almost certainly _not_ going to fall between those assignments in the event file; more likely, it will happen after all four assignments. If you’re using event order to try to associate SDU events with other events in the trial – say, looking for the SDU event that lies between a particular pair of `b` and `c` events – you’re going to get the wrong answer. Although what I said above is true in general, it may also be true that, in the case of `mwk_ultra.py` and the experiments it’s used with, using SDU instead of `stimulus_presented` would work correctly. Looking through the code, I think that _may_ be the case. However, I’m not 100% sure, which is why I recommended mostly keeping things as they are. Chris ---
**Author:** ![ggaziv](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@ggaziv](https://mworks.discourse.group/u/ggaziv)\ **Post date:** [September 10, 2024, 3:34pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/19 "2024-09-10T15:34:12Z")
Amazing! Thank you for the detailed walk through! Guy ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [October 28, 2024, 4:26pm UTC](https://mworks.discourse.group/t/extracting-stimdisplayupdate-events/1027/20 "2024-10-28T16:26:24Z")