The timing between turning laser on and stimulus on

Hi Chris,

Thank you for the Nightly Built. It solved the conflicting timing of two event codes being sent at the same time.

Now we’ve encountered a similar problem with turn_laser_on() and update_display(). This will be our priority for next week since we will be recording very soon. We would appreciate it if you could help us with this.

For this particular protocol, we would like to have the laser on while presenting a stimulus on the screen. Ideally, we would like the laser to be turned on and off at the same time as the stimulus (and laser_on_duration is the same as the stimulus_on_duration which is 300ms). Since both turning on the laser and the photodiode takes time, we tried two ways of positioning turn_laser_on() and update_display() in our protocol as following to see if laser duration can cover the entire stimulus presentation time:

  1. turn_laser_on() before photodiode_pixel_on() and update_display()
report('Laser On’)
turn_laser_on(laser_on_duration)

report('Stim On’)
queue_stimulus(target_stimulus[stimulus_type])
photodiode_pixel_on()
update_display()
vpixx_send_blackrock_sync_word(e_stimulus_on) //Stim on
vpixx_send_blackrock_sync_word(e_photodiode_on)

It turned out that the laser was turned on and off about 30ms before the photodiode was on and off. But both were indeed turned on for 300ms. I’ve attached a matlab figure if it helps. The blue trace is the laser detection trace, the pink trace is photodiode, the blue dot is the e_laser_on, and the pink dot is the e_stimulus_on.

  1. photodiode_pixel_on() and update_display()before turn_laser_on()

We recalled that there is an about 16 to 17ms delay between e_stimulus_on and photodiode on (due to the refresh rate of the monitor). Therefore, we tried to draw the stimulus first before turning the laser on, thinking that the time it took the laser to be turned on would overlap with this 16 to 17ms delay to make the two events happening closer in time.

report('Stim On’)
queue_stimulus(target_stimulus[stimulus_type])
photodiode_pixel_on()
update_display()

turn_laser_on(laser_on_duration)
report('Laser On’)

vpixx_send_blackrock_sync_word(e_stimulus_on) //Stim on
vpixx_send_blackrock_sync_word(e_photodiode_on)

However, the stimulus was still presented before the laser was turned on. There was an about 45ms difference between photodiode on and laser on. The laser was on for 300ms but the stimulus was on for over 300ms even though the stimulus_on_duration was 300ms. It seems like it was waiting for the stimulus drawing to be completed before turning the laser on. I’ve also attached a figure of what the traces look like this time.

We ran a few controls to troubleshoot this problem. We silenced either the turn_laser_on() or the update_display() in the protocol. When there was only turn_laser_on() command in a protocol, the e_laser_on and actual laser onset took place with a time difference of just around 2ms. When there was only update_display() and no turn_laser_on() command in a protocol, there is an about 16 to 17ms delay between e_stimulus_on and photodiode on time. The two commands alone worked as expected but there seems to be a problem when both commands are cued at the same time. In order to make sure that the laser covers the stimulus presentation, we tried to make the laser on duration 30ms longer than the stimulus on duration. It is not ideal but it solved the problem for now. Is this the only solution?

I’ve also attached the config_Laser file and a protocol where we cued both a stimulus and turn_laser_on() in the “Stimulus On” state.

Thank you again for helping!

Best,
Taylor

laserAmpStim_laserThenStim.fig (388 KB)

laserAmpStim_stimThenLaser.fig (369 KB)

config_Laser.mwel (5.28 KB)

protocol_LaserAmplitude_wStim.mwel (16.9 KB)

Hi Taylor,

The issue of synchronizing the laser with a stimulus presentation has come up before. Please see this explanation, which is still 100% valid. Basically, you should use approach #2 (update_display before turn_laser_on), but replace update_display with update_display_and_wait.

There was an about 45ms difference between photodiode on and laser on. The laser was on for 300ms but the stimulus was on for over 300ms even though the stimulus_on_duration was 300ms. It seems like it was waiting for the stimulus drawing to be completed before turning the laser on. I’ve also attached a figure of what the traces look like this time.

This is puzzling. Can you send me the definition of the photodiode_pixel_on macro? Actually, it’d probably be helpful to have the whole protocol that produced this result. Can you send that as well?

Thanks,
Chris

Hi Chris,

We just tried to implement the update_display_and_wait () as following and I’ve attached a .fig of the result of this implementation.

report(‘Stim On’)
vpixx_send_blackrock_sync_word(e_stimulus_on) //Stim on
vpixx_send_blackrock_sync_word(e_photodiode_on)
queue_stimulus(target_stimulus[stimulus_type])
photodiode_pixel_on()
update_display_and_wait ()

turn_laser_on(laser_on_duration)
report(‘Laser On’)

updateDisplayWait_TEST.fig (373 KB)

config_Photodiode.mwel (1.36 KB)

protocol_LaserAmplitudeWithStim.mwel (16.6 KB)

Hi Taylor,

Can you send me the protocol you ran to create updateDisplayWait_TEST.fig? The protocol you sent turns the laser on before showing the stimulus.

Thanks,
Chris

Hi Chirs,

Just one more point that I forgot to make clear. We would like the laser to be on and off before and after the stimulus is on and off (i.e. laser on, stimulus on … stimulus off, laser off). In order to make sure that the laser is on for the entire stimulus presentation time, it is ok for the laser to be turned on a bit before the stimulus is displayed and turned off a bit after the stimulus is cleared. The timing of the event codes is less of a concern at the moment but the physical on and off of the two devices is more of a focus for now. Our current hack in mind is to make the laser on duration 30ms longer than the stimulus on duration but ideally, we don’t want to stimulate the brain for longer than it needs to be. Therefore, we are seeking for any better way of doing this. I’ve attached a drawing of what we would like the timing between laser and stimulus to be. Hopefully, this makes more sense! I’ve also attached a .jpg of the .fig in case you have trouble viewing the .fig. Thank you again!

Best,
Taylor

Hi Chris,

Yes, please see the following attachment.

protocol_LaserAmplitudeWithStim_updateDisplayWait.mwel (16.8 KB)

Thanks.

I am still very puzzled by this. As you said, it seems like update_display must be waiting for too long, but I’m having a hard time thinking of reasons why.

Is it possible that, in addition to the invocation of update_display in state “Stimulus On”, another piece of your experiment is calling update_display around the same time? For example, is there a variable that has update_display in its attached actions that is being assigned somewhere?

I also have to wonder if this could be caused by the changes I made to prevent the event code sends from overlapping. If you want to test that hypothesis, you could temporarily revert to MWorks 0.12.2 and re-run your tests.

Let me think about this some more and get back to you.

Thanks,
Chris

Hi Chris,

We tested on the rig with the old MWorks version and the result was still the same - the stimulus was on before the laser and was off after the laser. The laser was on for 300ms but the stimulus was on for longer than that. The timing between the event codes and the actual event is different between two MWorks versions but not sure if that’s what caused the issue. I also don’t think we have two update_display() sending commands at the same time in the protocol. I’ve attached plots for this test as well.


Best,
Taylor

Hi Taylor,

Thanks for running that test. It looks like we can rule out the changes I made to prevent event code overlap as the cause of this issue.

I’ve attached a modified version of your “updateDisplayWait” protocol. I added some code to measure and report the time taken by each of the relevant steps in state “Stimulus On”. When run, the protocol will print a bunch of messages to the server console with the prefix "CJS: ". If possible, can you run this on your rig, then copy those messages from the server console and send them to me? If you can also re-generate your timing graphs for that run, that would be helpful, too.

Thanks,
Chris
protocol_LaserAmplitudeWithStim_updateDisplayWait_CJS_DEBUG.mwel (17.9 KB)

Hi Chris,

Please see the following attachment for the message in .txt and the plots in both .fig and .jpg. It seems like the laser is taking longer than we expected to turn on?

CJS_debug.txt (24.9 KB)

CJS_debug_plot.fig (365 KB)

Hi Taylor,

It seems like the laser is taking longer than we expected to turn on?

Yes, indeed! That is not what I expected. But, looking back through some QCUALOR-related emails, I found the following in a message from Yasmine to Karim on 1/24/2023:

We are measuring ~60 ms delay between the MWorks commands for initializing the laser and the actual initiation of a sinusoid (as measured by the optical sensor output). That seems awfully long. Any idea what we expect the minimum delay for this type of serial communication to be?

Yasmine later said that the issue seemed to be with a laser firmware update, but that’s all I heard about it.

In any case, this is good news: Assuming that the unexpected delay is happening while configuring the laser (which it almost certainly is), we can easily work around the problem by splitting up the configuration and activation steps. Currently, the definition of variable laser_turning_on (in config_Laser.mwel) looks like this:

var laser_turning_on = 0 {
    if (laser_turning_on == 1) {
        start_io_device (laser)
        vpixx_dout_laser_gate = true
        vpixx_send_blackrock_sync_word(e_laser_on)
    }
    if (laser_turning_on == 0) {
        vpixx_dout_laser_gate = false
        vpixx_send_blackrock_sync_word(e_laser_off)
        stop_io_device (laser)
    }
}

To eliminate the laser configuration delay, you should take the start_io_device action out of here and move it to some non-timing-critical portion of the protocol. You have to execute it after you set all the laser parameters, but apart from that, it can go anywhere, because the laser won’t activate until you set vpixx_dout_laser_gate to true. Looking at your protocol, it seems like the right place is probably earlier in state “Stimulus On”, right after you’ve set amplitude and laser_type. (You’ll also need to add start_io_device in the else, to handle the case where you don’t show a stimulus.)

Since stop_io_device is invoked asynchronously, you can leave it where it is, as it can’t block anything if it’s slow.

Maybe you can make this change and see if it eliminates the unexpected delay?

Thanks,
Chris

Hi Chris,

We also found this email and we were not able to recall the solution either… but thank you for doing the search.

We just tried your solution of moving start_io_device (laser) earlier and it did reduce the delay. The laser was turned on before the stimulus was on but still ended a bit earlier than the stimulus off. But both the laser and the stimulus were on for the intended 300ms. We will think about our hack of making the laser_on_duration slightly longer than the stimulus_on_duration and will get back to you!

Thank you again for helping us troubleshooting this urgent task!

Best,
Taylor

Hi Chris,

Just to double check a question I forgot to ask in the previous email. Since the configuration of the laser is now in the non-timing-critical portion of the protocol, the time it takes the laser to configure (~60ms) shouldn’t make the length of the trials with laser on 60ms longer than the length of the trials without laser, correct?

Best,
Taylor

Hi Taylor,

We just tried your solution of moving start_io_device (laser) earlier and it did reduce the delay. The laser was turned on before the stimulus was on but still ended a bit earlier than the stimulus off. But both the laser and the stimulus were on for the intended 300ms.

It’s going to be hard to align the laser and stimulus timing precisely if you use pulse to control the laser (as you do in macro turn_laser_on). It’d be much easier if you performed laser on/off synchronously, in states “Stimulus On” and “Stim success”. Here’s the gist of what I’m suggesting in MWEL:

state 'Stimulus On' {
    // Already did start_io_device(laser) earlier in the trial

    queue_stimulus(target_stimulus[stimulus_type])
    photodiode_pixel_on() 
    update_display (predicted_output_time = predicted_display_update_time)

    // laser_padding_time is the desired interval between laser on and
    // stimulus on -- maybe a few milliseconds?
    wait (predicted_display_update_time - laser_padding_time - now())

    vpixx_dout_laser_gate = true
    vpixx_send_blackrock_sync_word(e_laser_on)

    wait (predicted_display_update_time - now())

    // We've reached the point where we expect the stimulus to actually be
    // on, so start the timer and send e_stimulus_on here
    start_timer (timer = timer2; duration = stimulus_on_duration)
    vpixx_send_blackrock_sync_word(e_stimulus_on)
    vpixx_send_blackrock_sync_word(e_photodiode_on)

    goto (target = 'Stim success'; when = timer_expired(timer2))
    goto (target = 'Broke Central Fixation'; when = not eye_in_fixation)
}

state 'Stim success' {
    dequeue_stimulus(target_stimulus[stimulus_type])
    photodiode_pixel_off() 
    update_display (predicted_output_time = predicted_display_update_time)

    wait (predicted_display_update_time - now())

    // The stimuli should be off now
    vpixx_send_blackrock_sync_word(e_stimulus_off)
    vpixx_send_blackrock_sync_word(e_photodiode_off)

    wait (laser_padding_time)

    vpixx_dout_laser_gate = false
    vpixx_send_blackrock_sync_word(e_laser_off)
    stop_io_device (laser)

    //...
}

State “Broke Central Fixation” already has code for turning the laser off, so you don’t need to add it there. Does this make sense?

Since the configuration of the laser is now in the non-timing-critical portion of the protocol, the time it takes the laser to configure (~60ms) shouldn’t make the length of the trials with laser on 60ms longer than the length of the trials without laser, correct?

If, in the “laser off” trials, you didn’t configure the laser (i.e you didn’t call start_io_device), then those trials would be ~60ms shorter than trials with the laser on. But looking at your code, I see that even when laser_type is set to 0 (“off”), you still configure and activate the laser as usual – it’s just that in this case, “active” means all channels off. This means that you’re still doing the same amount of laser configuration, so the trial length should be the same.

That said, it still might be wise to move start_io_device completely out of any timed portion of the trial – perhaps all the way back in “Begin Trial”? This would protect against any variability in the time it takes to configure the laser. As I noted previously, you have to set all laser parameters before calling start_io_device, so you’d need to move that code back, too.

Cheers,
Chris

Hi Chris,

Got it! The wait() function might not be the best solution for our task but moving the configuration of the laser to the beginning seems to work for now. Configuring the laser for all trials also seems to help with making sure all trials have the same length. We will keep exploring the possible solutions to cover this time difference between the stimulus and laser. However, we are still wondering if you have any intuition of why the configuration of the laser takes this long? Could the problem lie in the software of the laser or could it be the communication between MWork, laser, and Vpixx?

Best,
Taylor

Hi Taylor,

However, we are still wondering if you have any intuition of why the configuration of the laser takes this long? Could the problem lie in the software of the laser or could it be the communication between MWork, laser, and Vpixx?

The VIEWPixx isn’t involved in configuration of the laser.

All MWorks does is send commands to and receive responses from the laser, as quickly as it can. I guess the delay could be happening at the OS level, but I think that’s unlikely.

My guess is that the problem lies with the laser. I suppose it could be a firmware or hardware issue, but I wouldn’t know.

Chris

Hi Chris,

We just zoomed with Karim and he confirmed that the problem shouldn’t be the laser.

If it is caused by the configuration of the laser, does it mean the delay is due to the serial communication from MWork to laser via USB?

Best,
Taylor