Hi Chris,
As I’m beginning to do physiology, I put a photodiode on the screen to measure delays. The photodiode detects the start of each trial and is recorded by MWorks through nidaq → nidaq_analog_input_voltage.
To my surprise, after checking the signals, I found that the delay measured by this MWorks → screen → photodiode → NIDAQ → MWorks loop increases linearly throughout the session, starting at 30ms and ending around 250ms after 2 hours.
I did a bunch of manipulations (won’t bore you with the details) to figure out where this delay increase was coming from, and ultimately narrowed it to the nidaq object in MWorks. In fact, if in MWorks I stop/start the nidaq io device in the inter-trial interval of each trial, this delay aggregation goes away.
This is a concerning, because there are things (such as reward) coming from that nidaq instance that are time-sensitive, and aggregating delays over the course of a session would make trial averaging impossible for data analysis. Also, I know that several others in our lab also don’t stop/start the nidaq io on each trial and were unaware of this issue.
Given this issue would impact experimental results and is not obviously detectable (for the past several months of animal training I had no idea it was happening), it’s probably worth doing something about. Perhaps fixing whatever is causing this, or aborting the code if nidaq is un-stopped for too long. But I’ll leave it to you to decide what to do!
Best,
Nick
Hi Nick,
I think I understand what’s going on here:
The NI-DAQ API doesn’t provide any time stamps for input data. When posting a value change for a digital input, MWorks just assigns it the time (on the MWorks clock) at which it received the value from the device. For analog inputs, we try to be a bit more clever: First, we record the time (on the MWorks clock) at which sample acquisition started. Then, as samples come in, we use the sampling rate and the total number of samples acquired to extrapolate a time stamp for each sample. As I recall, this was the method recommended by the NI folks. If you want to see all the details, check out this code.
The increasing delay you observe sounds like clock drift: The DAQ’s sample clock and MWorks’ clock are running at slightly different rates, and MWorks’ extrapolated time stamps for analog input samples are getting progressively less accurate. I know this is a general issue when comparing values between different clocks, and some hardware manufacturers recommend ways to compensate for it. I don’t recall any NI documentation or support folks mentioning it, though, and obviously I didn’t think of it myself while implementing this code, which is my mistake. Interesting that this hasn’t come up until now.
MWorks could guard against this issue by regularly checking that the cumulative number of samples it has received from the DAQ matches what it expects, within some tolerance, and issuing a warning or error message if it doesn’t. However, due to NI abandoning its macOS drivers, the recent 0.11 release is the last version of MWorks that will support NI devices. The nightly build is going to remove support for them entirely any day now, and there isn’t much sense in changing the code before then.
As you’ve discovered, the workaround for this issue is to regularly stop and restart the device. That will make MWorks record a new “time zero” from which to extrapolate times for analog input samples.
My apologies for any problems this has caused you. I’m happy to discuss this issue further, if you like.
Cheers,
Chris
Hi Chris,
Aha, I see, thanks for this detailed explanation!
I think you are right and I incorrectly figured MWorks’ nidaq object is the issue — sounds like the clock on my computer running MWorks is fast. This is also consistent with what I’ve observed in the logs from the physiology computers (which have similarly increasing delay relative to the MWorks timestamps).
To be 100% sure the clock drift is what’s going on, tomorrow I’ll try just passing a digital and analog trial-start signal MWorks → NIDAQ → MWorks (taking the photodiode out of the loop). I should see the digital loop have constant delay regardless of re-starting NIDAQ every trial, and the analog loop have linearly changing delay only when re-starting NIDAQ every trial.
Sorry for the false accusation! But I’m very relieved now — this means that the reward timing and everything was actually fine all along!
In light of this, there’s no issue on the MWorks side of things, so nothing needs to be changed.
Thank you very much!
– Nick