Draw timing

Support
61

I’m not sure a cross-correlation is necessarily the right way to look at this (particularly if there is a chance for “frame-shift” errors)

Thanks for pointing me in the right direction. I’ve redone the analysis like you suggested, the results are further down this post.

Right now, the data was obtained using only one photodiode, so the possibility of frame-shift errors remains. I’ve told our machine shop to build a diode-array and as soon as it is ready, I will do all this again. For now, I think the results are interesting enough to read on.

It seems like I was wrong all the time. The display update timestamps are not inaccurate and they truly reflect the time when something appears on the display. Something else is going wrong!

I’ve tested the previous described setup with the current nightly build and the DisplayLink-only version of MWorks. The experiment was one random dot pattern moving for 100 seconds. The distributions for the delay between display update announcements (bit code 0 to bit code 1) and photodiode rising edges look like this:

and like this for the DisplayLink-only version

The difference is small, almost neglect able. However, I did see the motion stutter when measuring the Nightly build, so I took a closer look. This is the “delta t” analysis I did previously (except that I took every second frame this time).
As one can see, delays between display update announcements are quite regular in the DisplayLink only version, but look a bit strange in the nightly build.


I noted previously, that this might be due to a “smoothing” of the timestamps performed by DisplayLink. But since the DisplayLink-only version also passed the lag-analysis, I was curious of how such a “smoothing” would take place. So I looked closer at the diode-activity in both versions and was extremely surprised to find that they are very different. All the irregularities found in the update announcements of the nightly build are also present in the diode activity.


If I am not completely mistaken, this means that the announcements for display updates are not wrong, they are quite accurate, but the screen starts to update irregularly when using the nightly build.

But is this even possible or did I miss something big here?

Confused greetings,
Philipp

One additional thing that might be or might not be of importance: Both versions have a display update announcement (the last one) which never gets displayed. This might be due to a premature dequeuing of the bit code stimulus. I’ll keep on investigating this. Apart from that update announcements match up quite obviously one-by-one to the diode activity. But as I said, as soon as the diode array is ready…

Oh and I forgot to mention it: the nightly build showed one warning for a lost frame. I also disabled the mirror window for both runs.

62

Attachments:

63

I’m having trouble orienting to what this plot is showing… what’s on the x-axis and what’s on the y-axis?

  • Dave
64

Hi Dave,

when I wrote the post I didn’t yet have the image addresses for the included images. Only after I posted the initial version that had a placeholder (drawtiming.jpg) set for all eight images I immediately updated the post to render with the correct images. Maybe this change didn’t get reported by email, in that case I apologize for the confusion.

For now, I’ll assume you meant this image instead:

It is a histogram for the difference of diode-time minus announce-time for every update announcement and it’s corresponding diode-event. This is what I call “lag”, which I define as the time between a display update announcement and the occurrence of the stimulus on the display.
On the y-axis are the number of events (diode-announcement pairs) that fall into each bin. There are 100 bins per histogram.

Isn’t that what you meant by “I think you’ll want to match the events up one by one, and plot the distribution of the differences between the two events” ?

Philipp

65

Phillip,

Ah yes, this one is actually a distribution – I was looking at the email which still has the original placeholder, which was very confusing.

OK, let me digest the actual real plots…

  • Dave
66

OK. Two questions:

  1. What was the update rate for the movie and the refresh rate of the screen? It looks like the frame to frame interval is ~33 ms, which is the same as the “lag”, though I’m not sure this plot correctly groups the photodiode events and frame update events (e.g. the stimDisplayUpdate event should come after the photodiode event it refers to… I think with this grouping, the lags should only be plus-minus a millisecond or two).

  2. The “Display Link” version you refer to is your version, without the fence and finish?

  • Dave
67

first the supporting figure (please ignore this post)

68

Attachment: onoff.jpg (43.5 KB)

69

To 1:

Update rate for the movie (it’s a random dot pattern) = refreshRate().
refreshRate() is fairly new parameter one can set, it should return the displays refresh rate in double precision.
The refresh rate of the screen is approximately 59.9Hz (the Quato screen reports a couple of digits after the decimal point).

Therefore the frame to frame interval is not ~33ms but ~16.6ms. However, because falling edges of the raw diode signal are more subject to screen decay noise, I excluded all of these events and took only the rising edges (that’s when the bit code changes from 0 to 1 and the screen from black to white, respectively).

That the “lag” is almost equal to two screen refreshes is just a coincidence imho. To give more evidence for this I compiled a script that performs one display update and then ends. This produces one single display update announcement and one single “diode event”. It also shows how I do the Schmitt-Trigger simulation (the broader blue line is the result of it)

It is, however, possible that in a “real” run, the grouping get’s messed up. As I said, I am waiting for my diode array and as soon as it is up and running I will try to confirm my current findings.

On the other hand, it doesn’t at all seem clear to me that “the stimDisplayUpdate event should come after the photodiode event it refers to”. My understanding of the stimDisplayUpdate event was that it refers to the time when the VBL is complete. When exactly the image gets displayed on screen depends on the screen, e.g. on it’s latency. The diode event would therefore always occur after the stimDisplayUpdate event, but with a small and mostly constant lag. And when I believe my observations, exactly this seems to be the case.

To 2:
Exactly. The DisplayLink tester has no fence and no finish. It solely relies on the DisplayLinkCallback. I previously posted it here

Philipp

70

Just to add this piece of information because it’s hard to see on the plot: The “lag” in this On/Off example is 31.8140 ms.

71

I’ve been busy lately, so this took a lot more time than expected, sorry about that.

Finally, I’ve been able to run the requested tests using a 4-bit diode array with the bitcode-stimulus. The test-setup is the same one that was described earlier, so I am going to spare the details here.

In the attached figure, you can see two runs of the test-experient which updates the screen on every refresh. Chris previously noted that refreshes could be skipped due to the needDraw function returning false even though the stimulus needs to draw. To rule out this explanation, I override the needDraw function in my dynamic stimulus (a random dot pattern). So no possible refreshes get skipped in neither version that I tested.

Figure:

The upper plots show for every #stimulusDisplayUpdate announcement the lag between it’s corresponding diode-event. A diode-event is valid, when all of the four diodes have changed into the value they are supposed to have on the next refresh. If no diode has to change from high to low (that’s from bright screen to dark screen), then it is not necessary to wait for the slow decay screen and thus the lag is much smaller. If at least one of the diodes has to change from high to low, than the diode-event is only complete after the screen manages to change the corresponding pixels to black, which takes a bit longer and thus the lag is bigger.

The lower plots show for the sequence of #stimDisplayUpdate announcements the time that passed between two announcements. It is expected that the mean of these differences equals the screen refresh rate and that the standard deviation is small. Also there should be no outliers.

As one can see, this measurement pretty much confirms what I was suspecting before. The only difference between the two versions is that the times between refreshes is more regular in my fixed version. If I am not completely misinterpreting the results, it also means that the relationship between update announcement and update is more or less constant in both versions (although the mean lag differs by 10ms between versions). That in turn means that in the nightly build, the screen does not get updated regularly, as opposed to my fixed version, that shows very nice draw timings and regular updates.

I am using the nightly build of august 8, but not much has changed since then, as far as I can tell.

I hope this helps and motivates you to take a closer look at my version after all. I would really like to use an official MWorks release one day.

Thank you,
Philipp

Attachments:
test-experiment, the source code that contains all of my modifications, figure

72

Attachments:

73

… and the matlab script as well as the source codes without the build folder (I guess the forum will delete the previously posted version because it’s way too big)

74

Attachments:

75

I am using the nightly build of august 8, but not much has changed since then, as far as I can tell.

Yep… seems to be true, although there ARE lots of changes and I really like the reworked DynamicStimulusDriver

76

Attachment: 20111024_nightly_vs_cnlfork_final.png (11 KB)

77

Yep… seems to be true, although there ARE lots of changes and I really like the reworked DynamicStimulusDriver

I don’t know how easy it is for you to re-run this test, but if it’s not too hard, I have a variant on the nightly build you could try. It’s similar to your custom build in that it doesn’t use the GL fence at all and instead uses the expected output time reported by the display link as the stimulus announce time.

The installer package is named MWorks-NIGHTLY-no_gl_fence.dmg.
It’s built from the code in the no_gl_fence branch. If you’ve already updated your dynamic stimuli to work with the current nightly, then they should compile against this build, too.

Chris

78

Hi Chris,

maybe you noticed that the peaks in the recent results graph are a bit further away from each other than on my previous test. This is due to a changed display calibration (changed peak luminance and black point) and my laziness to not recalibrate the diode array accordingly.

I will do this as soon as possible and then make the comparison, but it could take until end of the week. So far I’ve tested the no-fence branch on my iMac and everything runs and compiles as usual. And by running as usual I mean that it throws the usual warnings and stutters, but that’s how all versions behaved.

Looking at your changes I’ve noticed a big difference between my proposal and your branch: You stick to not drawing when there is nothing new to draw while my version draws on every single frame regardless of how redundant that might be. I’ve chosen to do this because I’m under the impression that the displayLink gets much better at estimating the time for the next refresh when it actually draws. If it idles for a while, it might become out of sync with the graphics card and then skip refreshes or worse. report faulty update times. I know that performance wise it’s a nightmare to calculate the same things over and over again, but really - I think we can afford that, easily.

It would be worth investigating if it is possible to protect the contents of the back buffer. That way, if we don’t really need to draw, it would be possible to flush the conserved back buffer once more and get solid DisplayLink estimates. The best thing is that we wouldn’t even have to let the stimuli issue any GL commands for this and thus it wouldn’t be necessary to call their draw methods (like I do it). This would avoid that compatibility for lots of plugins breaks (like in my version).

I’ll report to you as soon as I have data.
Thanks again for setting this no-fence branch up!

Philipp

79

Results for noFence branch

Hi Chris,

it seems like the problems have disappeared. I don’t see differences between our version any more. Sorry this took so long, but I had to recalibrate the whole system since the setup screen was luminance-corrected. Therefore it’s now the low-high transition of the diodes that takes a bit longer (in my previous measurements it was the other way around), but this is not really important as it is due to the way the diodes get adjusted.

The important thing is that your version seems to be even better in predicting the real update (smaller standard deviations) than mine.

There is only one thing that’s left: Both versions produce a first displayUpdateEvent that is time-stamped ~900ms after the diodes update. it’s only the very first one, all the other updates are correctly timed. Maybe the DisplayLink messes up the very first outValue? I’ve double/triple-checked my analysis script and I can’t find the error there.

But for now, really good! I’ll keep on testing your version.

Cheers,
Philipp

80

Attachment: cnl_vs_nofence.png (9.07 KB)