Libdc1394 plugin for mworks

Support
1

Hi Chris,
As we discussed offline, I am interested in using libdc1394 to grab images from camera and analyze them (similar to Eyelink). I was able to grab single images using python bridge in mworks, and currently working on extending it. Searching online I noticed that coxlab had an implementation of the camera with pydc1394 library: GitHub - coxlab/python-simple-dc1394: A very simple frame grabbing python binding for libdc1394 . I was wondering how this could be integrated in mworks, or whether you have other suggestion on how to approach this.

Thanks,
Eghbal

2

Hi Eghbal,

I’m not sure what you mean by “integrated in MWorks”. Do you mean that you want to get position information you’ve extracted from your images in to MWorks?

Also, if you’re not currently using the Cox Lab package, how are you invoking libdc1394 from Python?

Chris

3

Thanks Chris,
My impression was that the coxlab is a module in Mwork (similar to Eyelink), but I might be wrong.
Currently I have added pydc1394 (GitHub - jordens/pydc1394: python libdc1394 wrapper) to the library of system python and call the camera within python.
in either scenario I need to extract position information from python or C++ and send to mworks. For python it would be easier with get_var and set_var, but I am not sure if it slows down the frame capture rate.

Thanks,
Eghbal

4

Hi Eghbal,

My impression was that the coxlab is a module in Mwork (similar to Eyelink), but I might be wrong.

Actually, it’s just another Python wrapper around libdc1394, i.e. an alternative to pydc1394. However, pydc1394 has the advantage of being pure Python, so you don’t need to compile it.

in either scenario I need to extract position information from python or C++ and send to mworks. For python it would be easier with get_var and set_var, but I am not sure if it slows down the frame capture rate.

If your Python code is running inside MWServer (via run_python_file or run_python_string), then setvar is what you need. In this scenario, you should spawn a new Python thread to handle ongoing image capture, and invoke setvar from that thread whenever new data is available.

If your Python code is running in an external process (via a client- or server-side conduit), then you should use the conduit’s send_data method to get the positions in to MWorks.

In either case, I wouldn’t expect the data transfer in to MWorks to be a performance bottleneck.

Chris

5

Hi Chris,

Thanks, that makes sense. My current method is to do continuous capture by scheduled actions, and it seems to be working. Ideally I would want a window to display both image sequence online and a script in python to extract the positions and pass it on to mworks.

I was wondering what would be you suggestion regarding image display. would it be easier to pass the image as a matrix to mworks for display, or have python running on MWserver do it. I notice that sometimes MWServer crashes during acquisition.

My other question is regarding the second approach. I haven’t implemented the second approach, and curious about differences in capabilities. Can I have the external process running on a windows machine ?

Thanks,
Eghbal

6

Hi Eghbal,

I was wondering what would be you suggestion regarding image display. would it be easier to pass the image as a matrix to mworks for display, or have python running on MWserver do it.

It’d definitely be easier to display it using Python (maybe with matplotlib).

I notice that sometimes MWServer crashes during acquisition.

That’s very bad! If it happens again, can you send me a crash report?

I haven’t implemented the second approach, and curious about differences in capabilities.

The capabilities are similar, although setup is a little more complicated. The main (potential) benefit of using a conduit is that your code will be running in a separate process, meaning it can’t interfere with MWServer’s process (and vice versa).

Can I have the external process running on a windows machine?

No. To use a conduit, both processes must be running on the same machine. However, your Python code could talk to a process running on a Windows machine over the network. That would add complexity and some additional latency, but it’s definitely an option.

Chris

7

Hi Chris,

thanks for the information. I wanted to give you an update.
I tried matplotlib to continuously update a graph but for some reason I only get the plots in the run for the protocol. I am wondering if it is because the function call happen inside a scheduled action routine.

one of the errors that Mwserver reports periodically during the experiment is the following:

 ERROR: Scheduled action is starting 3046us behind intended schedule

I am uncertain what the source of this is. Can the processing power of the machine cause such an error, or this is caused by other sources.
I have also attached a copy of crash report. I’d appreciate your input.

Thanks,
Eghbal

8

Attachment: Mwserver_crash_report.txt (90.8 KB)

9

Hi Eghbal,

I tried matplotlib to continuously update a graph but for some reason I only get the plots in the run for the protocol. I am wondering if it is because the function call happen inside a scheduled action routine.

I’m not sure what you mean. Are you saying the plots only update when the experiment is running? If so, then you’re correct: Scheduled actions stop automatically when the experiment stops.

Instead of a scheduled action, I recommend handling the updates in a Python thread. That will keep running until you stop it, regardless of the experiment state.

one of the errors that Mwserver reports periodically during the experiment is the following:

I suspect your Python code is taking a (relatively) long time to run, and your scheduled action is getting behind schedule. Again, using a Python thread is a better approach.

I have also attached a copy of crash report. I’d appreciate your input.

Thanks for that. The crash is happening in a thread created by libdc1394. If you can reproduce it outside of MWorks, then it might be worth reporting to libdc1394’s developers.

Chris

10

Thanks Chris,

sorry for not being precise. What I actually meant was that the python plot does not update until the end of the protocol, and then I get a sequence of plot updates. Would it be possible if you could direct me to an example of python thread to use as a reference? I really appreciate it.
You are right, I think the python is falling behind as it is saving images at 200Hz, and it is falling behind the next call. when I run the python script without saving, the schedule action does not fall behind. I was wondering if you have any advice on that, I am going to test it on a faster mac system. I remember Mworks had issues with Apple MacPro systems in connecting to NIDAQ through thunderbolt connection. I was curious if it is fixed now.

Thanks,
Eghbal

11

Hi Eghbal,

Sorry for the delayed response. Taking your questions in reverse order:

I remember Mworks had issues with Apple MacPro systems in connecting to NIDAQ through thunderbolt connection. I was curious if it is fixed now.

It was an NI driver problem, not an MWorks problem. I don’t think anything has changed, but you could check with Evan in the Jazayeri Lab.

I think the python is falling behind as it is saving images at 200Hz, and it is falling behind the next call. when I run the python script without saving, the schedule action does not fall behind. I was wondering if you have any advice on that

Given that a typical video file is encoded at 30 or 60 frames per second, 200 images per second seems excessive. Are you sure you need to save all of them? Maybe it would be sufficient to save every second or third image?

Would it be possible if you could direct me to an example of python thread to use as a reference?

This turned out to be a little more complex than I initially expected. Importantly, I realized that creating plots inside MWServer (e.g. via run_python_file) really doesn’t work. Thus, I’ve attached two example experiments with associated Python files: one that runs its Python code in the server and does not do any plotting, and another that runs its Python in a separate process (via a conduit) and does do plotting.

The first example (in_server.mwel) uses a Python file resource and the run_python_string action to execute its Python code within MWServer. The Python code periodically updates two position variables, which in turn determine the position of a white dot on the stimulus display. If everything is working, the dot should change position once per second.

The second example (via_conduit.mwel) accomplishes the same task, but the Python code runs in a separate process and communicates with MWorks via a conduit. Also, in addition to generating positions, it continuously updates a scatter plot showing all the positions it has generated. To run it, first start the experiment, then run the Python code in one of two ways:

  1. From the command line, e.g. python via_conduit.py
  2. From MWClient’s Python Script Bridge window

I realize there’s a lot going on in these examples, so please let me know if you have questions. Also, I’d be happy to sit down with you and go over them, if that would be helpful.

Cheers,
Chris

12

Attachment: python_thread.zip (3.06 KB)