Online analysis with Python

Support
1

Hi there,

I have been looking for a tutorial or guide on how to set up an online analysis of my experiment via python.

I found these threads and pages:

I keep failing at the very first step: loading the mworks.conduit:

import sys; sys.path.append("/Library/Application Support/MWorks/Scripting/Python")
from mworks.conduit import *

Traceback (most recent call last):
  File "<input>", line 3, in <module>
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
  File "/Library/Application Support/MWorks/Scripting/Python/mworks/conduit.py", line 1, in <module>
    from ._mworks import (_IPCClientConduit,
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
    module = self._system_import(name, *args, **kwargs)
ImportError: dlopen(/Library/Application Support/MWorks/Scripting/Python/mworks/_mworks.so, 2): Symbol not found: _PyCObject_Type
  Referenced from: /Library/Application Support/MWorks/Scripting/Python/mworks/_mworks.so
  Expected in: flat namespace
 in /Library/Application Support/MWorks/Scripting/Python/mworks/_mworks.so

I am running MWorks Version 0.8 (a914d383), I have set up a conda python 3.7 environment on Mojave 10.14.6

Python 3.7.3 (default, Mar 27 2019, 16:54:48) 
[Clang 4.0.1 (tags/RELEASE_401/final)]

What am I missing?
My aim is to monitor, summarise, and plot experimental variables online, trial by trial with the Python Script Bridge

thanks
ant

EDIT:
I tried by switching to python 3.6 and 2.7 and this is what I got

>>> import sys
>>> sys.path.insert(0, '/Library/Application Support/MWorks/Scripting/Python')
>>> from mworks.data import MWKFile
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Application Support/MWorks/Scripting/Python/mworks/data.py", line 4, in <module>
    from ._mworks import ReservedEventCode, _MWKFile, _MWKStream
SystemError: initialization of _mworks raised unreported exception
2

Hi Ant,

I have been looking for a tutorial or guide on how to set up an online analysis of my experiment via python.

Sorry, MWorks’ documentation is still lacking in this area.

I found these threads and pages:

FYI, Python file resources and Python actions are used for running Python code within your experiment (i.e. inside MWServer), so they’re not relevant when using the Python Script Bridge.

I keep failing at the very first step: loading the mworks.conduit

As I think you discovered, MWorks 0.8 supports only Python 2.7 and 3.6. MWorks 0.9 supports Python 2.7 and 3.7. The current nightly build, which will eventually become MWorks 0.10, supports Python 3.5 and later. (Python 2.7 is no longer supported.)

However, in MWorks 0.8 and 0.9, the Python bridge window always launches the system Python 2.7 (/usr/bin/python). The option to choose the Python version to launch was added to the nightly build fairly recently. If, instead of the Python bridge window, you use a server-side conduit, you can launch your script using the compatible Python executable of your choice. (The example you referenced works in both client-side and server-side mode.) However, that does require you to run your Python code on the machine that’s running MWServer, so it’s not helpful when you’re connecting to a remote system.

I tried by switching to python 3.6 and 2.7 and this is what I got

SystemError: initialization of _mworks raised unreported exception

Huh. I don’t think I’ve seen that error before. The _mworks module imports numpy, so maybe numpy is failing to import for some reason?

Cheers,
Chris

3

Hi Chris,

thanks a bunch for the quick reply.

I tried several combinations of python and mworks versions and could not manage to read datafiles neither online nor offilen.

I then finally decided to try with MWorks 0.9 and Python 3.7 in the conda environment and finally managed to read data files offline from both my own experiment and from the example FindTheCircle.

I totally failed the conduit method though, not sure why, probably my incompetence.

So I am now considering installing the nightly build and try out the python bridge, which is perhaps something I can understand intuitively better (coming from matlab).

I wanted to ask you in advance if you could give me a hint on how to set up the Python Bridge script and how to setup the mwel experiment itself to work with the bridge.

Also, is it the bridge going to be kept in future versions of MWorks?

thanks
ant

4

Hi Ant,

I tried several combinations of python and mworks versions and could not manage to read datafiles neither online nor offilen.

Reading data files online (i.e. while they’re still being written) isn’t supported. However, you certainly should be able to read them offline. Can you tell me more about the errors you were encountering?

Regarding the “initialization of _mworks raised unreported exception” error, is it possible that you’re seeing
this issue?

I wanted to ask you in advance if you could give me a hint on how to set up the Python Bridge script and how to setup the mwel experiment itself to work with the bridge.

I’ve posted an updated Python bridge plotting demo on GitHub. It should be a good starting point for working with the Python bridge in the nightly build.

The Conduit class (in file conduit.py) hides some of the hairier details and sets things up so that any matplotlib windows you open remain responsive. You should be able to use it with multiple experiments without modification.

The experiment-specific Python file (python_bridge_plotting_demo.py) defines an event handler function that receives updated values for variables of interest, processes them in some way, and then draws one or more plots. It also invokes Conduit.main, passing it the custom function and the list of variable names to monitor. (Note that the function should not call pyplot.show, as that’s handled by Conduit.main.)

The only thing that the experiment file (python_bridge_plotting_demo.mwel) needs to do is tell the Python script when to start/stop collecting events. This is done via two variables (called start and stop in the example). Assigning a value to the first variable tells the script to begin collecting events. Assigning a value to the second variable tells the script to stop collection events and invoke the event handler function on the collected events. The example assigns start/stop at the beginning/end of each trial, respectively.

To use the example, first load the experiment, then load the experiment-specific Python file via the Python script bridge window. The first time you load the script, you’ll also have to select the Python executable to use (e.g. python3 in your local Anaconda installation).

Also, is it the bridge going to be kept in future versions of MWorks?

Yes.

Cheers,
Chris