# Patch for mw\_datatools **URL:** \ **Category:** Support\ **Created:** [January 26, 2011, 9:55pm UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693 "2011-01-26T21:55:12Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [January 26, 2011, 9:55pm UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/1 "2011-01-26T21:55:12Z")
Chris, I’m submitting a patch for mw\_datatools. It’s generated against the most recent git repository. Basically, it adds the following stuffs: MWKStream: - Writing Python objects (int, float, dict, tuple, list, etc.) to a mwk file (in scarab format) - Low-level scarab IO functions (\_scarab\_session\_read, \_scarab\_session\_write, etc.) MWKFile: - Appending new codes into a existing mwk file (mostly for merging) - Cached codec/reverse\_codec - Other small changes, such as managing temp files (again, for merging), real mwk files, etc. PythonDataBindingsHelpers.cpp: - Python object to scarab conversion function (convert\_python\_to\_scarab) - Bridge functions to support low-level IO functions in MWKStream (_scarab\_session_\*) - Some modifications adopted from PEP and Official Python Website (e.g., “return Py\_None;” to “Py\_RETURN\_NONE;”) Let me know if you have any questions. Thanks! Ha ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [January 26, 2011, 9:55pm UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/2 "2011-01-26T21:55:12Z")
Attachment: [mw\_datatools.patch](https://mworks.discourse.group/uploads/short-url/hWjJr55LBDu5uD7YX5Rz1gIeWAd.patch) (27.9 KB) ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [January 27, 2011, 7:55pm UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/3 "2011-01-27T19:55:00Z")
Thanks, Ha. I’ll take a look at this and get back to you. Chris ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [February 1, 2011, 5:54pm UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/4 "2011-02-01T17:54:55Z")
Hi Ha, This is a very nice patch! Thanks for taking the time to put it together. I’m happy with all the C++ code changes you’ve made. However, the stuff you added to data.py makes me nervous. I’m not sure we should be encouraging people to merge data files. If you or others want to do so, you should be free to try (so I’m fine with exposing the low-level Scarab I/O functions as private methods of MWKStream). However, I don’t want to be the person who has to provide support if something goes horribly wrong! I propose incorporating your C++ code changes (as well as the codec-caching and reworked MWKFile.unindex() from data.py) into mw\_datatools, while the file-merging stuff remains in your personal toolbox (which you’re free to share with others, of course). Does that sound OK? Thanks, Chris ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [February 2, 2011, 1:18am UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/5 "2011-02-02T01:18:20Z")
Hi Chris, Thanks for your note, and I agree with your points. I’m considering making a new class by inheriting `MWKFile` and putting all the merge stuffs into the new class. By this way, we can minimize some potential side-effects of the merge code while I can continue what I’m doing now and maintain/develop my “experimental” code without worrying too much. Meanwhile, I stripped off the merge code from `data.py` and attached a concise version of it. You might want to take a look at it. Thanks, Ha ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [February 2, 2011, 1:18am UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/6 "2011-02-02T01:18:20Z")
Attachment: [data\_concise.py](https://mworks.discourse.group/uploads/short-url/e0f8yiRHGyFCJ46eqR3rG2wJzNw.py) (7.08 KB) ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [February 2, 2011, 3:10pm UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/7 "2011-02-02T15:10:28Z")
Hi Ha, > I’m considering making a new class by inheriting MWKFile and putting all the merge stuffs into the new class. By this way, we can minimize some potential side-effects of the merge code while I can continue what I’m doing now and maintain/develop my “experimental” code without worrying too much. That sounds like a good plan. > Meanwhile, I stripped off the merge code from data.py and attached a concise version of it. Thanks. I made a few additional changes to the MWKFile class in data\_concise.py: - Removed the reserved codec code constants - Removed the backup-management methods - Renamed `empty_dir()` to `_empty_dir()` - Changed the default value of the `empty_dir` parameter of `unindex()` to True (to preserve existing behavior) I just [committed](https://github.com/mworks-project/mw_datatools/commit/61d3829a0b63448f84c0c18286cc83bf0bf2b03b) the changes. They should be in tonight’s nightly build. Chris ---
**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\ **Post date:** [July 19, 2022, 11:50pm UTC](https://mworks.discourse.group/t/patch-for-mw-datatools/693/8 "2022-07-19T23:50:59Z")