# Long saving time

**URL:** <https://mworks.discourse.group/t/long-saving-time/211>\
**Category:** Support\
**Created:** [October 22, 2019, 3:05pm UTC](https://mworks.discourse.group/t/long-saving-time/211 "2019-10-22T15:05:48Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [October 22, 2019, 3:05pm UTC](https://mworks.discourse.group/t/long-saving-time/211/1 "2019-10-22T15:05:48Z")

</div>

Hi Chris,

If you remember, a while ago, i had an issue with data not being saved. While this issue is resolved; I have a different one now. Mworks takes ages closing the data file (more than 1.5 hours at this moment and going). The file size so far is about 3.6GB. I am using v 0.9 as well. Any idea what causes this? if it’s unavoidable, is it possible to have another instance of Mworks to start my next experiment?

Cheers,  
Beshoy

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [October 22, 2019, 8:05pm UTC](https://mworks.discourse.group/t/long-saving-time/211/2 "2019-10-22T20:05:11Z")

</div>

Hi Beshoy,

It sounds like the [same root problem as before](https://mworks.tenderapp.com/discussions/problems/384#comment_47225092): At some point in the experiment, the rate at which events are written falls drastically below the rate at which events are generated, producing a huge backlog, and the data file can’t close until the backlog is cleared. Please see the linked comment for my suggestions on how to proceed.

Cheers,  
Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [October 23, 2019, 10:57am UTC](https://mworks.discourse.group/t/long-saving-time/211/3 "2019-10-23T10:57:22Z")

</div>

Hi Chris,

I checked the data read and write rates. During the experiment. THe data read/sec is around 40KB and the written is about 13-25MB. Are those normal numbers?

Cheers,  
Beshoy

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [October 23, 2019, 7:25pm UTC](https://mworks.discourse.group/t/long-saving-time/211/4 "2019-10-23T19:25:58Z")

</div>

Hi Beshoy,

> During the experiment. THe data read/sec is around 40KB and the written is about 13-25MB. Are those normal numbers?

Yes, those are totally reasonable I/O rates.

Can you try running the [special MWorks build](https://mworks.tenderapp.com/discussions/problems/384#comment_47225808) I provided previously? The log file it generates should provide some insight into what’s happening.

Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [November 27, 2019, 11:26am UTC](https://mworks.discourse.group/t/long-saving-time/211/5 "2019-11-27T11:26:14Z")

</div>

Hi Chris,

I tried this special build, but i am not sure how and where to find the log file to send you.

Cheers,  
Beshoy

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [November 27, 2019, 3:20pm UTC](https://mworks.discourse.group/t/long-saving-time/211/6 "2019-11-27T15:20:03Z")

</div>

Hi Beshoy,

It should be in the [same place as before](https://mworks.tenderapp.com/discussions/problems/384#comment_47220814): `/tmp/mwserver_event_file_log.txt`.

Thanks,  
Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [November 28, 2019, 1:19pm UTC](https://mworks.discourse.group/t/long-saving-time/211/7 "2019-11-28T13:19:35Z")

</div>

Hi Chris,

I attached the log file. Again, it took hours to finish saving the file.  
Thanks in advance for the help.

Cheers,  
Beshoy

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [November 28, 2019, 1:19pm UTC](https://mworks.discourse.group/t/long-saving-time/211/8 "2019-11-28T13:19:35Z")

</div>

Attachment: [mwserver\_event\_file\_log.txt.zip](https://mworks.discourse.group/uploads/short-url/ps4o3mAzp1cKpVUieNlhsgYKH0X.zip) (4.12 MB)

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [December 2, 2019, 8:41pm UTC](https://mworks.discourse.group/t/long-saving-time/211/9 "2019-12-02T20:41:42Z")

</div>

Hi Beshoy,

Thanks for the log file. I’ll take a look and see if it provides any new insight into the problem.

Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [December 5, 2019, 9:02pm UTC](https://mworks.discourse.group/t/long-saving-time/211/10 "2019-12-05T21:02:08Z")

</div>

Hi Beshoy,

The log file confirms that writing events to disk is taking an unreasonably long time (e.g. 100ms or more per 1000 events, where a reasonable duration would be \<10ms). I can induce this problem on my Mac Pro by using a stress-testing tool that loads the system with disk writes.

I’ve also discovered that I can eliminate the problem, even with the stress-testing tool running, by setting SQLite’s [“synchronous” flag](https://sqlite.org/pragma.html#pragma_synchronous) to “OFF”. Disregarding for a moment whether this is really a good idea, it’d be interesting to know if this change resolves the issue for you, too. If you’re willing to try it, I’ve created a modified build of MWorks that you can get at

> **[MWorks-sqlite\_synchronous\_off.dmg](https://www.dropbox.com/s/chojes7fa5eepl9/MWorks-sqlite_synchronous_off.dmg?dl=0)**
>
> Shared with Dropbox

It’s identical to the current nightly build, except for the change to the synchronous flag.

I should note that, when I start the stress-testing tool, Activity Monitor’s data written/sec figure jumps from around 60MB to 700MB or more. Since you aren’t seeing a similarly high write volume, you may be experiencing a different issue. Still, I think this is a worthwhile test to run.

Thanks,  
Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [December 7, 2019, 5:29pm UTC](https://mworks.discourse.group/t/long-saving-time/211/11 "2019-12-07T17:29:56Z")

</div>

Hi Chris,

Thanks for the modified version. I will test it on Monday and let you know how it goes!

Cheers,  
Beshoy

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [December 9, 2019, 6:27pm UTC](https://mworks.discourse.group/t/long-saving-time/211/12 "2019-12-09T18:27:34Z")

</div>

Hi Chris,  
This worked! File size looks appropriate and I took a quick look at the events and things seem in order.

Cheers,  
Beshoy

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [December 10, 2019, 9:22pm UTC](https://mworks.discourse.group/t/long-saving-time/211/13 "2019-12-10T21:22:11Z")

</div>

Hi Beshoy,

That’s great news! I have to think a bit more about whether this should be the default configuration going forward, although my current feeling is that it should.

Cheers,  
Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [January 7, 2020, 3:43pm UTC](https://mworks.discourse.group/t/long-saving-time/211/14 "2020-01-07T15:43:17Z")

</div>

Hi Beshoy,

> I have to think a bit more about whether this should be the default configuration going forward

I thought about it and [decided](https://github.com/mworks/mworks/commit/28588dc1bf3f6dd2b0f1d47d4fe9aa2b006169f1) that this is the best default. The change is now in the [nightly build](https://mworks.github.io/downloads/) and will be included in the next MWorks release.

Cheers,  
Chris

---

<div class="post-metadata">

**Author:** ![cstawarz](https://yyz1.discourse-cdn.com/flex031/user_avatar/mworks.discourse.group/cstawarz/32/3_2.png) [@cstawarz](https://mworks.discourse.group/u/cstawarz)\
**Post date:** [July 19, 2022, 8:40pm UTC](https://mworks.discourse.group/t/long-saving-time/211/15 "2022-07-19T20:40:36Z")

</div>