Author Topic: Lag Debugging  (Read 1959 times)

Offline beren

  • Acquaintance
  • *
  • Posts: 17
    • View Profile
    • http://midnightsun2.org:3328
Lag Debugging
« on: January 05, 2015, 04:53:39 AM »
I am using the latest version of FluffOS from github on a Centos7 64bit VM. (finally moving away from cygwin)
We are on a 2 second heartbeat but on occasions the interval between heartbeats is 4 or 6 seconds due to some commands taking longer to complete.

I managed to pin point and correct a few problems but on occasion we still get these small lag occurrences.

Is there a way to determine which commands were executed by the driver during the previous heartbeat ?
If I enable the LPC debug on the driver the driver lags almost constantly as it attempts to write all the commands to the debug.log or stdout so
that was not very useful.
Any other ideas ?
Midnight Sun - http://midnightsun2.org:3328

Once you enter a land where even the sun
does not go to sleep at night... will you?

Offline quixadhal

  • BFF
  • ***
  • Posts: 620
    • View Profile
    • A Waste of Time
Re: Lag Debugging
« Reply #1 on: January 05, 2015, 04:54:22 PM »
The usual way is to start adding checkpointing code to your hearbeat functions and see what reports on each cycle.  IE: get the current time value and store it in a variable on the object.  Then, on the next heartbeat, compare the time to your stored value.  If it's in the expected range (<= 3 seconds), update the stored value and carry on.

If it's > 3 seconds, emit a warning to your logs and update the stored value, and move on.

If you see all kinds of random things reporting, it sounds like a driver issue.  If the same small number of things keep reporting all the time, it might be something they all inherit that's very expensive.

Offline beren

  • Acquaintance
  • *
  • Posts: 17
    • View Profile
    • http://midnightsun2.org:3328
Re: Lag Debugging
« Reply #2 on: January 06, 2015, 03:00:23 AM »
Will try that even though I am not certain it will work.

Apart from things happening on heartbeat we have dozen call outs that not always happen on heartbeat intervals doing a variety of things that might be causing some of the lag.
Was hoping that there was a way that the driver kept an in memory log of things executed last heartbeat so I would just dump those into a file when I noticed the lag.

Thanks for the pointer though
Midnight Sun - http://midnightsun2.org:3328

Once you enter a land where even the sun
does not go to sleep at night... will you?

Offline trappolo

  • Acquaintance
  • *
  • Posts: 6
    • View Profile
Re: Lag Debugging
« Reply #3 on: March 15, 2015, 08:34:44 AM »
Try to use dumpallobj() function.
You can find description here:
http://fluffos.github.io/doc/efun/internals/dumpallobj.html
or check objects() function which objects are too much in the memory.
I think one of your code heart beat have a new() function, which create every hb a new object.