Author Topic: standard malloc and brk in smalloc.c  (Read 2712 times)

Offline silenus

  • BFF
  • ***
  • Posts: 165
    • View Profile
standard malloc and brk in smalloc.c
« on: May 13, 2013, 06:30:06 AM »
I have been wondering if the sbrk brk calls are used anywhere other than in smalloc.c. I have been wondering how problematic it would be to have the custom allocators replaced by the default standard library one.

Regards,

Sil.

Offline FallenTree

  • BFF
  • ***
  • Posts: 482
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #1 on: May 13, 2013, 06:42:58 AM »
Are you using deletion? These are on my list to deletion.

Offline FallenTree

  • BFF
  • ***
  • Posts: 482
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #2 on: May 13, 2013, 01:31:14 PM »
sorry, my phone kinda of screwed my response above.

due to the way our malloc wrapper works, (macro vs library),   with switching to c++ and  STL, it will manage less and less memory.

I plan to delete all implementations very soon, the only option will be using system malloc  or using google's tcmalloc library. (which is going to be the default).

Offline quixadhal

  • BFF
  • ***
  • Posts: 629
    • View Profile
    • WileyMUD
Re: standard malloc and brk in smalloc.c
« Reply #3 on: May 13, 2013, 04:36:52 PM »
What's google's tcmalloc?

Personally, I still think a garbage collecting malloc is the only really useful alternative to the system malloc.  Performance was the issue back in 1990, but I doubt anyone cares these days, so as long as we have a statistics wrapper to let you know what's allocated where, it's all good.

The only gain from garbage collection is minimizing the number of slow leaks from bugs in either the driver core, or more likely in user-supplied packages... but for muds which like to never reboot, over a few months that can be quite a gain.

Offline FallenTree

  • BFF
  • ***
  • Posts: 482
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #4 on: May 13, 2013, 06:21:54 PM »
There is no "garbage" in the driver. Because the driver doesn't work that way.  Everything cross the LPC <->driver boundary is strictly maintained.  Everything involved in LPC function or efun calls is allocated entering the scope then deallocated when out of scope (have you notice there is no "malloc" function inside LPC? the only thing can live outside the scope is by creating a object).

The only thing that can be possibly garbage collected in the driver are  "objects".  (anything else count as leaks, various hashtable, shared string is something else to talk about).

The "objects" garbage collection, is already handled by "reclaim_objects()" routine.

So, once again, there is no "garbage" in the POV of driver.  The problem with sysmalloc is

1) performance issue
2) limited control / incapable of timely free memory to system.

which both solved nicely by tcmalloc. (by the way I work for google so I know it's good).  It is fast and you can control how fast to release to the system.
The library also contains good utility that I want to use in the driver. Thus the reason to add it.

Offline FallenTree

  • BFF
  • ***
  • Posts: 482
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #5 on: May 13, 2013, 06:39:55 PM »
and with the small leaks you mentioned in efuns etc. gc will probably not help. It will only help for cases of stray malloc that got used once and left on the floor. but those kind of issue has been eliminated mostly.

The problem right now involves some global state that get allocated and left there forever( like the outstanding case in unique_mapping now, horrible code that I havn't understand yet).

Offline chaos

  • BFF
  • ***
  • Posts: 291
  • Job, school, social life, sleep. Pick 2.5.
    • View Profile
    • Lost Souls
Re: standard malloc and brk in smalloc.c
« Reply #6 on: May 13, 2013, 07:48:29 PM »
the only thing can live outside the scope is by creating a object

I do not believe this to be the case.

Code: [Select]
mixed array foo = ({ 1, 0 });
foo[1] = foo;

Presto, scope-independent garbage that will never be cleaned up, at present.

Offline FallenTree

  • BFF
  • ***
  • Posts: 482
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #7 on: May 13, 2013, 09:12:08 PM »
"foo" needs to belong to one object or within one function.

when object get destructed or goes out of scope on that function it will be "de-allocated"

The self-referencing problem prevent it to be fully de-allocated and that is not really related to the garage collection. (let alone the malloc implementation!)   I actually don't know this problem exist now,it should be solved similar to reclaim objects

Offline chaos

  • BFF
  • ***
  • Posts: 291
  • Job, school, social life, sleep. Pick 2.5.
    • View Profile
    • Lost Souls
Re: standard malloc and brk in smalloc.c
« Reply #8 on: May 13, 2013, 11:37:25 PM »
"foo" needs to belong to one object or within one function.

when object get destructed or goes out of scope on that function it will be "de-allocated"
Maybe.  It isn't my preference, because you can't have that without weirdness where the cure is IMO worse than the disease, like copy-on-write for arrays and mappings passed via call_other.

The self-referencing problem prevent it to be fully de-allocated and that is not really related to the garage collection. (let alone the malloc implementation!)   I actually don't know this problem exist now,it should be solved similar to reclaim objects
It does seem like we're talking about different levels of abstraction at different points in this thread.  Self-reference leaks are definitely related to the issue of LPC-level garbage collection, though.  I mean, they're the whole reason anyone does mark-and-sweep instead of refcounting.

Offline silenus

  • BFF
  • ***
  • Posts: 165
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #9 on: May 13, 2013, 11:50:32 PM »
Thanks Sunyc for the reply above. Will C++ code be accepted into the driver? I do not really see why not nowadays given that even on a lot of embedded systems nowadays one can bootstrap into gcc and do a c++ compile. If not I will write what I want to do in C.

Offline FallenTree

  • BFF
  • ***
  • Posts: 482
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #10 on: May 13, 2013, 11:53:59 PM »
next-3.0 has already converted to C++ , so you are free to submit c++ code/patch/pull-request to

https://github.com/fluffos/fluffos

I'm inclined to suggest try to keep it conforming to http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml

Offline wodan

  • BFF
  • ***
  • Posts: 434
  • Drink and code, you know you want to!
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #11 on: May 17, 2013, 10:13:19 AM »
sorry, my phone kinda of screwed my response above.

due to the way our malloc wrapper works, (macro vs library),   with switching to c++ and  STL, it will manage less and less memory.

I plan to delete all implementations very soon, the only option will be using system malloc  or using google's tcmalloc library. (which is going to be the default).

Does that free memory when you realloc to a smaller block? that happens quite a lot in the driver, it did not do so when I last tried, which is why the malloc64 and malloc32 options exist.

Offline FallenTree

  • BFF
  • ***
  • Posts: 482
    • View Profile
Re: standard malloc and brk in smalloc.c
« Reply #12 on: May 17, 2013, 10:45:09 AM »
nothing free memory immediately to the system.  in case of TCmalloc,  it is being added to freelist and  by default it release to system at 1Mb/second.  it at least have the tools and means to analyze it.