LPMuds.net

In August 2019, I finally finalised the JIT compiler for DGD. It is implemented as an extension module, so that it will work with both DGD and Hydra, and indeed makes use of LLVM. It does not yet work on Windows, but should work on any other platform with LLVM support; it was tested on i386, x86_64 and ppc.

The extension starts and then communicates with an external program, which decompiles LPC bytecode to LLVM IR, and then uses clang to create a native shared object which the extension can load into DGD's memory. LPC bytecode files, LLVM IR files and shared object files are all cached, so that this translation needs to happen only once.

Keeping the LPC=>LLVM decompiler separate from DGD avoids issues with possible memory leaks and crashes, though of course an error in the bytecode translation could still be fatal. While translating a LPC bytecode image to a shared object takes much longer than an inline JIT compiler would, it runs completely independently from DGD, which is never delayed. Even loading the shared object into DGD's runtime memory is handled by a separate thread.

DGD: https://github.com/dworkin/dgd
LPC extension modules: https://github.com/dworkin/lpc-ext

STM has the problem that it appears to scale worse than a well-designed implementation using locking, and you still have most of the same design headaches as you have with locking for multi-threaded code. Hydra doesn't require or implement STM, though it uses some of the same underlying ideas.

In any case, FluffOS & DGD are already overpowered for MUDs on present-day hardware, and DGD with JIT compilation is ridiculously overpowered. The speedup offered by JIT compilation will probably not play out in the realm of MUDs.

I noticed that you started an issue for JIT compilation, where you consider the option to compile to LLVM IR directly.

Another option would be to do both, preserving lightning-fast LPC compilation. For JIT compilation for DGD, I chose to compile from VM bytecode to LLVM IR in multiple passes; detecting basic blocks, determining the types of expressions, doing dataflow analysis, and finally generating LLVM IR which can be compiled with clang. I have this mostly working (mud boots and I can walk around), but there are a few bugs to iron out.

You can find the code at https://github.com/dworkin/lpc-ext/tree/master/jit.  It could probably be adopted for FluffOS without too much difficulty.

I might be wrong about this but I suspect that function pointers in dgd work be okay if you assume a sort of late binding i.e. make the function point just hold the object or program on which the pointer is called plus a string and arguments. I.e. a sort of on demand delayed call_other with no guarantees that it points to anything of if it ever did. I am not sure however with object upgrades how lambda closures (anon-funcs) could be made to work since in principle you either carry the code with the variable or compile the code into an object which is nameless and point to it. Could you not have an nameless object hold the code and have the function pointer point to the code in a nameless object that gets compiled on the fly? Depending on the copy semantics one could perhaps even use an lwo.

This discussion is wandering far afield, but the difficulty in combining object upgrading with anonymous functions is that you cannot unambiguously match an anonymous function reference to code in a newly upgraded object. Thus anonymous functions cannot be automatically upgraded.

None of this pertains to Fluffos, which doesn't have object upgrading.

Actually, you don't need multithreading.  You need to not abort LPC thread execution when the time's up, but save the stack machine's state for that object and context switch to the next one.

Right now, if a thread exceeds their eval ticks, it just gets killed.  How hard would it be to save state and restore it next time around the loop?

That is so 20th century.  Event-driven, non-blocking, asynchronous I/O models are the height of fashion now: Node.js, Akka, Vert.x.

Because one of my high level goals is to reach both 1st & 3rd world infrastructures, I'm trying to figure out an infrastructure model that would be essentially UI agnostic.  Maybe something Client/Server Obj/Event ...

People in third-world countries don't have computers, but they do have phones; in a few years they'll have Android smartphones. Given that you want to reach this enormous population, perhaps you should drop support for the tiny and shrinking group of people who will connect with telnet-like clients from first-world computers. 

I'm entirely certain you were successful in pulling all the game-mechanical cruft out of DGD, and hopefully all of the freakishly broken "performance optimizations" too.  (My favorite is the behavior I discovered under LDMud after various of my daemon objects stopped working: if nothing attempts to call a function on an object via call_other() for some period of time, reset() stops getting called!  And there is no driver config switch for this behavior!  So awesome.  You can imagine how much of my hair I pulled out trying to debug that idiocy.)  The only assumption I had in mind for DGD was that of being a telnet server.

The telnet part is optional; some DGD mudlibs use HTTP.