Author Topic: JIT compiler revisited for Fluffos/dgd  (Read 2365 times)

Offline silenus

  • BFF
  • ***
  • Posts: 199
    • View Profile
JIT compiler revisited for Fluffos/dgd
« on: November 05, 2015, 01:38:00 pm »
I noticed some old posts mentioning the inclusion of a jit compiler for fluffos might be in the works. Is this now no longer the case? I think that the upcoming 3.8 release of llvm with patchpoint and stackmaps might be suitable for this sort of thing. The work involved however is quite significant since fluffos unlike dgd has a lot of efuns and packages that directly manipulate the stack which would all have to be reimplemented to work with the new virtual machine. I am also curious also about the old c code generation that mudos IIRC use to have... Is this code still available somewhere? I am sort of curious how this problem was addressed given the plethora of efuns.

I am actually tempted to take a few months to work on this and give it a go. Converting the icode.c to generate llvm code directly and rewriting some of the apply and call hooks to call llvm::Functions pointed to by new program tables would be the main challenge.

Offline Dworkin

  • Acquaintance
  • *
  • Posts: 34
    • View Profile
Re: JIT compiler revisited for Fluffos/dgd
« Reply #1 on: March 26, 2020, 09:54:09 am »
In August 2019, I finally finalised the JIT compiler for DGD. It is implemented as an extension module, so that it will work with both DGD and Hydra, and indeed makes use of LLVM. It does not yet work on Windows, but should work on any other platform with LLVM support; it was tested on i386, x86_64 and ppc.

The extension starts and then communicates with an external program, which decompiles LPC bytecode to LLVM IR, and then uses clang to create a native shared object which the extension can load into DGD's memory. LPC bytecode files, LLVM IR files and shared object files are all cached, so that this translation needs to happen only once.

Keeping the LPC=>LLVM decompiler separate from DGD avoids issues with possible memory leaks and crashes, though of course an error in the bytecode translation could still be fatal. While translating a LPC bytecode image to a shared object takes much longer than an inline JIT compiler would, it runs completely independently from DGD, which is never delayed. Even loading the shared object into DGD's runtime memory is handled by a separate thread.

DGD: https://github.com/dworkin/dgd
LPC extension modules: https://github.com/dworkin/lpc-ext