Jactl Continuations and Virtual Threads in Java 8

Introduction Jactl is a secure, embeddable scripting language for Java applications. When I first started developing Jactl, I wanted a scripting language that compiled to bytecode for optimum performance, was secure so applications could control exactly what scripts could and couldn't do, and most of all, did not block the execution thread when long-running, blocking operations were performed. When I began developing Jactl, Java 21 and Virtual Threads did not exist, and event-driven, reactive applications (such as ones based on Vert.x) were the way in which high-throughput Java applications were written. I also had a need for a scripting language that would work on applications still stuck on Java 8 or Java 11. noteNow, with the later versions of Java, Java applications that want to use Virtual Threads rather than adopting an event-based architecture can choose to disable the built-in Jactl async mechanism described here by configuring the JactlContext.async(false) flag. A reactive application is one where a pool of event-loop threads process events from a queue. The golden rule is that events should never block as this will pause one of the event-loop threads, preventing it from processing any more events until that blocking operation completes. A blocking operation is one where the thread is no longer actively processing code but is waiting for the result of an operation such as a database request or a remote procedure call. If blocking operations can occur on an event-loop thread, you will eventually see situations where all threads are waiting for long-running operations and no events are being processed. I wanted a scripting language that could be invoked from an event-loop thread but, when it performed any blocking operation, it would somehow save its state and return, freeing up the thread to process further events. When the result of the long-running operation was available the script would be resumed from the point where it left off and continue with its processing. In Java 21 and later, Virtual Threads provide the same functionality - they preserve the call stack with all the local variables and allow the thread to continue performing other work and when the blocking operation is complete the call stack is restored and the program continues from where it left off. The goal was only to save the execution state of the Jactl code, not the state of the Java code that was invoking a Jactl script. Since the Java application is event-based, the script will complete as a new event on an event-loop thread and a completion callback will be invoked once the script finishes that calls back into the Java application with the script result. The callback provided by the application can hold onto any state that the application needs. Continuations In Java 8, of course, there is no way to preserve the call stack, either in Java or in JVM bytecode, so I had to use a different mechanism to achieve the same end. Imagine that we have a script that needs to invoke a function (or method) that performs a long-running operation. For the sake of the example, let's assume that the function needs to perform sleep() for some period of time before it does something else. There will be a Java call stack with a stack frame for each nested method call and then the rest of the stack will be Jactl stack frames, one for each nested Jactl function call, with the topmost stack frame being the stack frame for the sleep() function itself. Each stack frame tracks where in the code the function invocation occurs, along with the values for its local variables: To capture the execution state, I figured that the easiest thing to do would be to throw an exception at the start of a long-running operation like sleep() and generate code in each Jactl method/function that catches the exception, saves its state, and throws a new exception that is chained to the one it just caught. I called the class for the exception being thrown Continuation since a continuation is a representation of the execution state of a program. The implementation of the sleep() function will then look something like this: public static Object sleep(long timeMs) { Continuation continuation = new Continuation(); scheduleEvent(timeMs, () -> continuation.continueExecution()); throw continuation;} As this exception unwinds the call stack, the generated code for each Jactl stack frame catches it, creates its own Continuation to record where it was up to (the location of the call it was waiting on) and the values of its local variables, and then throws a new Continuation chained to the one it just caught. By the time the exception reaches the bottom of the Jactl call stack, the original call stack has been replaced by a chain of Continuation objects, one per frame, that together capture the entire execution state of the script: noteFor anyone who has done performance tuning of Java applications, the idea of throwing exceptions instantly makes one think of the cost involved, but in reality, the cost of throwing an exception is mostly in the generation of the stack trace that goes along with it. As long as you throw an exception that does not fill in the stack trace, it is actually very efficient. Async Functions The Jactl compiler knows which global functions are functions that can perform long-running operations and can throw a Continuation object. These functions are called async functions and the Jactl compiler tracks which methods and functions invoke these async functions and marks these as async as well. This continues up the call chain so the compiler knows at any point in time whether a call could potentially throw a Continuation object, even if the built-in global function that is the one to actually throw the first Continuation is buried many levels within a set of nested calls. Invoking Async Methods/Functions When the compiler is generating code that invokes a function that has been flagged as async, it wraps the call in a try/catch that catches any Continuation that is thrown. The code for the catch block creates a new Continuation object and stores a MethodHandle and a location inside it. The MethodHandle points to the current function and the location is a logical location that records where in the current function the call to the async function that threw the Continuation occurred. As well as the MethodHandle and location, the compiler generates code to also store the values of the local variables in scope at the time and any values that are currently on the local stack. There are two separate arrays used: a long[] that is used for local variables and stack values that are primitives, and an Object[] that is used for all other types. Every async function is implicitly passed a Continuation object as its first argument. The first time through, the argument is null, but if the function was suspended due to a long-running operation and is then later resumed, it will be reinvoked with the Continuation it originally threw when it was suspended. The generated code checks if the continuation argument is non-null and if so, uses the location within the `Continuation object to work out where in the function to jump to in order to continue execution. Here is some pseudocode that shows what the generated code from the compiler for a function that invokes another async function might look like: static MethodHandle processOrderHandle = MethodHandles.lookup().findStatic("processOrder");Object processOrder(Continuation cont, ...) { Order order; Widget widget; int count; if (cont != null) { // Resume from where we left off after restoring any local variables switch (cont.location) { case 0: // Restore locals order = cont.objArr[0]; widget = cont.objArr[1]; count = cont.longArr[0]; goto LOCATION_0; case 1: ... goto LOCATION_1; } } // Code for the function ... try { checkInventory(widget, count); } catch (Continuation c) { throw new Continuation(c, processOrderHandle, 0, // the location new long[]{ count }, new Object[]{ order, widget }); } LOCATION_0: ...} Note that the Continuation constructor chains itself to the just caught Continuation so that the chain starts with the Continuation from the top of the stack. Resuming Execution Once a long-running operation completes, the initial Continuation object in the chain is resumed by invoking its continueExecution(Object result) method. This method extracts the MethodHandle and calls it, passing in the Continuation as previously described so that the function can restore any local variables and work out where to continue from. Since the call stack no longer matches the original call stack, when the function returns, instead of it returning to the original parent function, it will return to the Continuation.continueExecution() method which then extracts the next Continuation object in the chain and calls its MethodHandle. This continues until there are no more Continuation objects in the chain and the registered completion from the application is invoked with the final result. Invoking a Subsequent Async Function While walking the chain of Continuation objects and resuming them, another async function may be invoked that throws a new Continuation object for a new long-running operation, possibly from a function many nested calls further in. When this happens, we take the new chain of continuations and add the remainder of the existing chain to the end of that chain: When the new long-running operation completes and its Continuation chain is resumed, the chain now consists of all the new continuations as well as the remaining old ones. It will first resume each of the new continuations and then continue on to the remaining ones in the old chain. Checkpointing Execution State