-
Notifications
You must be signed in to change notification settings - Fork 9
In Process Sandbox
To run the compiled code quickly, Jeed loads it into the current JVM process. Since the submitted code is untrusted, Jeed takes measures to confine each task, securing the process and other confined tasks from it. There are three categories of concerns:
- Confined tasks must not be able to perform privileged operations to manipulate the host machine.
- Confined tasks must completely stop after reaching the configured timeout.
- Confined tasks must not be able to ruin the environment for other tasks.
TODO!
TODO!
TODO!
Classes from the classpath—the Java standard library and other libraries we use—are shared between all code running in the JVM process. Changes in their static fields can affect all tasks. This is a severe problem when those static fields refer to threads or thread groups. The standard Kotlin coroutines dispatcher keeps a pool of threads and distributes work among them. In the best case, the pool gets created inside a sandbox and is ruined when that confined task is done; in the worst case, untrusted code gets dispatched to unconfined threads used by Jeed itself.
To give different confined tasks independent sets of static variables, we reload the problematic library classes, isolating the confined tasks' copies from each other. When asked for a class that is set to be isolated, our sandboxed classloader defines a new copy instead of getting the existing Class object from the parent classloader. The bytecode for reloaded classes is cached across all tasks for speed, but every confined task gets a different live Class based on it. To ensure that confined tasks can be shut down reliably, reloaded classes are subject to the same bytecode transformation (described below) as untrusted classes.
We always isolate the coroutines libraries so that coroutines can run securely. If you have additional libraries on your classpath that need to be reloaded, you can add them to the isolatedClasses of the ClassLoaderConfiguration.
The Java SecurityManager is an excellent start, preventing untrusted threads/classes from invoking privileged methods, but unfortunately not sufficient to address all concerns.
After the timeout for a confined task expires, we call stop on all the threads in the thread group. This method is generally advised against and deprecated, but it is the only way to stop a thread without its cooperation. It works by fabricating a ThreadDeath error and throwing it in the target thread. There is nothing special about ThreadDeath as a throwable: it can be caught by catch blocks and will trigger finally blocks, so the thread is not actually immediately killed. Malicious code can simply ignore the exception and continue running. The security manager is not notified when methods catch exceptions. To gain control over them, we need the cooperation of the thread, so we have to change the bytecode, creating a new sandboxed classloader with a transformed version of every untrusted class.
The ASM framework allows us to examine and manipulate classes element by element. Each visitor receives calls from upstream (the ClassReader for the input bytecode, or the previous visitor in the chain) and can intercept or modify aspects of the class before passing the information to the downstream visitor toward the output ClassWriter. Our SandboxingMethodVisitor is responsible for most of our transformation. Before any of the labels that control can be nonlinearly transferred to are visited, our visitor is notified of try-catch blocks with visitTryCatchBlock. Each try-catch or try-finally is represented in bytecode as a range of protected instructions (try block), the class of exception to catch, and the handler position to jump to if the exception occurs in the protected range. (Finally blocks are implemented as try-catch blocks without a filter, so they are entered for all exceptions so they can run the finally code before rethrowing the exception. Interestingly, each finally block appears twice in the bytecode, once for successful completion and once for if an exception is thrown.) If the exception filter is broad enough to catch an exception type the task shouldn't be allowed to survive, our visitor records the handler label as in need of rewriting. When such a label is visited with visitLabel, the visitor prepares to rewrite it, but does not do so until the following visitFrame. That way, the new instructions will be covered by the stack map entry for the catch/finally block where the caught exception was pushed onto the stack.
Immediately at the beginning of the suspicious catch/finally block, the visitor inserts a call to the checkException function of our RewriteBytecode object, passing the caught exception. The exception object at the top of the stack is duplicated so that the original catch/finally code after the call still has access to it. This makes the operand stack one item taller, so the visitor's visitMaxs function specifies a maximum stack height one greater than the original to make sure there's space. (ASM's ClassWriter can calculate the maximums for us, but specifying them ourselves is faster.) At runtime, our checkException function checks whether the type of the throwable is unsafe, i.e. should be unsurvivable. If so, it rethrows the throwable, unwinding the call stack to prevent the untrusted class from running more code. This function also helps accelerate shutdown by always throwing if the timeout has been exhausted.
We always consider Error and subclasses, including ThreadDeath, to be unsafe. You can add additional unsafe throwable types to unsafeExceptions of your ClassLoaderConfiguration.
In Java, it is impossible to write a try-catch block that "handles" an exception by directly retrying the try block from the top. In bytecode, however, the handler label for a try-catch block is allowed to be in the protected "try" region. The Java compiler sometimes produces such try-catch entries near synchronized blocks for unclear reasons. Usually they don't cause a problem because no exception ever gets thrown in them, but if one does due to our thread termination, it will be thrown and caught infinitely, freezing or even crashing the JVM!
Clearly that is a problem, so such try-catch entries must be removed entirely. Unfortunately ASM requires that visitTryCatchBlock is called before the start, end, and handler labels are visited with visitLabel, so SandboxingMethodVisitor could not by itself determine whether a try-catch block is bad until it has already been passed along. We therefore do a "preinspection" of the same bytecode with another class visitor that visits methods with PreviewingMethodVisitor. This method visitor doesn't pass anything toward a ClassWriter; it just records what it's told by the ClassReader. It keeps track of the order in which try-catch entries and labels are visited. Once the method has been completely visited, the visitor can cross-reference the try-catch entries with the labels to see which try-catch blocks handle exceptions by jumping back inside the protected region. Fortunately try-catch entries have a specific order in the bytecode, so the entry visited first in one read is also visited first for another read. The indexes of bad try-catch entries are recorded for use by the SandboxingMethodVisitor, which removes the offending entries by not forwarding the corresponding visitTryCatchBlock calls downstream.
Objects can override finalize to perform final cleanup before their storage is freed by the garbage collector. This feature is, as the Javadoc now says, "inherently problematic." Finalizers run on a dedicated thread outside any of our thread groups, and by the time they run the untrusted object's confined task might have completed and been forgotten by Jeed, so they would allow untrusted code to perform arbitrary operations. They are therefore removed: our sandboxing class visitor simply does not pass finalize() methods downstream.
Every Java object has an associated monitor that can be used for locking/synchronization with synchronized blocks. Since many objects, particularly Class objects, are accessible by all confined tasks, synchronization is another way for tasks to interact with each other even when they should be independent. More concerningly, the monitorenter bytecode instruction used to enter a synchronized block cannot be interrupted by calling stop on the thread if it is waiting for the monitor to become available.
We therefore must rewrite uses of objects' intrinsic monitors to use task-specific monitors in a way that can be terminated by thread death. Our MonitorIsolatingMethodVisitor, which SandboxingMethodVisitor derives from, transforms monitorenter and monitorexit instructions into calls to the enterMonitor and exitMonitor functions of our RewriteBytecode object by sending a method call instruction to the downstream visitor instead of the monitor instruction it received from upstream. These functions simulate those JVM monitor instructions using a ReentrantLock. Each monitor object in each sandbox is mapped to a different ReentrantLock, so different confined tasks cannot interfere with each other's synchronization.
There are five synchronization-related methods defined on Object, all of which rely on the current thread owning that object's intrinsic monitor. Since intrinsic monitors are no longer used by synchronized blocks, we must rewrite calls to those five methods as well. When SandboxingMethodVisitor is told of a method call instruction for one of those five methods with visitMethodInsn, it produces a call to the corresponding condition* method of RewriteBytecode instead. (Conveniently, the stack for a object.instanceMethod(a, b) call is the same as for SomeClass.staticMethod(object, a, b).) Those functions simulate the intrinsic condition methods using a Condition associated with the appropriate ReentrantLock.
Methods with the synchronized modifier automatically synchronize on the intrinsic monitor of the instance or, for static methods, Class object. To correctly simulate this lock, we must ensure that the equivalent lock is held for the duration of the method body. Our sandboxing class visitor sends two methods downstream for each synchronized method received from upstream:
- The original method, transformed by
SandboxingMethodVisitor, with$syncbodyappended to its name, itssynchronizedmodifier removed, and its access changed to private. - A wrapper method with the original name and non-
synchronizedmodifiers that calls the original, now-private method while holding the appropriate lock.
The wrapper method is generated by emitSynchronizedBridge. It preserves parameter names and modifiers by using those from the preinspection of the original method. The generation code looks complicated, but the resulting method is simple. It enters the monitor (by emitting a monitorenter instruction transformed into a method call by the downstream MonitorIsolatingMethodVisitor), enters a try block, calls the original method passing all of its own parameters, exits the monitor with monitorexit, and returns the result of the original method (if any). If the original method throws for any reason, the handler likewise exits the monitor then re-throws the exception. Because the generated wrapper methods never call Object condition/synchronization methods, MonitorIsolatingMethodVisitor is not responsible for transforming such calls, only JVM monitor instructions.