Trusted OS and Applet execution

Let us see how two concurrent TamaGo unikernels can be scheduled in domains of different privilege level.

Warning

While in our example the Trusted Applet is also a TamaGo unikernel, it can be any application capable of running in user mode and implementing GoTEE API, such as freestanding C or Rust programs.

The monitor package provides privileged support for instantiating bare metal Go in user mode, while the applet package provides Trusted Applet support.

We call the privileged unikernel the "Trusted OS", running in privileged modes (PL1) while the unprivileged (PL0) unikernel is known as the "Trusted Applet", both are meant to run in TrustZone Secure World.

Let us assume that our Trusted OS already has the Trusted Applet ELF binary in memory, we can Load. it in an execution context as follows:

ta, err := monitor.Load(taEntry, taMemory, true)

The first argument represents the applet entry point, the second argument is a DMA region set with Start and Size variables to the physical memory start address and size that we want to assign to the Trusted Applet, the GoTEE example implements a basic memory layout.

It is important to note that the true boolean argument requests a secure execution context (TrustZone Secure World) for the applet.

The returned ta is a monitor.ExecCtx structure which holds the entire execution context state.

We can invoke Print to see the initial applet state:

 r0:00000000  r1:00000000  r2:00000000  r3:00000000
 r4:00000000  r5:00000000  r6:00000000  r7:00000000
 r8:00000000  r9:00000000 r10:00000000 r11:00000000 cpsr:00000000 (Unknown)
r12:00000000  sp:00000000  lr:00000000  pc:8206dab8 spsr:000001d0 (USR)

As expected all user registers are blank at the beginning, while the program counter matches the entry point information found within the ELF binary by Load.

The Saved Program Status Register is initialized automatically with user mode when requesting a secure context.

Using TamaGo dma package the ELF is placed in physical memory between appletStart and appletStart+appletSize which is now reserved exclusively for this context and not directly mapped by the Trusted OS Go runtime.

Using TamaGo arm package ConfigureMMU, the memory has already been flagged with PL0+PL1 privilege levels, while the Trusted OS Go runtime remains PL1 only by default.

We can now Run the applet context to schedule its execution:

ta.Run()

Here we can see the GoTEE example trusted applet running concurrently with the Trusted OS:

00:00:00 SM starting mode:USR ns:false sp:0x00000000 pc:0x8206dab8
00:00:00 applet tamago/arm (go1.16.4) • TEE user applet (Secure World)
00:00:00 applet obtained 16 random bytes from monitor: 2745e86bd483211a7de9672072b01049
00:00:00 applet requests echo via RPC: hello
00:00:00 applet received echo via RPC: hello
00:00:00 applet will sleep for 5 seconds
00:00:01 applet says 1 mississippi
...
00:00:05 applet says 5 mississippi

Note that on a single core context "concurrent" means that CPU time is yielded back to the Trusted OS in PL1 until one of the following conditions arises:

• the applet triggers an error exception (e.g. invalid memory access)
• the applet triggers a supervisor exception (e.g. system call)
• a hardware interrupt is received

When either of these conditions apply the Trusted OS Go runtime will get the chance to:

1. schedule goroutines using runtime.Gosched()
2. serve GoTEE system calls using package syscall
3. handle exceptions using TamaGo arm package handlers

The Run function takes care of all of this automatically, the applet will terminate if the Exit GoTEE system call is invoked.

The GoTEE default system call handler can be overridden by setting the overriding the execution context handler with a custom function.

Additionally the TamaGo default exception handler can be overridden using the arm.SetVectorTable function.

The combination of GoTEE and TamaGo APIs give flexibility in implementing a Trusted OS with arbitrary supervision for the trusted applet.

The GoTEE example trusted applet triggers an invalid memory access to demonstrate its handling, this is what happens:

applet is about to read secure memory at 0x90010000
   r0:90010000  r1:948220c0  r2:90010000  r3:00000000
   r4:00000000  r5:00000000  r6:00000000  r7:00000000
   r8:00000007  r9:00000044 r10:948000f0 r11:942cba81 cpsr:600001d7 (ABT)
  r12:00000000  sp:9484ff04  lr:94168868  pc:9401132c spsr:600000d0 (USR)
PL1 stopped mode:USR ns:false sp:0x9484ff04 lr:0x94168868 pc:0x9401132c err:ABT

It can be seen that the data abort exception is triggered by the trusted applet while attempting to read PL1 memory, the Trusted OS can inspect the applet state and take the desired action.

Next

➞ System Calls