Passing arrays?

[togekk]

Hi,
I noticed that arrays can only been manipulated within wasm modules when I'm using the playground. Is it possible to pass arrays (either typed or dynamic ones) to and from javascript at the moment? Thanks!

[dcodeIO]

Indirectly, yes. WASM has only four basic types (i32, i64, f32, f64), and what you are passing to/from JS is the offset (i32) of the array structure in memory.

[togekk]

So it has performance overhead like passing strings, right? Would it be faster if I use a loop in JavaScript or AssemblyScript to pass the numbers in the array one by one?

[dcodeIO]

Depends, calling in and out of WASM has some overhead as well. If performance is a concern, I'd suggest to create (and then reuse) a typed array view of the memory region in question on the JS side, and use that to read respectively write the values.

[togekk]

I've managed to use the loader to share my Float64Array between JS and WASM. Another question is, I couldn't get the correct offset through bitwise operators as here mentioned. So I had to manually put a value in the first of the array from WASM to get the offset from JS. For example, "34" for Int32Array and "17" for Float64Array. But these offsets didn't seem to be regular, so I had to do the test again when creating a new array.

Moreover, when I re-declared the same array name with a different length, the offset shifted again. So while writing on the WASM side, I couldn't get the corresponding array from JS anymore. For the normal arrays it got more complex. Is there any simple and safe ways to do this?

[dcodeIO]

When returning a Float64Array from WASM, the JS-side receives the offset of the respective structure in memory. Arrays have one additional level of indirection to their backing ArrayBuffer, as explained here, so that might be where the random offset you described comes from. What you want to do there is to read the buffer offset first, and then use that to access the data of the backing buffer.

[togekk]

I've found out a safer way now. Here are the steps:

1. On the JS side, use allocate_memory to get every pointer for every items, and write them into an array:

for (let i = 0; i < length; i++)
       ptr_array[i] = module.allocate_memory(1);

2. Write the item values into the ArrayBuffer using the bitwise operators:

array.forEach((item, index) => {
       const ptr = ptr_array[index];
       module.F64[ptr >>> 3] = item;
});

3. Send the starting pointer while calling all the WASM function:

module.myFunction( ptr_array[0], array.length, arg3..... );

4. On the WASM side, get the first pointer, the following items are certain bytes after it depending on which kind of type array you're using. For example, Float64Array uses 8 bytes each. It's described here well. So we use "load" and "store" function to read and write the memory.

for (let i = ptr; i < ptr + len * 8; i += 8) {
       let item = load<f64>(i);
       item = item / 300;
       store<f64>(i, item);
}

5. Back on the JS side, simply use the bitwise operators again to retrieve those items:

console.log(module.F64[ptr_array[index] >>> 3]);

And all is done! Remember to free/reset memory after that. Use 3 typed arrays in my project without any problem :)

But I still couldn't figure out how to get the typed array offset from the WASM side. The document link provided looks like the HEX code from the WASM file? I still have no luck to find "17" and "27" these 2 offsets as mentioned above...

[dcodeIO]

More precisely, what you get back is a pointer to the Float64Array structure in WASM memory. That structure does not contain the actual data, but another pointer to an ArrayBuffer that holds the data. So, the first 4 bytes at the offset returned are the pointer to the ArrayBuffer, the next 4 bytes is the byteOffset into that ArrayBuffer, the next 4 bytes is the byteLength of that ArrayBuffer. What you'd want to do is:

1. Retrieve the pointer to the Float64Array by returning it from WASM
2. Read the first 4 bytes at that pointer to obtain the pointer to the ArrayBuffer, bytes 4-7 to obtain the byteOffset and bytes 8-11 to obtain the byteLength (the last is only necessary if you'd like to do a range check).
3. Read the value from the address of the ArrayBuffer (as obtained in step 2). The data of an ArrayBuffer starts at offset 8, then add byteOffset + index * elementByteSize (here: 8).

╒══ TypedArray ═╕
 7 6 5 4 3 2 1 0  bits
├─┴─┴─┴─┴─┴─┴─┴─┤
│    .buffer    │ points at the backing ArrayBuffer
├───────────────┤
│  .byteOffset  │ offset in bytes from the start of .buffer
├───────────────┤
│  .byteLength  │ length in bytes from the start of .buffer
└───────────────┘

╒═ ArrayBuffer ═╕
 7 6 5 4 3 2 1 0  bits
├─┴─┴─┴─┴─┴─┴─┴─┤
│  .byteLength  │ number of bytes
├───────────────┤
│       0       │ free space due to alignment
├───────────────┤
│      ...      │ N=byteLength bytes of data
└───────────────┘

(updated the wiki page accordingly)

[togekk]

So all the pointers must be retrieved from the I32/U32 array view (or create an Int32Array/Uint32Array view on your own), and the code will look like this:

WASM side:

export function show_array(): Float64Array {
  let array: Float64Array = new Float64Array(5);
  array[0] = 99.88;
  array[1] = 88.77;
  array[2] = 77.66;
  array[3] = 66.55;
  array[4] = 55.44;
  return array;
}

JS side:

const array_ptr = module.show_array();
const buffer_ptr = module.I32[array_ptr >>> 2];
const byte_offset = module.I32[array_ptr + 4 >>> 2];
const byte_length = module.I32[array_ptr + 8 >>> 2];
const element_byte_size = 8;
const array_start_ptr = 8 + byte_offset;

for (let i = array_start_ptr ; i < array_start_ptr + byte_length; i += element_byte_size)
    console.log(module.F64[buffer_ptr + i >> 3]);

Thanks for all the answers and patience, now it all works :)

[msbasanth]

@togekk even I have similar issue where I need to pass and return a u8[].

When converted I got,
function decodeRLE(encodedBuffer: u32): u32;

even though I have my AS function signature,
function decodeRLE(encodedBuffer: u8[]): u8[]

How can I retrieve the argument from passed array pointer? Similarly how can I retrieve full u8[] (UInt8Array) from returned array pointer.

I tried the snippet you shared I am getting module.I32 as 'undefined'. Should I do something to get I32 array view?

[inkeliz]

It's not possible to get the pointer of Buffer pointer directly in AssemblyScript? I would like to do something like:

let x = new Uint8Array(1000);

export function Pointer() : u32 {
return pointerOf(x[0])
}

In that case the host can receive the pointer and manipulate the memory directly?

[MaxGraey]

let x = new Uint8Array(1000);

export function Pointer() : usize {
  return x.dataStart;
}

or

export function Pointer() : usize {
  return changetype<usize>(x);
}

It depends on what you actually need. In first case it will be pointer to raw data. In second case it will be pointer to Uint8Array's object