Using this in AWS Lambda

[milantimotijevic]

Is there a way to run this inside a NodeJS AWS Lambda? I am able to install the .deb package on my own Ubuntu, but is there some kind of JS library or a file that could be run from within a Lambda?

This is what I tried so far:

1. Created a docker container using this Dockerfile (this is the setup AWS Lambda uses)
   FROM public.ecr.aws/lambda/nodejs:12
   RUN mkdir -p "/var/task"
   CMD [ "index.handler" ]

2. sshed into that container

3. yum install wget

4. wget https://github.com/KhronosGroup/KTX-Software/releases/download/v4.1.0-rc2/KTX-Software-4.1.0-rc2-Linux-x86_64.rpm

5. yum install KTX-Software-4.1.0-rc2-Linux-x86_64.rpm

And I ended up with the following error output:

sh-4.2# yum install KTX-Software-4.1.0-rc2-Linux-x86_64.rpm Loaded plugins: ovl Examining KTX-Software-4.1.0-rc2-Linux-x86_64.rpm: ktx-software-4.1.0-1.x86_64 Marking KTX-Software-4.1.0-rc2-Linux-x86_64.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package ktx-software.x86_64 0:4.1.0-1 will be installed --> Processing Dependency: libm.so.6(GLIBC_2.27)(64bit) for package: ktx-software-4.1.0-1.x86_64 amzn2-core | 3.7 kB 00:00:00 --> Processing Dependency: libstdc++.so.6(GLIBCXX_3.4.26)(64bit) for package: ktx-software-4.1.0-1.x86_64 --> Processing Dependency: libktx.so.4()(64bit) for package: ktx-software-4.1.0-1.x86_64 --> Finished Dependency Resolution Error: Package: ktx-software-4.1.0-1.x86_64 (/KTX-Software-4.1.0-rc2-Linux-x86_64) Requires: libm.so.6(GLIBC_2.27)(64bit) Error: Package: ktx-software-4.1.0-1.x86_64 (/KTX-Software-4.1.0-rc2-Linux-x86_64) Requires: libstdc++.so.6(GLIBCXX_3.4.26)(64bit) Error: Package: ktx-software-4.1.0-1.x86_64 (/KTX-Software-4.1.0-rc2-Linux-x86_64) Requires: libktx.so.4()(64bit) You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest

Is there something wrong with the .rpm binary, or am I doing something wrong? I can easily install the .deb binary on Ubuntu, but unfortunately that's not the distribution I need it on

[MarkCallow]

I don't know anything about "AWS Lambda". Keep that in mind when you read the rest of this.

A wasm version of libktx is available in KTX-Software-4.1.0-rc2-Web-libktx.zip. The wasm module is the full library including encoders. The package also includes a JS wrapper that supports opening of KTX files and uploading of the textures to WebGL. I'd be very happy to accept a PR to expose more of the libktx functionality to JS via the wrapper.

[milantimotijevic]

Is there some documentation for this JS wrapper? The only thing I need to be able to do is convert .jpg (and similar) files into .ktx. If I could do that with the JS library it would solve so much for me

[MarkCallow]

The documentation is here. Currently the libktx wrapper does not support what you want to do. Also jpg, png or similar decoders are out of scope for libktx. However all web browsers, so presumably node.js too, have jpg and png decoders. There are 2 ways to accomplish what you want: extend the JS wrapper and write a JS app or write a C++ app and compile it with Emscripten.

JS

• Extend the wrapper to support ktxTexture[12]_Create, ktxTexture_SetImageFromMemory and one or more of the ktxTexture_WriteTo* functions. The ktxTexture_SetImageFromMemory wrapper function will need to accept data from an ImageBitmap or ImageData. I am not familar with these DOM types so I don't know which is more appropriate here.
• Use JS to decode a jpg image to an ImageBitmap or ImageData. Construct a ktxTexture2 that matches the image parameters and pass the ImageBitmap or ImageData to ktxTexture_SetImageFromMemory of the constructed ktxTexture2.
• Call your chosen ktxTexture_WriteTo function to get the KTX file and figure out how to get the KTX file to its final destination.

C++/Emscripten

• Write some JS to decode a jpg image to an ImageBitmap or ImageData.
• Use the Emscripten JS interfaces to call that code from your C++ app then copy the resulting Image{Bitmap,Data} into the Emscripten heap.
• Call ktxTexture[12]_Create to construct a texture matching the image parameters.
• Call ktxTexture_SetImageFromMemory to upload the image data.
• Call the ktxTexture_WriteTo* function of your choice and figure out how to get the KTX file to its final destination.

[MarkCallow]

Is there something wrong with the .rpm binary, or am I doing something wrong? I can easily install the .deb binary on Ubuntu, but unfortunately that's not the distribution I need it on

I never answered your original question. Sorry. My previous answer was based on the assumption you wanted JS code to run in node.js.

If creating the texture within the Docker container meets your need then the toktx tool should do what you want, provided we can figure out the installation issue.

The 3 Linux packages are all built on Ubuntu Focal - that's the only Linux available in the CI service - and are just different packages of the same build. What OS and OS version is the Docker container running? Have you tried the .deb or .tar.bz2 packages?

You could try --skip-broken as suggested in the error message. I had similar errors with some MacPorts packages after upgrading to a new macOS & MacPorts release. It turns out the packages had specific glib versions wired into them but they were perfectly happy to run with the later glib installed by the updated MacPorts.

[MarkCallow]

In response to #649.

I am sorry you are still having problems @milantimotijevic. I do not have an environment where I can test the rpm package. It is built by cpack from the same package configuration as the .deb and .tar packages so it almost certainly contains the same components.

Have you tried --skip-broken? However I don't understand why we see this in the output which --skip-broken is unlikely to fix.

Error: Package: ktx-software-4.1.0-1.x86_64 (/KTX-Software-4.1.0-rc3-Linux-x86_64) Requires: libktx.so.4()(64bit)

I am confident libktx.so is included in the package. If you try --skip-broken perhaps yum will install what it has and then you can ls /usr/local/lib/*ktx* and see if the versioned libktx.so.4` link is there and if it points to a a real file.

What does the other command do that yum is suggesting you try rpm -Va --nofiles --nodigest?

[milantimotijevic]

Hi Mark,

Sorry for duplicating the question. For a while I used a Debian image which got the job done, but it turns out I have to make this work in AWS Lambda after all. As a result I completely forgot that I had this question still open.

Anyway, if I try the --skip-broken command, it still doesn't work as it ends up with an error message that a library is missing. That other command (rpm -Va --nofiles --nodigest) does nothing as far as I'm concerned, it produces no output, no error message.

Is there any chance you can test this out on your end if I provide you with the exact same setup I have? I'll go ahead and describe it here (let me know if you would like a repo instead):

1. You need to install Docker on your machine and ensure the daemon is running
2. Create a folder called docker-lambda and place the following files into it:

Dockerfile
FROM amazon/aws-lambda-nodejs:12
COPY . .
CMD [ "functions.helloLambda" ]

functions.js
exports.helloLambda = async (event) => {
const response = {
isBase64Encoded: false,
statusCode: 200,
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
message: "Containers on lambda!",
}),
};
return response;
};

KTX-Software-4.1.0-rc3-Linux-x86_64.rpm (basically, your rpm binary)

3. Open a terminal and cd into this folder and run the following command:
   docker build -t docker-lambda .
4. Then run this:
   docker run --name docker-lambda -p 9000:8080 docker-lambda
5. The command will give you an output looking something like this: 08 Nov 2022 08:17:33,256 [INFO] (rapid) exec '/var/runtime/bootstrap' (cwd=/var/task, handler=) - it means the container is running
6. Open another terminal (with the previous one still running) and run:
   docker exec -it docker-lambda sh
7. Once inside, simply run:
   yum install KTX-Software-4.1.0-rc3-Linux-x86_64.rpm

You'll get the same error message as I did and you'll be able to try the --skip-broken flag, etc. I would greatly appreciate your help. Perhaps someone else is also having a similar problem and in solving it for me you'll have solved it for them as well.

Let me know if you can try this out or if you want me to provide a link to a repo instead.

Thanks in advance!