[GR-34769] Stack depth more limited with musl static native image, how to increase?

[borkdude]

Describe the issue

I am using GraalVM 21.1.0 Java 11 CE to compile static binaries for linux using musl 1.2.2-1 (from Debian unstable). The stack depth is much more limited than a similar binary compiled without --static. I tried to increase the limit with "-H:CCompilerOption=-Wl,-z,stack-size=10485760" but to no avail.

Steps to reproduce the issue

Compile the following JavaRecur.java program and run it:

public class JavaRecur {

    public static void foo(int x) {
        System.out.println(x);
        foo(x+1);
    }

    public static void main(String [] args) {
        foo(0);
    }
}

javac JavaRecur.java
"$GRAALVM_HOME/bin/native-image" --no-server --no-fallback "--static" "--libc=musl" "-H:CCompilerOption=-Wl,-z,stack-size=10485760" JavaRecur

./javarecur

You will likely see that the stack depth is limited to around 2500 while with the non-static binary we can go up to around 260000.

Note that a basic C program like this:

#include <stdio.h>

void recur(int x)
{
  if (x % 10000 == 0)
    printf("%d\n", x);
  recur(x + 1);
}

int main()
{
  recur(0);
  return 0;
}

behaves pretty much the same with gcc and musl-gcc (without passing linker options to either):

$ gcc -o main main.c
$ ./main
0
10000
...
250000
260000
[1]    31779 segmentation fault  ./main

Similar output with musl-gcc -o main --static main.c.

Describe GraalVM and your environment:

• GraalVM version 21.1.0 JDK 11
• Linux
• Amd64

More details
See https://gist.github.com/borkdude/3a36ccc5dd186ac9f1a761b9b7b7cd36 for a stack trace that contains more info.
See https://gist.github.com/borkdude/8395ac49d7b90ebe4b9a911565304b3d for options passed to the linker printed with "-H:CCompilerOption=--verbose" and "-H:+TraceNativeToolUsage".

Workaround

Compile with -H:CCompilerOption=-Wl,-z,stack-size=10485760 (or higher stack size if needed)
and then start your main program in a thread instead.

This is surely a workaround, the main issue should still be addressed.

/cc @gradinac

[borkdude]

As discussed with @gradinac, it seems to be an issue with the stack size of the main thread. As a workaround you can set the main thread's stack size like this:

public class JavaRecur {

    public static void foo(int x) {
        if (x % 10000 == 0)
            System.out.println(x);
        foo(x+1);
    }

    public static void main(String [] args) throws InterruptedException {
        Runnable runnable =
            () -> { foo(0); };

        Thread thread = new Thread(null, runnable, "foo", 10000000);
        thread.start();
        thread.join();
    }
}

Compiled with musl this prints all the way up to 310000.

The unclear bit:

Just for my understanding... how come the main thread's stack size is different in musl binary than a glibc linked binary?

Aleksandar Gradinac  1 minute ago
that's the bit I'm unsure of - musl should always have a smaller stack than glibc by default (according to the musl official docs), but I'm not sure why the small C program didn't behave differently on the two libcs - I'll investigate it a bit further

[borkdude]

From Slack:

Aleksandar Gradinac 5 minutes ago
hmm, after removing stack overflow checks (edit in GraalVM, Michiel), the musl native-image runs to the expected recursion depth 😄 it seems like the problem lies in querying the main thread's stack top, quite possibly due to some difference between musl and glibc

Aleksandar Gradinac 5 minutes ago
I'll look into that a bit more later today to find where the difference is exactly

[wirthi]

According to our internal ticket GR-34769 this was fixed in October 2022.