ASAN: global-buffer-overflow in functions Sleef_sinf4_u10, Sleef_cosf4_u10, Sleef_tanf4_u10 on s390x

[AlekseiNikiforovIBM]

When using functions Sleef_sinf4_u10, Sleef_cosf4_u10, Sleef_tanf4_u10 on s390x, out of bounds read is attempted.

Steps to reproduce on s390x:

$ git clone https://github.com/shibatch/sleef
$ mkdir sleef/build
$ cd sleef/build
$ CFLAGS="-O0 -g -ggdb -march=z15 -mvx -mzvector -fsanitize=address" CXXFLAGS="-O0 -g -ggdb -march=z15 -mvx -mzvector -fsanitize=address" LDFLAGS="-fsanitize=address" CMAKE_BUILD_TYPE=Debug cmake -DBUILD_TESTS=OFF ..
$ make
Or ninja if it's available
Now prepare source file:
$ cat sleef_crash.c
#include <stdio.h>
#include <math.h>

#include <sleef.h>

int main(int argc, char **argv)
{
        __attribute__((vector_size(16))) float in = { INFINITY, -INFINITY, NAN, -NAN};
        __attribute__((vector_size(16))) float out;

        out = Sleef_cosf4_u10(in);

        printf("result = %f, %f, %f, %f\n", out[0], out[1], out[2], out[3]);

        return 0;
}
$ gcc -O0 -g -ggdb -fsanitize=address -march=z15 -mvx -mzvector -Iinclude -Llib -lsleef sleef_crash.c -o sleef_crash
$ LD_LIBRARY_PATH=./lib ./sleef_crash
Just ensure that no libsleef.so* is available in system

Result:

=================================================================
==2043801==ERROR: AddressSanitizer: global-buffer-overflow on address 0x03ffa03d3470 at pc 0x03ff9f80d7ac bp 0x03fff72f9a10 sp 0x03fff72f9a00
READ of size 4 at 0x03ffa03d3470 thread T0
    #0 0x3ff9f80d7ab in vgather_vf_p_vi2 /home/user/sleef/src/arch/helpers390x_128.h:127
    #1 0x3ff9f80d7ab in rempif /home/user/sleef/src/libm/sleefsimdsp.c:553
    #2 0x3ff9f80d7ab in Sleef_cosf4_u10vxe2 /home/user/sleef/src/libm/sleefsimdsp.c:1024
    #3 0x3ffa03c460d in disp_cosf4_u10 /home/user/sleef/build/src/libm/disps390x_128.c:52
    #4 0x3ffa03c4657 in Sleef_cosf4_u10 /home/user/sleef/build/src/libm/disps390x_128.c:52
    #5 0x2aa16780cd9 in main /home/user/sleef/build/sleef_crash.c:11
    #6 0x3ff9ef29dd5 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #7 0x3ff9ef29eab in __libc_start_main_impl /usr/src/debug/sys-libs/glibc-2.37-r1/glibc-2.37/csu/libc-start.c:360
    #8 0x2aa16780b0f  (/home/user/sleef/build/sleef_crash+0xb0f)

0x03ffa03d3470 is located 0 bytes to the right of global variable 'Sleef_rempitabsp' defined in '/home/user/sleef/src/libm/rempitab.c:986:34' (0x3ffa03d2e00) of size 1648
SUMMARY: AddressSanitizer: global-buffer-overflow /home/user/sleef/src/arch/helpers390x_128.h:127 in vgather_vf_p_vi2
Shadow bytes around the buggy address:
  0x10007ff407a630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10007ff407a680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00[f9]f9
  0x10007ff407a690: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a6a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a6b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a6c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10007ff407a6d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==2043801==ABORTING

It looks like in function vgather_vf_p_vi2 p is Sleef_rempitabsp, and vi2 contains values 412. The problem is that Sleef_rempitabsp only has 412 elements, i.e. only indexes from 0 till 411 are valid, 412 is not a valid index.