ARM cross-compile does not use v8 snapshots #266

orangemocha · 2015-11-24T18:53:00Z

nodejs/node#3962 enabled snapshots in Node however this doesn't seem to work with the ARM cross-compiler, as seen in https://ci.nodejs.org/job/node-cross-compile/588/nodes=cross-compiler-pi1p/console

/home/iojs/build/workspace/node-cross-compile/nodes/cross-compiler-pi1p/out/Release/mksnapshot: 1: /home/iojs/build/workspace/node-cross-compile/nodes/cross-compiler-pi1p/out/Release/mksnapshot: Syntax error: word unexpected (expecting ")")

I changed the node-cross-compile to pass --without-snapshot to configure, so that we can continue running CI while we figure the best long-term approach.

A few questions:

Is it absolutely impossible to produce a snapshot on the build platform, if it's different from the target platform? Or is it just a limitation in the compiler?
Are ARM release builds coming out of the cross-compile job?
Can we live without snapshots for ARM?

/cc @nodejs/build

The text was updated successfully, but these errors were encountered:

bnoordhuis · 2015-11-24T18:57:49Z

Is it absolutely impossible to produce a snapshot on the build platform, if it's different from the target platform?

Only if you have an emulator. What the snapshot does is compile and execute JS to native machine code, then save the generated machine code to an image.

indutny · 2015-11-24T19:00:50Z

qemu?

indutny · 2015-11-24T19:06:42Z

Also a question how does V8 handle this? I'm sure that Google cross-compiles it for arm. cc @ofrobots

targos · 2015-11-24T19:32:00Z

There is some documentation about this here: https://github.com/v8/v8/wiki/Cross-compiling%20for%20ARM

indutny · 2015-11-24T19:34:47Z

@targos I'm afraid that this page is very likely to be outdated. V8 does not use scons for a very long time now.

indutny · 2015-11-24T19:37:52Z

@orangemocha is there any way to check that want_separate_host_toolset was 1 during our builds?

indutny · 2015-11-24T19:38:59Z

It looks like it was 0. This is very likely a cause of the problem.

indutny · 2015-11-24T19:39:45Z

Wait, are we cross-compiling there? Or is it an actual raspberry pi?

ofrobots · 2015-11-24T19:45:29Z

Looking output:

Building remotely on node-msft-cross-compiler-1 (cross-compiler-pi1p) in workspace /home/iojs/build/workspace/node-cross-compile/nodes/cross-compiler-pi1p

The job ran on https://ci.nodejs.org/computer/node-msft-cross-compiler-1/ which is described as "Ubuntu 14.04 LTS with Raspberry Pi cross compiler installed", so I would say this is a cross-compilation.

indutny · 2015-11-24T19:55:18Z

This is a config.gypi:

{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'arm_float_abi': 'hard',
                 'arm_fpu': 'vfpv2',
                 'arm_thumb': 0,
                 'arm_version': '6',
                 'asan': 0,
                 'gas_version': '2.24',
                 'host_arch': 'arm',
                 'icu_small': 'false',
                 'node_byteorder': 'little',
                 'node_install_npm': 'true',
                 'node_prefix': '/usr/local',
                 'node_release_urlbase': '',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_perfctr': 'false',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'python': '/usr/bin/python',
                 'target_arch': 'arm',
                 'uv_parent_path': '/deps/uv/',
                 'uv_use_dtrace': 'false',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 0,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_random_seed': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 0}}

For some reason host_arch is arm there...

ofrobots · 2015-11-24T19:59:28Z

Indeed. Odd.

orangemocha · 2015-11-24T20:01:08Z

Definitely cross-compiling. Better get the nitty gritty details from @joaocgreis, who set this up and also did some experiments with qemu.

ofrobots · 2015-11-24T20:02:02Z

I think @indutny might be right. The job seem to be doing a configure instead of configure --dest-cpu=arm. On my x64 box, this is the config.gypi I get:

lemongrab :: ~/src/node ‹master› % ./configure --dest-cpu=arm
creating  ./icu_config.gypi
{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'arm_float_abi': 'default',
                 'arm_fpu': 'vfpv2',
                 'arm_thumb': 0,
                 'arm_version': 'default',
                 'asan': 0,
                 'gas_version': '2.24',
                 'host_arch': 'x64',
                 'icu_small': 'false',
                 'node_byteorder': 'little',
                 'node_install_npm': 'true',
                 'node_prefix': '/usr/local',
                 'node_release_urlbase': '',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_perfctr': 'false',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'python': '/usr/bin/python',
                 'target_arch': 'arm',
                 'uv_parent_path': '/deps/uv/',
                 'uv_use_dtrace': 'false',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 0,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_random_seed': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 1}}

joaocgreis · 2015-12-07T04:30:16Z

I believe I have this working now and will be able to re-enable snapshots when nodejs/node#4117 lands. V8 can handle cross compiling of snapshots if the {CC,CXX}_host variables are defined, by compiling the mksnapshot executable with the host compiler.

joaocgreis · 2015-12-09T16:11:11Z

nodejs/node#4117 has landed and CI is building snapshots for arm now: https://ci.nodejs.org/job/node-test-commit-arm-fanned/768/ . I will close this for now, please reopen if there are issues.

kikijhu · 2016-01-14T06:42:26Z

It's still failed when building v5.4.1 on ubuntu 14.04 for android

make[1]: ** [/home/kiki/node.js/node-v5.4.1/out/Release/obj.target/v8_snapshot/geni/snapshot.cc] Error 2
make[1]: Leaving directory `/home/kiki/node.js/node-v5.4.1/out'
make: ** [node] Error 2

{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'OS': 'android',
                 'arm_float_abi': 'default',
                 'arm_fpu': 'vfpv3',
                 'arm_thumb': 0,
                 'arm_version': '7',
                 'asan': 0,
                 'gas_version': '2.24',
                 'host_arch': 'arm',
                 'icu_small': 'false',
                 'node_byteorder': 'little',
                 'node_enable_v8_vtunejit': 'false',
                 'node_install_npm': 'true',
                 'node_prefix': '/usr/local',
                 'node_release_urlbase': '',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_perfctr': 'false',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'python': '/home/kiki/node.js/node-v5.4.1/android-toolchain/bin/python',
                 'target_arch': 'arm',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 0,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_random_seed': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 0}}

kikijhu · 2016-01-14T07:00:24Z

I have to modified android-configure to pass --without-snapshot to configure.

export TOOLCHAIN=$PWD/android-toolchain
mkdir -p $TOOLCHAIN
$1/build/tools/make-standalone-toolchain.sh \
    --toolchain=arm-linux-androideabi-4.9 \
    --arch=arm \
    --install-dir=$TOOLCHAIN \
    --platform=android-19
export PATH=$TOOLCHAIN/bin:$PATH
export AR=$TOOLCHAIN/bin/arm-linux-androideabi-ar
export CC=$TOOLCHAIN/bin/arm-linux-androideabi-gcc
export CXX=$TOOLCHAIN/bin/arm-linux-androideabi-g++
export LINK=$TOOLCHAIN/bin/arm-linux-androideabi-g++

./configure \
    --dest-cpu=arm \
    --dest-os=android \
    --without-snapshot

joaocgreis · 2016-01-14T14:40:05Z

@kikijhu Instead of adding --without-snapshot, can you try to add

export CC_host=cc
export CXX_host=c++

I don't have the Android NDK here for a quick test, so let us know if this works. Thanks!

fornwall · 2016-01-15T03:08:07Z

@joaocgreis When setting CC_host the host build picks up the CFLAGS environment variable, which I've used to configure the target, so the build fails for me.

How should CFLAGS be set for the target when wanting to cross compile with snapshot?

joaocgreis · 2016-01-15T10:33:02Z

GYP uses the variables {CC,CXX}_host and {CC,CXX}_target to override {CC,CXX} when cross compiling. I'm not sure if it also handles CFLAGS, but it would need a similar duplication to be complete. On our side, it would be great to have configure supporting every combination, but most people committing to configure are not cross compiling so it would break easily. Work like that would require someone to keep a more constant attention over it, that's why I chose to do the least disruptive modification to configure and support only {CC,CXX}_host. Contributions are welcome!

@fornwall Perhaps the best way is not using the CFLAGS variable at all, and adding its contents to the proper {CC,CXX} variable in the script. Something like:

export CC="$TOOLCHAIN/bin/arm-linux-androideabi-gcc -O? -f..."
export CXX="$TOOLCHAIN/bin/arm-linux-androideabi-g++ -O? -f..."
export CC_host="cc -O? -f..."
export CXX_host="c++ -O? -f..."

kikijhu · 2016-01-18T03:01:29Z

I add these lines to ./android-configure

export CC_host=cc
export CXX_host=c++

Build failed again.

  ln -f "/home/kiki/node.js/node-v5.4.1/out/Release/obj.target/deps/v8/tools/gyp/libv8_libbase.a" "/home/kiki/node.js/node-v5.4.1/out/Release/libv8_libbase.a" 2>/dev/null || (rm -rf "/home/kiki/node.js/node-v5.4.1/out/Release/libv8_libbase.a" && cp -af "/home/kiki/node.js/node-v5.4.1/out/Release/obj.target/deps/v8/tools/gyp/libv8_libbase.a" "/home/kiki/node.js/node-v5.4.1/out/Release/libv8_libbase.a")
  c++ '-D_GLIBCXX_USE_C99_MATH' '-DV8_TARGET_ARCH_ARM' '-DCAN_USE_ARMV7_INSTRUCTIONS' '-DCAN_USE_VFP3_INSTRUCTIONS' '-DCAN_USE_VFP32DREGS' '-DENABLE_DISASSEMBLER' '-DUSE_EABI_HARDFLOAT=0' -I../deps/v8  -Wall -Wextra -Wno-unused-parameter -m32 -ffunction-sections -fdata-sections -fno-omit-frame-pointer -fPIE -fdata-sections -ffunction-sections -O2 -fno-rtti -fno-exceptions -std=gnu++0x -MMD -MF /home/kiki/node.js/node-v5.4.1/out/Release/.deps//home/kiki/node.js/node-v5.4.1/out/Release/obj.host/mksnapshot/deps/v8/src/snapshot/mksnapshot.o.d.raw   -c -o /home/kiki/node.js/node-v5.4.1/out/Release/obj.host/mksnapshot/deps/v8/src/snapshot/mksnapshot.o ../deps/v8/src/snapshot/mksnapshot.cc
  c++ -rdynamic -m32 -fPIE -pie  -o /home/kiki/node.js/node-v5.4.1/out/Release/mksnapshot /home/kiki/node.js/node-v5.4.1/out/Release/obj.host/mksnapshot/deps/v8/src/snapshot/mksnapshot.o /home/kiki/node.js/node-v5.4.1/out/Release/obj.host/deps/v8/tools/gyp/libv8_base.a /home/kiki/node.js/node-v5.4.1/out/Release/obj.host/deps/v8/tools/gyp/libv8_nosnapshot.a /home/kiki/node.js/node-v5.4.1/out/Release/obj.host/deps/v8/tools/gyp/libv8_libplatform.a /home/kiki/node.js/node-v5.4.1/out/Release/obj.host/deps/v8/tools/gyp/libv8_libbase.a -llog -ldl -lrt
/usr/bin/ld: cannot find -llog
collect2: error: ld returned 1 exit status
make[1]: *** [/home/kiki/node.js/node-v5.4.1/out/Release/mksnapshot] Error 1
make[1]: Leaving directory `/home/kiki/node.js/node-v5.4.1/out'
make: *** [node] Error 2

peter279k · 2016-01-22T10:52:44Z

Hi all, I have same problems during crossing compile node.js.
When I edit android-configure file and add this text : --without-snapshot , It's successful to cross compile.
Here is my android-configure file :

#!/bin/bash

export TOOLCHAIN=/home/lee/android-toolchain
export PATH=$TOOLCHAIN/bin:$PATH
export AR=$TOOLCHAIN/bin/arm-linux-androideabi-ar
export CC=$TOOLCHAIN/bin/arm-linux-androideabi-gcc
export CXX=$TOOLCHAIN/bin/arm-linux-androideabi-g++
export LINK=$TOOLCHAIN/bin/arm-linux-androideabi-g++

./configure \
    --dest-cpu=arm \
    --dest-os=android \
    --without-snapshot

Following texts are a part of crossing compile nodejs's message.

openssl/libopenssl.a /home/lee/node/out/Release/obj.target/deps/zlib/libzlib.a /home/lee/node/out/Release/obj.target/deps/http_parser/libhttp_parser.a /home/lee/node/out/Release/obj.target/deps/uv/libuv.a /home/lee/node/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a /home/lee/node/out/Release/obj.target/deps/v8/tools/gyp/libv8_libbase.a /home/lee/node/out/Release/obj.target/deps/v8/tools/gyp/libv8_nosnapshot.a -Wl,--end-group -llog -lm -ldl
  touch /home/lee/node/out/Release/obj.target/node_dtrace_header.stamp
  touch /home/lee/node/out/Release/obj.target/node_dtrace_provider.stamp
  touch /home/lee/node/out/Release/obj.target/node_dtrace_ustack.stamp
  touch /home/lee/node/out/Release/obj.target/node_etw.stamp
  touch /home/lee/node/out/Release/obj.target/node_perfctr.stamp
  touch /home/lee/node/out/Release/obj.target/specialize_node_d.stamp
make[1]: Leaving directory '/home/lee/node/out'
ln -fs out/Release/node node

fornwall · 2016-01-22T13:47:48Z

Should a new issue be created for Android cross-compile does not work with v8 snapshots, since this issue is closed (and the Android issue doesn't seem ARM-specific)?

joaocgreis · 2016-01-25T19:26:02Z

Discussion moved to nodejs/node#4860

michael-ts · 2017-03-01T22:06:37Z

I am seeing this issue cross-compiling for Linux on ARM (but NOT Android). I'm using the following commands:

export CC=/home/michael/tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-gcc
export CXX=/home/michael/tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-g++
export AR=/home/michael/tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-ar
export RANLIB=/home/michael/tools/arm-unknown-linux-gnueabi/bin/arm-unknown-linux-gnueabi-ranlib
./configure --without-snapshot --dest-cpu=arm --dest-os=linux --with-arm-float-abi=hard
make

Per the above suggestion, I also tried adding:

export CC_host=cc
export CXX_host=c++

I'm not sure I understand what the fix for this is supposed to be. Is there something I am doing wrong, or is this a bug?

fornwall · 2017-03-01T22:09:49Z

@michael-ts I think this is the issue to track: nodejs/node#9707

michael-ts · 2017-03-01T22:31:00Z

@fornwall Ah, i see they are using --without-intl. Using that I seem to get a successful build.

fornwall · 2017-03-01T23:05:33Z

@michael-ts Are you building node 6.x or 7.x?

michael-ts · 2017-03-01T23:39:40Z

@fornwall 6.9.1

orangemocha mentioned this issue Nov 24, 2015

problems with node-test-pull-request #263

Closed

MylesBorins mentioned this issue Dec 1, 2015

configure: v8_use_snapshot should be true nodejs/node#3962

Closed

joaocgreis mentioned this issue Dec 2, 2015

configure: respect CC_host in host arch detection nodejs/node#4117

Merged

joaocgreis closed this as completed Dec 9, 2015

richardlau mentioned this issue Jan 25, 2016

Build failure when cross-compiling for ARM/Android nodejs/node#4860

Closed

imyller mentioned this issue Apr 13, 2016

reason for --without-snapshot in configure imyller/meta-nodejs#31

Closed

fornwall mentioned this issue May 2, 2016

Building for Android not possible with snapshot nodejs/node#6521

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARM cross-compile does not use v8 snapshots #266

ARM cross-compile does not use v8 snapshots #266

orangemocha commented Nov 24, 2015

bnoordhuis commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

targos commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

ofrobots commented Nov 24, 2015

indutny commented Nov 24, 2015

ofrobots commented Nov 24, 2015

orangemocha commented Nov 24, 2015

ofrobots commented Nov 24, 2015

joaocgreis commented Dec 7, 2015

joaocgreis commented Dec 9, 2015

kikijhu commented Jan 14, 2016

kikijhu commented Jan 14, 2016

joaocgreis commented Jan 14, 2016

fornwall commented Jan 15, 2016

joaocgreis commented Jan 15, 2016

kikijhu commented Jan 18, 2016

peter279k commented Jan 22, 2016

fornwall commented Jan 22, 2016

joaocgreis commented Jan 25, 2016

michael-ts commented Mar 1, 2017 •

edited

Loading

fornwall commented Mar 1, 2017

michael-ts commented Mar 1, 2017 •

edited

Loading

fornwall commented Mar 1, 2017

michael-ts commented Mar 1, 2017

ARM cross-compile does not use v8 snapshots #266

ARM cross-compile does not use v8 snapshots #266

Comments

orangemocha commented Nov 24, 2015

bnoordhuis commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

targos commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

indutny commented Nov 24, 2015

ofrobots commented Nov 24, 2015

indutny commented Nov 24, 2015

ofrobots commented Nov 24, 2015

orangemocha commented Nov 24, 2015

ofrobots commented Nov 24, 2015

joaocgreis commented Dec 7, 2015

joaocgreis commented Dec 9, 2015

kikijhu commented Jan 14, 2016

kikijhu commented Jan 14, 2016

joaocgreis commented Jan 14, 2016

fornwall commented Jan 15, 2016

joaocgreis commented Jan 15, 2016

kikijhu commented Jan 18, 2016

peter279k commented Jan 22, 2016

fornwall commented Jan 22, 2016

joaocgreis commented Jan 25, 2016

michael-ts commented Mar 1, 2017 • edited Loading

fornwall commented Mar 1, 2017

michael-ts commented Mar 1, 2017 • edited Loading

fornwall commented Mar 1, 2017

michael-ts commented Mar 1, 2017

michael-ts commented Mar 1, 2017 •

edited

Loading

michael-ts commented Mar 1, 2017 •

edited

Loading