ansible: update minimum gcc on linux-ppc64le to gcc-6

[sam-github]

No description provided.

[refack]

Actually let's wait with this as little bit. I want to test GCC-8 binaries on machines with only 4.9.

[mhdawson]

Pushed a commit. When testing on a new PPC machine the dependencies were not quite right between roles. The new commit fixes that.

[mhdawson]

LGTM

[mhdawson]

Created a new machine (test-osuosl-ubuntu1404-ppc64_le-4), configure it with this script and test job to ensure builds ok without changes to select-compiler.

https://ci.nodejs.org/job/node-test-commit-plinux-mdawson-test/nodes=test-osuosl-ubuntu1404-ppc64_le-4/1/console

[mhdawson]

@refack in terms of testing GCC 8 binaries on machine with only 4.9.4 one thing to keep in mind is that it depends on what symbols are used from the libraries that come with the compiler. Even it things run today (I think we've seen problems in the past) there is nothing stopping the use of new of new symbols in the future which might break the backwards compatibility. The great thing about the RHEL developer toolset is that it understands the deltas introduced in newer gcc versions and links statically instead of dynamically (based on my understanding) avoiding that problem. From the IBM perspective I understand that our binaries might not be quite as fast having been built with an older gcc, but I value the stability/compatibility of using the older compiler more until we can get the developer toolset in place.

[refack]

depends on what symbols are used from the libraries that come with the compiler.

A minimal sanity test is checking that the libraries have a minimal version. The error we're seeing now is:

./node
/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by ./node)

while the library does export it's minimal supported version:

strings /lib64/libstdc++.so.6 | grep LIBCXX
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_LDBL_3.4
GLIBCXX_LDBL_3.4.7
GLIBCXX_LDBL_3.4.10
GLIBCXX_DEBUG_MESSAGE_LENGTH

[refack]

Another simple option is to simply statically link libstdc++, and in that case IIUC we are only dependent on the system's libc

P.S. that's who it's done for Windows

[mhdawson]

@refack back when we ramped up support for PPC we explored the available options including statically linking. Unless we do that for all platforms (x86) included we did not feel it was a good option. I think the developer toolset has progressed and has better platform support (as does our options for OS's at OSU) but as mentioned I don't think we can count on figuring that out before 12.X goes out.

[mhdawson]

@sam-github noticed one other problem in that after setup on a new machine the wrong gcc is the default. We need to figure out how to update the ansible script to fix that

[mhdawson]

I think the select compiler script will still do the right thing if the default is wrong, but it would be better if the default was set properly for cases were we have not properly configured the jobs.

[refack]

@sam-github noticed one other problem in that after setup on a new machine the wrong gcc is the default. We need to figure out how to update the ansible script to fix that

@rvagg noticed that too in #1722 (comment) I think this will require us to use a compiler selector for all platforms.

(I'm now thinking that we should start storing that information in the node repo and read it for each checkout)

[mhdawson]

Build with PR being used to select the compiler as gcc-6 : https://ci.nodejs.org/job/node-test-commit-plinux-mdawson-test/nodes=test-osuosl-ubuntu1404-ppc64_le-4/2/

[sam-github]

re: #1723 (comment)

I don't see why the default matters, we explicitly set the compiler version with select-compiler.sh. If someone forgets to use select-compiler.sh, they will get the wrong compiler for some cases (it will be wrong for 9.x+, or wrong for 11.x+, but can't be correct for both).

I've no idea how to change the default. I'll check, but I suspect it depends on the order in which packages are installed, which depends on current machine state, so is indeterminant.

[sam-github]

switching order might make the last installed the default, but if gcc-4.9 is already installed, YMMV

[refack]

I've no idea how to change the default. I'll check, but I suspect it depends on the order in which packages are installed, which depends on current machine state, so is indeterminant.

https://askubuntu.com/questions/26498/how-to-choose-the-default-gcc-and-g-version

But you are right it doesn't matter.

[refack]

P.S. maybe we should run:

sudo update-alternatives --remove-all gcc 
sudo update-alternatives --remove-all g++

So there will be no default... But that's for a separate PR.

[rvagg]

P.S. maybe we should run:
... update-alternatives --remove-all

yeah, I like that a lot: always explicit and fail when unspecified

[sam-github]

My ansible skills are barely up to copy-n-paste, they are not up to adding a new shell command like #1723 (comment). I don't know where to put it, and I don't know how to make sure it only runs after the packages are installed, and ideally, run after if-and-only-if packages are installed. I can test it if I'm given some pointers to what kind of config to put in what place.

[refack]

@sam-github IMHO this PR looks good as is. The default discussion just gravitated here. I'll pick it up in a separate PR.

[sam-github]

@mhdawson PTAL, to make sure the changes you pushed are back

[mhdawson]

@sam-github I think it looks good.

[mhdawson]

So given that we agree this can move forward we need to FIRST configure all of the machines to have gcc6 otherwise it will break test/build.

[richardlau]

So given that we agree this can move forward we need to FIRST configure all of the machines to have gcc6 otherwise it will break test/build.

Perhaps we could we break this up into two parts:

1. The ansible changes, which could land first (and then be used to put gcc6 on the machines)
2. The select-compiler.sh changes which should only land when all the machines have gcc6

[sam-github]

Breaking this into two commits is trivial, if that's the right way, I can do it.

Note that @mhdawson is already in process of using the PR branch to update the machines, so the whole PR can be merged.

[refack]

Another off-shoot idea. Store select-compiler.sh in the node repo, not here. That way we can keep it coupled to the version it lives in.
(Jenkins job could check if it's there. If not, use the old script).

P.S. I'm working on it (as I see it's very natural to add that logic to configure.py)...

[mhdawson]

I'm fine either way (1 PR or 2). 2 is probably better in case we need to revert the compiler selection for some reason.

in test le-2, le-4 are updated I'll plan to do le-1 later today and therelease machine some time next week (or possibly the week after since I'm on holiday next week even though I'm not going anywhere.)

[mhdawson]

Deployed across the 3 test machines and the release machine. Should be good to land. After landing please what both test/release jobs for a few days to make sure things look ok.

One extra todo is that we need to add

ubuntu1404-ppc64_le-4: {ip: 140.211.168.140}

to inventory.yml.

I created that as a replacement for ubuntu1404-ppc64_le-3 which was completely broken

[sam-github]

@mhdawson #1739

[mhdawson]

Landed as 03631de