Force rebuild all packages in CI #8912

vicroms · 2019-11-05T18:34:30Z

No description provided.

Neumann-A · 2019-11-06T15:29:32Z

qt5-tools regression on linux due to automatically picked up external dependency to some XML library in one of the qt5-ports. qt is really annoying with this automatically picked up dependencies.

vicroms · 2019-11-07T00:23:18Z

It will take some time to fix all of these regressions 😞

JackBoosY · 2019-11-07T02:29:12Z

@vicroms Don't worry, some of them are expected.

PhoebeHui · 2019-11-07T03:01:13Z

Check port failures on Linux and macOS:

Port	Triplets	Failures
google-cloud-cpp	Linux macOS	Conflict with google-cloud-cpp-common
libmesh	Linux macOS	Conflict with boost-algorithm
mmx	Linux	Conflict with jsonnet
nanovg	Linux macOS	Conflict with stb
otl	Linux macOS	File does not have expected hash: File path: [ /ci/myagent/_work/2/s/downloads/temp/otlv4_h2-4.0.447.zip ] Expected hash: [ 07663442272a327a7d9154f6e817c0166ed254cfe3b9d043762179e96180a70d3ba4b3762e5ef1ebdb18492e3425467c9ddad3a2c68aa93bb7d51d54e9712008 ] Actual hash: [ 03616606ffb903e43af2e8fec0cd6ecd22380b711966145ca7fe5f1ccde151dcaff044ce7c81389e4055cdab72d019cc301d023a6e179492d1df8b5f2787f4d9 ]
xmsh	Linux macOS	Failed to download file. The source https://github.com/libxmsh/xmsh/archive/v0.4.1.tar.gz doesn’t exist. Need update. -- Downloading https://github.com/libxmsh/xmsh/archive/v0.4.1.tar.gz... Failed. Status: 22;"HTTP response code said error"
rapidstring	Linux macOS	Failed to download file. Cannot access the https://github.com/boyerjohn/rapidstring -- Downloading https://github.com/boyerjohn/rapidstring/archive/ee433955c1e9cacfaecbf53c0a13318ab5825bd4.tar.gz... Failed. Status: 22;"HTTP response code said error"
tiny-process-library	macOS	Downloading https://gitlab.com/eidheim/tiny-process-library/-/archive/273270d0f9d0cf4a8282fadd589060a7b0eab425/tiny-process-library-273270d0f9d0cf4a8282fadd589060a7b0eab425.tar.gz... Failed. Status: 22;"HTTP response code said error"
tinyfiledialogs	Linux macOS	Failed to download file. The source URL https://git.code.sf.net/p/tinyfiledialogs/code could not access.
cppkafka	Linux	Build failures
qt5-tools	Linux	Build failures
quickfix	Linux macOS	Build failures
xalan-c	Linux	Build failures
libosip2	macOS	Build failures
llvm	macOS	Build failures
mlpack	macOS	Build failures
mongo-c-dirver	macOS	Build failures

PhoebeHui · 2019-11-07T07:32:51Z

Ports failures on windows:

Port	Triplets	Failures
google-cloud-cpp	x64-windows x86-windows x64-windows-static	conflict with google-cloud-cpp-common
ogre	x64-windows x86-windows x64-windows-static	Conflict with ogre-next
mmx	x64-windows x86-windows x64-windows-static x64-uwp arm64-windows arm-uwp	conflict with jsonnet
nanovg	x64-windows x86-windows x64-windows-static arm64-windows	conflict with stb
otl	x64-windows x64-windows-static x64-uwp arm64-windows arm-uwp	File does not have expected hash: File path: [ /ci/myagent/_work/2/s/downloads/temp/otlv4_h2-4.0.447.zip ] Expected hash: [ 07663442272a327a7d9154f6e817c0166ed254cfe3b9d043762179e96180a70d3ba4b3762e5ef1ebdb 18492e3425467c9ddad3a2c68aa93bb7d51d54e9712008 ] Actual hash: [ 03616606ffb903e43af2e8fec0cd6ecd22380b711966145ca7fe5f1ccde151dcaff044ce7c81389e4055cd ab72d019cc301d023a6e179492d1df8b5f2787f4d9 ]
parmetis	x64-windows x86-windows x64-windows-static	Failed to download file Downloading http://glaros.dtc.umn.edu/gkhome/fetch/sw/parmetis/parmetis-4.0.3.tar.gz... Failed. Status: 6;"Couldn't resolve host name"
rapidstring	x64-windows x86-windows x64-windows-static	Failed to download file. Cannot access the https://github.com/boyerjohn/rapidstring -- Downloading https://github.com/boyerjohn/rapidstring/archive/ee433955c1e9cacfaecbf53c0a13318ab5825bd4.tar.gz... Failed. Status: 22;"HTTP response code said error"
cnl	x64-windows-static	Failed to download file. Downloading https://github.com/johnmcfarlane/cnl/archive/48428242e72b8239cb14c963e40f724929d54df7.tar.gz... Failed. Status: 22;"HTTP response code said error"
tiff	X64-uwp	-- Downloading http://download.osgeo.org/libtiff/tiff-4.0.10.tar.gz... Failed. Status: 7;"Couldn't connect to server"
tinyfiledialogs	x64-windows x86-windows	Failed to download file. The source URL https://git.code.sf.net/p/tinyfiledialogs/code could not access.
libfabric	x64-windows	Build failures
pangomm	x64-windows	Build failures
quickfix	x64-windows x86-windows x64-windows-static x64-uwp arm64-windows arm-uwp	Build failures
tesseract	x64-windows x86-windows x64-windows-static	Build failures
gsoap	x86-windows arm-uwp	Build failures
libxslt	arm64-windows	Build failures
qhull	arm-uwp	Build failures

PhoebeHui · 2019-11-07T10:07:42Z

Exist PRs to address above issue:
otl: #8937
xmsh: #7155

vicroms · 2019-11-07T21:53:48Z

I have removed the failing tombstones and will rerun the tests, hopefully this will reduce the number of regressions

vicroms · 2019-11-07T22:10:20Z

/azp run

PhoebeHui · 2019-11-08T08:50:00Z

4 ports failed totally on windows In latest CI results:

3 ports failed to download the source file:
rapidstring: the source has gone, I can not find the new address instead.
parmetis: http://glaros.dtc.umn.edu/ seems down, I can't access it.
tinyfiledialogs: failed get source from https://git.code.sf.net/p/tinyfiledialogs/code, also failed to download from https://sourceforge.net/projects/tinyfiledialogs/

1 port libxslt failed with arm64-windows, I can't repro this issue with vs2017 and vs2019 on windows locally, failures:
libxml2.lib(libxml2.dll) : fatal error LNK1112: module machine type 'ARM64' conflicts with target machine type 'ARM'

vicroms · 2019-11-08T19:57:17Z

rapidstring seems to have disappeared from the face of the earth (the author deleted their GitHub repo and with it all existing forks); so we should probably delete the port as well.

I've updated the current commit ID for tinyfiledialogs, the failure is unexepected as I'm still able to browse their repo at the previous commit but vcpkg is not able to get the sources.

I'm not sure what to do about parmetis, as the top-level domain is still alive and kicking (belongs to the Unniversity of Minessota), it might come back it might not... for the time being I'll just disable that port in our CI.

I'll take a look at libxslt, it has failed consistently on our CI machines so it is not a false negative.

vicroms · 2019-11-08T23:24:38Z

quickfix fails when double-conversion is installed, this is due to it using an embedded dependency of double-conversion.

vicroms · 2019-11-09T01:01:58Z

/azp run

JackBoosY · 2019-11-09T06:12:52Z

Current regression ports:
Linux:

qt5-tools:

g++: error: /ci/myagent/_work/1/s/installed/x64-linux/debug/lib/libEGL.so: No such file or directory
g++: error: /ci/myagent/_work/1/s/installed/x64-linux/debug/lib/libicui18n.a: No such file or directory
g++: error: /ci/myagent/_work/1/s/installed/x64-linux/debug/lib/libicuuc.a: No such file or directory
g++: error: /ci/myagent/_work/1/s/installed/x64-linux/debug/lib/libicudata.a: No such file or directory

OSX:

llvm:

Undefined symbols for architecture x86_64:
  "_iconv", referenced from:
      _xmlIconvWrapper in libxml2.a(encoding.c.o)
  "_iconv_close", referenced from:
      _xmlFindCharEncodingHandler in libxml2.a(encoding.c.o)
      _xmlCharEncCloseFunc in libxml2.a(encoding.c.o)
  "_iconv_open", referenced from:
      _xmlFindCharEncodingHandler in libxml2.a(encoding.c.o)
ld: symbol(s) not found for architecture x86_64

xmsh:
The build process failed(vcpkg_install_cmake) and no error log was provided.

Windows(arm64-windows):

libxslt:

libxml2.lib(libxml2.dll) : fatal error LNK1112: module machine type 'ARM64' conflicts with target machine type 'ARM'

cenit · 2019-11-09T18:13:50Z

@vicroms
OTL maintainers has the very bad habit of updating zip overwriting previous one. Hash was failing because it was updated to .448 in the meantime, so you can refer to that version now since you updated hash in 2a32220

vicroms · 2019-11-11T22:50:01Z

@cenit thanks for the heads-up, I changed the CONTROL version.

vicroms · 2019-11-12T01:27:44Z

/azp run

JackBoosY · 2019-11-12T01:54:50Z

I can't reproduce the arm64-windows error on my machine, it seems that everything is fine.
Looks like that the C:\vsts_work\2\s\installed\arm64-windows\debug\lib\libxml2.lib saved on the CI machine is not correct.

JackBoosY · 2019-11-12T04:07:44Z

And I can't repro x64-linux error on the machine. libEGL.so/libicui18n.a/libicuuc.a/libicudata.a does not exist on my machine, but qt5-tools has been successfully built.

@Neumann-A I think qt5 automatically activates some options, what do you think?

Neumann-A · 2019-11-12T09:03:27Z

@JackBoosY: Yeah I fear we must be explicit about all the qt options and make them a feature (#8213) or the default. qt-imageformats for example has 4 options and at least 3 can make problems in vcpkg from which two are explicitly set (tiff, webp) and jasper is missing. Those options can be discovered by looking for a config_help.txt or by searching for a number of configure.json in the source tree. But without #8269 solved I fear that static builds might be difficult to link correctly due to all the possible options in building dependent libraries.
The other question is how can we make features which are only available on certain platforms. Qt has a lot of options which only apply to Unix or Windows. So having target dependent features would be nice to have (like target dependent Build-Depends)

I already have a small number of qt fixes in different PRs to silence CI:
libpq fix
e4a5509
qt-imageformat (jasper fix)
db51886
f1e3e10
d0f1917
c1be2ac

The current errors seen in qt5-tools are probably due to the following options in qt5-base:

  -icu ................. Enable ICU support [auto]
  -opengles3 ........... Enable OpenGL ES 3.x support instead of ES 2.x [auto]
  -egl ................. Enable EGL support [auto]

To make static builds work you need to probably pass ICU_LIBS and EGL_LIBS with the correct libraries.

but there are a few more options which should be checked like:

  -glib ................ Enable Glib support [no; auto on Unix]

(basically everything which is auto in the config_help.txt should be checked.)
You can always check theconfig-<triplet>-err.logto see which options have been enabled.

vicroms · 2019-11-13T18:48:36Z

/azp run

vicroms · 2019-11-15T18:03:00Z

/azp run

c72578 · 2019-11-16T05:52:45Z

@vicroms
OTL maintainers has the very bad habit of updating zip overwriting previous one. Hash was failing because it was updated to .448 in the meantime, so you can refer to that version now since you updated hash in 2a32220

In the meantime OTL has been updated again.
Since 2019-11-15 it is at version 4.0.449
New sha512:
055bc52233553738fbdc52a1c5885fad72f3f7bfa9df7bd5880ab47dd656439d00ff8870b1c9a807dc32e205921abbf0ca34bd2bc29aed257fd822685fee8c9e

cenit · 2019-11-16T08:17:07Z

@vicroms
OTL maintainers has the very bad habit of updating zip overwriting previous one. Hash was failing because it was updated to .448 in the meantime, so you can refer to that version now since you updated hash in 2a32220

In the meantime OTL has been updated again.
Since 2019-11-15 it is at version 4.0.449
New sha512:
055bc52233553738fbdc52a1c5885fad72f3f7bfa9df7bd5880ab47dd656439d00ff8870b1c9a807dc32e205921abbf0ca34bd2bc29aed257fd822685fee8c9e

https://sourceforge.net/p/otl/discussion/208859/thread/fdb9d3bacc/

vicroms · 2019-11-17T02:40:59Z

In the meantime OTL has been updated again.

@c72578 Thanks for the heads-up!

https://sourceforge.net/p/otl/discussion/208859/thread/fdb9d3bacc/

@cenit Thank you very mcuh for taking the time to report this upstream! I read the thread and I'm glad that the maintainers were so quick to address this.

The latest version is still under the dynamic link, so I'm thinking whether to pin this to one version behind the latest for the time being... a possible solution could be to allow --head to work with vcpkg_download_distfile (and skip the hash validation) for links like this one.

Neumann-A · 2019-11-17T22:49:43Z

qt-base requires fontconfig on linux due to automatic detection. Add FONTCONFIG_LIBS and also pass expat and libiconv libraries in that variable. Missing symbols seem to be due to expat. Alternativly disable fontconfig for now via -no-fontconfig

JackBoosY · 2019-11-18T06:23:47Z

Linux:

nanovg: conflict files with stb.
libmesh: conflict files with boost/eigen3/metis/netcdf-c.
mmx: conflict files with jsonnet.
google-cloud-cpp: conflict files with google-cloud-cpp-common.
cppkafka: function be16toh/be32toh not found.
quickfix: ISO C++ forbids declaration of ‘DISALLOW_COPY_AND_ASSIGN’.
xanlan-c: library icu is not linked.
qt5-tools: library libxml2 is not linked.

OSX:

quickfix: unknown type name 'DISALLOW_COPY_AND_ASSIGN', no member named 'TrimTrailingZeros' in 'FIX::double_conversion::StringBuilder'.
mongo-c-driver: Undefined symbols for architecture x86_64: std::terminate()/vtable for __cxxabiv1::__class_type_info/vtable for __cxxabiv1::__si_class_type_info/vtable for __cxxabiv1::__vmi_class_type_info/___cxa_begin_catch/___cxa_call_unexpected/___cxa_pure_virtual/___dynamic_cast/___gxx_personality_v0.

Neumann-A · 2019-11-18T10:01:47Z

qt5-tools: library libxml2 is not linked.

@JackBoosY It is expat not libxml2. libxml2 uses different symbols. (see #8912 (comment)) https://www.xml.com/pub/a/1999/09/expat/reference.html

cbezault · 2019-11-18T13:57:27Z

/azp run

vicroms · 2019-11-20T22:34:20Z

/azp run

vicroms · 2019-11-22T09:25:49Z

/azp run

azure-pipelines · 2019-11-22T09:25:54Z

Pull request contains merge conflicts.

cenit · 2019-11-22T09:37:09Z

(otl is "broken" on master, you might already know that...)

cenit · 2019-11-22T09:46:42Z

@JackBoosY you just broke @vicroms PR in the same way the master is broken.... @vicroms original otl's portfile was ok

JackBoosY · 2019-11-22T09:51:08Z

@cenit Ops...I thought this port has been updated.

coryan · 2019-11-23T15:43:51Z

@JackBoosY I think the google-cloud-cpp issues should be resolved now (see #8986 ) please let me know if I can help in any way.

JackBoosY · 2019-11-25T05:49:32Z

/azp run

cenit · 2019-11-25T16:59:05Z

rapidstring seems to have disappeared from the face of the earth (the author deleted their GitHub repo and with it all existing forks); so we should probably delete the port as well.

I have a clone of that repository, if you want. But not being the original library author, I think it might not be the best idea to point to my repository...

https://github.com/cenit/rapidstring

vicroms · 2019-11-25T20:02:11Z

As CI has been restored, this full rebuild is no longer necessary, I'll separate fixes made in here into their own PRs.

Thanks to everyone that helped with solving the flakiness in our ports!

Force rebuild all packages in CI

057ca5b

vicroms added 2 commits November 8, 2019 11:40

Delete rapidstring port as library repo no longer exists

d91d263

[tinyfiledialogs] Fix git commit hash

a542919

dan-shaw mentioned this pull request Nov 8, 2019

CI System Temporarily Offline #8949

Closed

vicroms added 3 commits November 8, 2019 12:20

[otl] Fix download hash

2a32220

[otl] cleanup portfile

95e94a2

[otl] Don't use cached download in CI

16182e9

[xmsh] Update to 0.5.2

6d2f873

[otl] Use correct version number

fc9488d

[qt5-base] Add dependencies on icu and angle

896b9ae

vicroms added info:internal This PR or Issue was filed by the vcpkg team. wip labels Nov 15, 2019

[otl] Use stable download link

4d85fe5

LilyWangL mentioned this pull request Nov 22, 2019

[otl] Create file otlv4.h #9070

Closed

Merge branch 'master' into pr/rebuild-all

58e639a

[otl]Revert 58e639a

483a1f1

vicroms closed this Nov 25, 2019

LilyWangL mentioned this pull request Nov 27, 2019

[otl] Fix header file name and version number #9107

Merged

This was referenced Nov 29, 2019

[rapidstring[core]:x64-windows] build failure - download failed appears the site doesn't exist any longer #9115

Closed

[rapidstring] Remove port #9143

Merged

Neumann-A mentioned this pull request Dec 11, 2019

[qt5-base] Fix Qt5 linux build and be a bit less flaky in CI #9260

Merged

vicroms deleted the pr/rebuild-all branch October 27, 2020 06:13

Force rebuild all packages in CI #8912

Force rebuild all packages in CI #8912

Conversation

vicroms commented Nov 5, 2019

Neumann-A commented Nov 6, 2019

vicroms commented Nov 7, 2019

JackBoosY commented Nov 7, 2019

PhoebeHui commented Nov 7, 2019 • edited Loading

PhoebeHui commented Nov 7, 2019

PhoebeHui commented Nov 7, 2019 • edited Loading

vicroms commented Nov 7, 2019

vicroms commented Nov 7, 2019

PhoebeHui commented Nov 8, 2019 • edited Loading

vicroms commented Nov 8, 2019

vicroms commented Nov 8, 2019

vicroms commented Nov 9, 2019

JackBoosY commented Nov 9, 2019 • edited Loading

cenit commented Nov 9, 2019

vicroms commented Nov 11, 2019

vicroms commented Nov 12, 2019

JackBoosY commented Nov 12, 2019 • edited Loading

JackBoosY commented Nov 12, 2019

Neumann-A commented Nov 12, 2019 • edited Loading

vicroms commented Nov 13, 2019

vicroms commented Nov 15, 2019

c72578 commented Nov 16, 2019

cenit commented Nov 16, 2019

vicroms commented Nov 17, 2019 • edited Loading

Neumann-A commented Nov 17, 2019

JackBoosY commented Nov 18, 2019 • edited Loading

Neumann-A commented Nov 18, 2019

cbezault commented Nov 18, 2019

vicroms commented Nov 20, 2019

vicroms commented Nov 22, 2019

azure-pipelines bot commented Nov 22, 2019

cenit commented Nov 22, 2019

cenit commented Nov 22, 2019

JackBoosY commented Nov 22, 2019

coryan commented Nov 23, 2019

JackBoosY commented Nov 25, 2019

cenit commented Nov 25, 2019 • edited Loading

vicroms commented Nov 25, 2019

PhoebeHui commented Nov 7, 2019 •

edited

Loading

PhoebeHui commented Nov 7, 2019 •

edited

Loading

PhoebeHui commented Nov 8, 2019 •

edited

Loading

JackBoosY commented Nov 9, 2019 •

edited

Loading

JackBoosY commented Nov 12, 2019 •

edited

Loading

Neumann-A commented Nov 12, 2019 •

edited

Loading

vicroms commented Nov 17, 2019 •

edited

Loading

JackBoosY commented Nov 18, 2019 •

edited

Loading

cenit commented Nov 25, 2019 •

edited

Loading