[GDAL] switch build to CMake #5423
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The GCC version is increased to 8 since the Windows build hit an internal compiler error: ``` [20:26:11] In function ‘CPLErr GDALResampleChunk32R_ConvolutionT(double, double, double, double, const T*, int, const GByte*, int, int, int, int, int, int, int, int, GDALRasterBand*, void*, int, float, FilterFuncType, FilterFunc4ValuesType, int, bool, float) [with T = unsigned char]’: [20:26:11] cc1plus: internal compiler error: in gimplify_modify_expr, at gimplify.c:5636 [20:26:11] [ 34%] Building CXX object ogr/ogrsf_frmts/mssqlspatial/CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj [20:26:11] cd /workspace/srcdir/gdal-3.5.1/build/ogr/ogrsf_frmts/mssqlspatial && /opt/bin/x86_64-w64-mingw32-libgfortran4-cxx11/x86_64-w64-mingw32-g++ --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/ -DDEBUG -DDONT_DEPRECATE_SPRINTF -DGDAL_CMAKE_BUILD -DGDAL_COMPILATION -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME @CMakeFiles/ogr_MSSQLSpatial.dir/includes_CXX.rsp -fvisibility=hidden -g -Os -Wall -Wextra -Winit-self -Wunused-parameter -Wmissing-declarations -Wlogical-op -Wshadow -Wmissing-include-dirs -Wformat -Werror=format-security -Wno-format-nonliteral -Werror=vla -Wno-clobbered -Wdate-time -Wnull-dereference -Wfloat-conversion -Wnon-virtual-dtor -Woverloaded-virtual -Wsuggest-override -fno-operator-names -Wzero-as-null-pointer-constant -ftrapv -std=gnu++11 -MD -MT ogr/ogrsf_frmts/mssqlspatial/CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj -MF CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj.d -o CMakeFiles/ogr_MSSQLSpatial.dir/ogrmssqlgeometryparser.cpp.obj -c /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/mssqlspatial/ogrmssqlgeometryparser.cpp [20:26:12] 0x78493a gimplify_modify_expr [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:5636 [20:26:12] 0x77c18e gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11237 [20:26:12] 0x77f148 gimplify_stmt(tree_node**, gimple**) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:6496 [20:26:12] 0x7823e3 gimplify_cond_expr [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:3973 [20:26:12] 0x77c836 gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11194 [20:26:12] 0x77f148 gimplify_stmt(tree_node**, gimple**) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:6496 [20:26:12] 0x78203e gimplify_cond_expr [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:3866 [20:26:12] 0x77c836 gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11194 [20:26:12] 0x77bbc1 gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11966 [20:26:12] 0x77c4ae gimplify_expr(tree_node**, gimple**, gimple**, bool (*)(tree_node*), int) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify.c:11339 [20:26:12] 0x7879a9 force_gimple_operand_1(tree_node*, gimple**, bool (*)(tree_node*), tree_node*) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify-me.c:78 [20:26:12] 0x787a7d force_gimple_operand_gsi_1(gimple_stmt_iterator*, tree_node*, bool (*)(tree_node*), tree_node*, bool, gsi_iterator_update) [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/gimplify-me.c:115 [20:26:12] 0xaa991d rewrite_use_nonlinear_expr [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7242 [20:26:12] 0xaa991d rewrite_groups [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7466 [20:26:12] 0xaa991d tree_ssa_iv_optimize_loop [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7827 [20:26:12] [ 34%] Building CXX object gcore/CMakeFiles/gcore.dir/gdalmultidim.cpp.obj [20:26:12] 0xaa991d tree_ssa_iv_optimize() [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop-ivopts.c:7859 [20:26:12] cd /workspace/srcdir/gdal-3.5.1/build/gcore && /opt/bin/x86_64-w64-mingw32-libgfortran4-cxx11/x86_64-w64-mingw32-g++ --sysroot=/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/ -DDEBUG -DDONT_DEPRECATE_SPRINTF -DGDAL_CMAKE_BUILD -DGDAL_COMPILATION -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_GEOS -DHAVE_SSE_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME -DPAM_ENABLED -DSQLITE_ENABLED @CMakeFiles/gcore.dir/includes_CXX.rsp -fvisibility=hidden -g -Os -Wall -Wextra -Winit-self -Wunused-parameter -Wmissing-declarations -Wlogical-op -Wshadow -Wmissing-include-dirs -Wformat -Werror=format-security -Wno-format-nonliteral -Werror=vla -Wno-clobbered -Wdate-time -Wnull-dereference -Wfloat-conversion -Wnon-virtual-dtor -Woverloaded-virtual -Wsuggest-override -fno-operator-names -Wzero-as-null-pointer-constant -ftrapv -Wold-style-cast -Weffc++ -std=gnu++11 -MD -MT gcore/CMakeFiles/gcore.dir/gdalmultidim.cpp.obj -MF CMakeFiles/gcore.dir/gdalmultidim.cpp.obj.d -o CMakeFiles/gcore.dir/gdalmultidim.cpp.obj -c /workspace/srcdir/gdal-3.5.1/gcore/gdalmultidim.cpp [20:26:12] 0xac2900 execute [20:26:12] /workspace/srcdir/gcc-7.1.0/gcc/tree-ssa-loop.c:549 [20:26:12] Please submit a full bug report, [20:26:12] with preprocessed source if appropriate. [20:26:12] Please include the complete backtrace with any bug report. [20:26:12] See <https://gcc.gnu.org/bugs/> for instructions. [20:26:12] make[2]: *** [gcore/CMakeFiles/gcore.dir/build.make:617: gcore/CMakeFiles/gcore.dir/overview.cpp.obj] Error 1 [20:26:12] make[2]: *** Waiting for unfinished jobs.... ```
without this it would pick up dependencies like ``` -- Could NOT find ZSTD (missing: ZSTD_DIR) -- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.0") -- Found ZSTD: /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/lib/libzstd.so (found version "1.5.2") ```
It seems to find the wrong postgres: ``` [07:34:13] In file included from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogr_pg.h:35, [07:34:13] from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogrpgdatasource.cpp:31: [07:34:13] /opt/aarch64-linux-gnu/aarch64-linux-gnu/sys-root/usr/local/include/libpq-fe.h:29:10: fatal error: postgres_ext.h: No such file or directory [07:34:13] #include "postgres_ext.h" [07:34:13] ^~~~~~~~~~~~~~~~ ``` This reverts commit da9909d.
This patch is based on a similar patch for the Glib build. The error this fixes is: ``` [08:24:11] /opt/bin/x86_64-unknown-freebsd12.2-libgfortran5-cxx11/x86_64-unknown-freebsd12.2-clang++ --sysroot=/opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/ -fPIC -fno-finite-math-only -fvisibility=hidden -O3 -DNDEBUG -Wl,--no-undefined -shared -Wl,-soname,libgdal.so.31 -o libgdal.so.31.0.1 @CMakeFiles/GDAL.dir/objects1.rsp -Wl,-rpath,/workspace/destdir/lib: -lm /workspace/destdir/lib/libz.so /workspace/destdir/lib/libcurl.so /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/libcrypto.so /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/libssl.so /workspace/destdir/lib/libzstd.so /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/liblzma.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libtiff.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libtiff.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libgeotiff.so.5.2.0 /workspace/destdir/lib/libtiff.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libpng.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libLerc.so /workspace/destdir/lib/libzstd.so /workspace/destdir/lib/libpng.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libjpeg.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libpng.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libcurl.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libz.so /workspace/destdir/lib/libopenjp2.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libsqlite3.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libexpat.so /workspace/destdir/lib/libgeos_c.so.1.17.0 /workspace/destdir/lib/libproj.so.25.9.1.0 -lpthread /opt/x86_64-unknown-freebsd12.2/x86_64-unknown-freebsd12.2/sys-root/usr/lib/libcrypto.so -Wl,-rpath-link,/workspace/destdir/lib [08:24:12] /opt/x86_64-unknown-freebsd12.2/bin/x86_64-unknown-freebsd12.2-ld: port/CMakeFiles/cpl.dir/cpl_spawn.cpp.o: in function `CPLSpawnAsync': [08:24:12] cpl_spawn.cpp:(.text+0x822): undefined reference to `environ' [08:24:12] clang-13: error: linker command failed with exit code 1 (use -v to see invocation) ```
giordano
requested changes
Sep 4, 2022
Comment on lines
+24
to
+28
| if [[ "${target}" == *-freebsd* ]]; then | ||
| # Our FreeBSD libc has `environ` as undefined symbol, so the linker will | ||
| # complain if this symbol is used in the built library, even if this won't | ||
| # be a problem at runtime. This flag allows having undefined symbols. | ||
| export LDFLAGS="-undefined" |
There was a problem hiding this comment.
You're shifting a linking time error to a runtime error, it doesn't look like a great tradeoff
There was a problem hiding this comment.
Ugh, I see, it's https://docs.binarybuilder.org/stable/troubleshooting/#undefined-reference-to-environ'-1. Sigh
There was a problem hiding this comment.
Yeah it's used in a posix_spawnp call. Interestingly that file got this FreeBSD patch recently: OSGeo/gdal@4ff30d5
There was a problem hiding this comment.
Then it'd probably be better to apply that patch than to change link flags.
There was a problem hiding this comment.
Sorry, that patch is already included. With or without it I get the same issue, I tried playing around with it a bit. In https://bugs.webkit.org/show_bug.cgi?id=138420 they resolved it by avoiding using environ altogether. Don't know if that is feasible. It is only used once, in this function:
https://github.com/OSGeo/gdal/blob/v3.5.1/port/cpl_spawn.cpp#L578-L612
On this line:
https://github.com/OSGeo/gdal/blob/v3.5.1/port/cpl_spawn.cpp#L708
current error is: ``` [11:32:50] In file included from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogr_pg.h:35, [11:32:50] from /workspace/srcdir/gdal-3.5.1/ogr/ogrsf_frmts/pg/ogrpgtablelayer.cpp:31: [11:32:50] /opt/aarch64-linux-gnu/aarch64-linux-gnu/sys-root/usr/local/include/libpq-fe.h:29:10: fatal error: postgres_ext.h: No such file or directory [11:32:50] #include "postgres_ext.h" [11:32:50] ^~~~~~~~~~~~~~~~ [11:32:50] compilation terminated. ```
9 tasks
|
I think I'm starting to understand why the PostgreSQL driver is not working. The latest error was
When looking in the include directory of https://github.com/JuliaBinaryWrappers/LibPQ_jll.jl there is no such header file. I thought we needed libpq but we probably need to build PostgreSQL including libpq. I guess the best way forward would be to create a separate PostgreSQL_jll that builds the whole thing. Would be a bit odd to have that separate from LibPQ_jll, though I'm assuming the authors went for a libpq-only build on purpose there, to keep things light. cc @maxfreu because of #5349. https://github.com/JuliaPackaging/Yggdrasil/blob/master/L/LibPQ/build_tarballs.jl Based on that I'd say let's not include PostgreSQL in this PR, but leave it for a follow up. |
giordano
approved these changes
Sep 4, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As mentioned in #4979, GDAL deprecated the autoconf build system, so we needed to make the switch. It was probably a good idea to wait until GDAL 3.5.1, since there were a bunch of CMake build fixes there. Actually it seems that it's quite a painless transition. All patches are removed. I did have to update to GCC 8 since I hit an internal compiler error on GCC 7 (details in b84cce5).
I tried to directly include Postgres support from #5349 but it seems to still pick up the wrong one so I reverted that for now in ddcfbab.