Dropping support for windows binaries

[springmeyer]

For many years we've supported windows in the node-mapnik project, keeping builds working on appveyor and providing binaries for all downstream users.

We've been providing windows binaries since 2014 (v1.4.8 release): #202 (comment)

The time has come to drop support however. This is a tough decision because I'd like to keep supporting windows, but I don't have the ability to.

So, windows binaries are no longer going to be supported for > node-mapnik v3.6.3.

I'm sorry to those of you who have been depending on these binaries over the years (especially to those that might have downstream users or companies depending on them as well). This news probably comes as a major inconvenience/disappointment/surprise.

I've been developing in C++ for over 10 years now and much of that time I've been maintaining builds across mac, linux, windows for a variety of projects. Getting Mapnik running on Windows is the one of the hardest and most labor intensive things I've worked on due to:

• how poor the support for modern C++ features have been in Visual Studio (for C++11 and C++14) (or instead how poor my understanding is, since I primarily develop on unix)
• the large number of dependencies mapnik has and the lack of good consistent scripts or binary packages for them (requiring my team to maintain https://github.com/mapbox/windows-builds/tree/master/scripts)
• the fact that mapnik's core build system does not support windows, requiring the maintenance of a totally separate build system (https://github.com/mapnik/mapnik-gyp), which therefore easily breaks.

All of these things combine with the fact that I'm managing a team at Mapbox now and I don't have the time to dig into failed builds. Also my colleague @BergWerkGIS, who was a major force behind making this system work for many years, has moved on to work on other things at Mapbox unrelated to C++ builds. Lastly we've been supporting windows for a couple years now without actually needing it at Mapbox (we don't depend on the windows builds of mapnik or node-mapnik since moving our focus to Mapbox GL - refs https://www.mapbox.com/help/upgrading-from-classic/ and https://www.mapbox.com/help/upgrading-from-classic/).

As such mapnik is no longer building on windows, and its tests have been failing for some time: https://ci.appveyor.com/project/Mapbox/mapnik/history.

If someone were to pick this up they would need to:

• Bring back to life https://github.com/mapnik/mapnik-gyp (or write some other build system like cmake that supports windows) and build Mapnik correctly with it.
• Bring back to life https://github.com/mapbox/windows-builds, or some kind of set of scripts to build all mapnik dependencies with the same visual studio version and create a tarball of mapnik binaries to build against
• Point node-mapnik at that tarball

  node-mapnik/scripts/build-appveyor.bat

  Line 26 in 176a87d

  SET MAPNIK_SDK_URL=https://mapnik.s3.amazonaws.com/dist/dev/mapnik-win-sdk-%MAPNIK_GIT%-%platform%-%msvs_toolset%.0.7z

Since there is nobody that I know to step up to maintain this system, then my only reasonable option at this point is to drop support for node-mapnik on windows going forward.

So, my recommended solution for developers on windows that are depending on node-mapnik is to pin your usage in your package.json to v3.6.2 like:

  "dependencies": {
    "mapnik": "3.6.2",
  }

Mentioning a few people on this ticket who, from looking at past issues (https://github.com/mapnik/node-mapnik/search?q=windows&type=Issues&utf8=%E2%9C%93), have been users of node-mapnik on windows: @donnyv @eczajk1 @vjpr @BHare1985 @andrelom @bmehta @TimJMartin @germanjoey @aparshin @qiyexue @fnicollet @jbrwn @rclark @n9 @mohsentaleb @djodonnell @iceaxe @mojodna @beaugunderson @strk @mapbox/core-tech

[eczajk1]

Thanks for the heads up. And thanks to you and your team for this project!

[jbrwn]

RIP windows builds and the cottage industry of bored developers hacking away at mapnik on their work issued windows boxes. Brings a tear to my eye thinking about the good old days.

Thanks to everyone who kept it rolling through the years.

[strk]

RIP all proprietary software - not a tear from me :)

[springmeyer]

Thanks everyone for your comments and support. @flippmoke is kindly working on the next release of node-mapnik which will be the release in which we drop windows support. Per @flippmoke's plan at #808 (comment) this next release will be called v3.7.0. This indicates to me that anyone who is using ~3.6.2 in their package.json to install node-mapnik will be isolated from this breakage. However those using ^3.6.2 will be broken - since that will pull in 3.7.0 automatically which will lack windows binaries.

An example of an application that will currently break due to ^ usage and the plan @flippmoke has to release 3.7.0 is https://github.com/koopjs/koop, which depends on https://github.com/koopjs/koop-tile-plugin which depends on mapnik-tiles. This will break because mapnik-tiles is using ^ to install node-mapnik: https://github.com/chelm/mapnik-tiles/blob/6e2b2469edd851560566be917d87652e9d61e92e/package.json#L20. So I've created chelm/mapnik-tiles#2 as an example of a change that will help reduce the fallout. I encourage others to proactively do this asap to help protect their downstream users from breakages.

[springmeyer]

I've made the above PRs by manually paging through https://github.com/search?o=desc&q=%22mapnik%22+extension%3A.json+language%3AJSON&s=indexed&type=Code&utf8=%E2%9C%93. I don't feel motivated to develop a more systematic approach, but if someone does I presume a better approach would be to write a script to download all the modules depending on mapnik via https://www.npmjs.com/search?q=mapnik&page=1&ranking=optimal and detect which are using ^. If someone is interested in taking this on, please comment here. Otherwise I see no next actions for myself, so will close this for now. @flippmoke is taking on next steps at #808. I've also created #855 to track full removal of window support in the repo.

[wayofthefuture]

I understand comments mentioned above. I think it is important to note, however, that Windows comprises 90% of market share. I switched to Linux and then back to Windows because I was always battling "business" compatibility. Business funding helps to drive all of this stuff so IMO dropping Windows is a huge mistake.

[strk]

@wayofthefuture: are yougoing to maintain that support then ? This is a community project, anyone is welcome to invest in whichever aspect of it.

[wayofthefuture]

Good point, can't argue with that.

[pedro-vicente]

FYI, this fork

https://github.com/pedro-vicente/mapnik

has a CMake build for the 3.0 and 2.3 branches with the minimum dependencies

the 2.3 branch builds without errors and runs a test program with the OSM plugin
the 3.0 branch has a couple of compiling errors, with Visual Studio 2019

I posted these errors here

mapnik/mapnik#4062

the errors are not actually that many, it seems to me that with a little effort, the Windows build is doable;

how poor the support for modern C++ features have been in Visual Studio (for C++11 and C++14) (or instead how poor my understanding is, since I primarily develop on unix)

in this case, we just have to write code that compiles both for Visual Studio and gcc, even if it means getting rid of esoteric C++14 features

the large number of dependencies mapnik has and the lack of good consistent scripts or binary packages for them

you are right, there are a lot of code dependencies ; but this is the thing that once done, never has to be done again

the fact that mapnik's core build system does not support windows, requiring the maintenance of a totally separate build system

this is a case, where having CMake as the build system is a must have

I used a script to build the dependencies

for the 3.0 branch, this is the CMake script used, not complete regarding list of source files

https://github.com/pedro-vicente/mapnik/blob/master/CMakeLists.txt

list of dependencies can be built with

https://github.com/pedro-vicente/mapnik/blob/master/build/bld.bat

https://github.com/pedro-vicente/mapnik/blob/master/build/dep.bat

EDIT

all this concerns mapnik itself, not node-mapnik

[dalelutz]

Following up on @pedro-vicente 's post, we'd love to find someone we could pay to get the 3.0 branch compiling and passing tests on VS 2019 (again mapnik, not node-mapnik). Pedro himself is not available for quite some time and we'd love to see this happen sooner. Feel free to write me at dale.lutz@safe.com if you're interested.

[KuDeSnik33ra]

Mapnik incompatibility with boost 1.7 prevent it from building with vcpkg - vcpkg has no support for different lib versions - only the last added one. Are you planing to support it?

[am2222]

@dalelutz, @KuDeSnik33ra the fork in my profile is compatible with vcpkg and I am currently adding the port into vcpkg so it is possible to build it on the windows.
However I am working on mapnik tarball generation to be able to make nodejs mapnik work again.

[RavelDevelopment]

@springmeyer I'd be interested in bringing Windows support back to life and was wondering if you might have some guidance on where to begin.

[am2222]

I have a windows version of mapnik on my github which can be possibly useful. We might probably fix its vcpkg version as well. I have tried to do it but couldn't spend much time on it. Its pull request is available in vcpkg issues.

…

On Sat., Apr. 17, 2021, 12:33 a.m. RavelDevelopment, < ***@***.***> wrote: @springmeyer <https://github.com/springmeyer> I'd be interested in bringing Windows support back to life and was wondering if you might have some guidance on where to begin. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#848 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABD5JLZSGBUKM7CECJWIGEDTJEFTNANCNFSM4EPJJTFA> .