Build process and CI improvements #409
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openssl 1.1.0+ can't have some empty fields specified, so instead don't specify them at all. See #408
- Add Ubuntu 18.04 (Bionic) and Debian 9 (Stretch) to our package
building process.
- Remove Ubuntu 12.04 (Precise) and Debian 7 (Wheezy) from our package
builds. Ubuntu 12.04 has reached its end of support and Debian 7 will
at the end of this month.
- Upgrade from Traffic Server v5 to v7. This stemmed from needing to
build under Ubuntu 18.04 where OpenSSL v1.1 is the default, and we
need at least Traffic Server v6 to build against OpenSSL 1.1.0. There
might still be some keepalive issues to sort out (which is why we were
holding back), but since Traffic Server v5 isn't supported any longer,
this update is due.
- Rework the build process. While this is a bit messy to get wrapped in
with these other updates, this pulls over various improvements to the
cmake build process that were in the lapis-postgres branch. I keep
hitting issues that would benefit from these build improvements, and
maintaining things on both branches is becoming more of a hassle, so
we'll manually pull over the bulk of these build process improvements.
The improvements mainly revolve around better splitting up the build
process into discrete targets. This better organizes the build files,
but also provides easier ways to perform the build in specific steps,
which helps with the Docker build process and better caching.
- Make various improvement to the Docker development environment so
performing builds and re-builds are easier/more reliable. Now the
symlinking happens in an entrypoint script, so that it still takes
place on "run" commands. Also improve the build "configure" script so
it always performs builds in the /build directory. All of these
changes better ensure builds happens as they did on initial build even
if you're manually running these commands on an existing docker dev
container.
- Some other misc improvements pulled over from the lapis-postgres
branch. Again not quite ideal to be manually pulling these over, but
this should help get some of these more general improvements live
before lapis-postgres is merged in.
- Switch from luatz to icu-date for more proper timezone handling.
- Bring over some improvements to startup logic and default yaml
config logic.
The IDs should be functionally equivalent, this is just a pure-Lua implementation to reduce our need for custom nginx modules.
- Ensure test dependencies are installed in CI environment. - Use system libxml and libxslt libraries for nokogiri (to make security updates easier and speed up installs). - Optimize bundle installations so that a common bundle directory is used across bundles for better caching. - Fix unnecessary make re-builds caused by using the dynamic PATH environment variable. Instead use a hard-coded base PATH.
We may revisit this, but since it's probably not on the immediate horizon, go ahead and remove this code, so we're not trying to maintain it or its dependencies.
This mostly ports over the setup used on the lapis-postgres branch where we had started this migration.
globstar isn't apparently set by defualt in CircleCI, so `**` wasn't picking up files in nested directories.
We don't actually have any archives in this directory anymore, so this clean task doesn't actually do anything.
We'll do the upgrades to v7 in a separate branch.
There are no longer any urandom descriptors opened with lua-resty-txid, so we no longer need to check to ensure it's greater than 0. But still keep the basic test intact to ensure we don't accidentally leak things in the future.
This fixes the elasticsearch index names missing the "v" part of the "v1" version number in the name after the various version config changes in this branch. This also pulls in some of the elasticsearch refactoring to add the helper library for handling connections and queries (originally on the lapis-postgres branch).
The machine executor doesn't have caching, which means testing would be very slow given our current reliance on the cache for caching our build files. So instead, go back to the docker test environment, which lacks IPv6 support, so disable our one IPv6 test in the CI environment (not ideal, but for now seems like the simplest option).
Also disable and remove liblogging dependency for rsyslog, since it looks like it's becoming deprecated in a future version, and since we don't actually use it, we might as well get rid of an extra dependency.
Since CircleCI's matching is based on prefixes, with the previous approach it mean the cache for the `updates-trafficserver` branch could inadvertently get used when building the `updates` branch. So reorder the naming convention around so conflicts like this aren't as likely (unless a branch name happened to contain the system arch name).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've had a few different branches that have been in development for a while. In an attempt to clean things up and start to merge some of the changes in more piece-meal, this pull request integrates some of those changes from the in-progress branches.
There's not much functionally different in this pull request, this primarily focuses on improving the build process: