Releases: hpc/charliecloud
0.38
What’s new:
- Documentation/examples:
ch-image
:ch-run
:- -W/--write-fake mount point is now /srv rather than /mnt (#1895)
- miscellaneous bug fixes & improvements in:
More details: https://github.com/hpc/charliecloud/milestone/44?closed=1
Even more details: v0.37...v0.38
0.37
Highlights:
-
ch-image
can get into a state where it has images in the storage directory that are marked cached, but the build cache has no record of them. We were not able to find a root cause, but this is now a warning rather than a crash (#1824). If you see the warning, please get in touch so we can find the root cause. -
ch-image
: new option--pdb
that drops into the Python debugger at the specified module and line (#1837) -
Build/install: Version 0.36 introduced a dependency on
pkg-config
at./autogen.sh
time that gives misleading errors about “undefined macro” if not met (#1845). This is now fixed.
What else is new:
ch-image
:RUN
: trailing backslashes no longer sometimes parsed literally (#1679)- now errors out if blob downloaded incompletely or incorrectly (#1758)
- better pretty-printing of Dockerfile parse trees (#1816)
- new hot pink log function for
printf(3)
-style debugging (#1825) - fixed syntax warnings with Python 3.12 (#1834)
ch-fromhost
:ch-completion.bash
: new man page (#1839)ch-convert
: tab completion now works for images with colon in name (#1841)- docs: revised contributor’s guide (#1178)
- miscellaneous bug fixes & improvements in:
More details: https://github.com/hpc/charliecloud/milestone/43?closed=1
Even more details: v0.36...v0.37
0.36
Highlights:
-
ch-run
: new option--write-fake
/-W
that overlays a writeable tmpfs atop a read-only image (#1793). This can also be used to bind-mount on directories that do not exist in read-only images (#96). The feature does require recent-ish kernel support (see FAQ). -
ch-image build
with seccomp root emulation (the default) now interceptsmknod(2)
only if creating a device (#1808). In particular, Dockerfiles can now create named pipes (FIFOs) (#1779). -
Extended attributes (xattrs) support now defaults to disabled (#1787). The feature proved to be too finicky for a default setting, and xattrs do not seem to be well supported across the Linux ecosystem.
What else is new:
ch-image
:ch-run
:ch-convert
:ch-test
:configure
- no longer reports the presence of Buildah and Docker (#1757)
- miscellaneous bug fixes & improvements in:
0.35
Highlights:
-
(Bad news) Previously, the build cache’s optional large file support (i.e.
ch-image --cache-large
set to a non-default value) used hard links to bring large files and out of images. This assumes that hard links are copy-on-write, which they are not. Therefore, use of this feature was likely to corrupt the cache. We now copy large files instead of hard linking them, which is often a significant performance impact. (#1740) -
(Good news)
ch-image
now uses kernel fast paths for file copying when available. In particular, some filesystems offer a “reflink” operation that copies only metadata (notably BTRFS, XFS, and recent ZFS). This should provide a general performance increase, but in particular lets the build cache’s large file operations remain fast meta-data only operations. See the “Best practices” document for a detailed discussion. (#1742) -
(Good news)
ch-image build
supports a new non-standard Dockerfile instruction,RSYNC
(#1708). This copies files from the context directory into the image withrsync(1)
semantics, which are better defined thanCOPY
and provide more control, notably for symlinks. -
(Good news) Interrupting
ch-image
with SIGINT or SIGTERM (notably, control-C) is now much less likely to leave the storage directory in an invalid date (#1518).
What else is new:
ch-image build
andch-run
:$HOME
is always set now (#1687)ch-image
: new argument--quiet
to reduce verbosity (#1613)configure --libexecdir
and similar arguments now work (#683)- miscellaneous bug fixes & improvements in:
More details: https://github.com/hpc/charliecloud/milestone/41?closed=1
Even more details: v0.34...v0.35
0.34
Highlights:
-
ch-image
: seccomp root emulation mode (ch-image --force=seccomp
, or bare--force
in previous versions) is now the default behavior (#1572). Routine use of--force
is no longer needed, and in fact bare--force
without an argument will likely mean a more intrusive root emulation mode in the near future. -
A pre-print of our build cache performance study is now available. TL;DR: it works, has structural advantages, and may even outperform overlay-based caches in many scenarios.
What else is new:
ch-image
and (sometimes)ch-run
now re-print warnings when exiting (#1674)- Shell programs no longer hang if
docker(1)
has been aliased topodman(1)
(#1656) - Bash tab completion now deals correctly with mid-line editing (#1709)
ch-image
:ch-run
:- new option
--set-env0
that accepts zero-separated files (#1124)
- new option
- miscellaneous bug fixes & improvements in:
- refactoring & cleanup (#1693, #1697, #1714)
- documentation & logging/errors (#1672, #1689, #1691, #1696, #1698, #1703, #1715, #1719, #1721)
- test suite and examples (#1707, #1716, #1723)
- bugs both introduced and fixed during release cycle (#1718)
More details: https://github.com/hpc/charliecloud/milestone/40?closed=1
Even more details: v0.33...v0.34
0.33
Highlights:
ch-image
andch-run
now have tab-completion for Bash (#1618, #1652)ch-ssh
has been removed (#1379). We were able to identify no one who wanted to keep it.
What else is new:
ch-convert
:ch-image
:- miscellaneous bug fixes & improvements in:
More details: https://github.com/hpc/charliecloud/milestone/39?closed=1
Even more details: v0.32...v0.33
0.32
Request for feedback:
We are considering removing ch-ssh
, a utility program to facilitate SSH connections from one Charliecloud container into an equivalent container on another host. Please respond to and/or comment on our poll, especially if you use this tool, in discussion #1600.
Highlights:
-
ch-image build --force
now uses a new modeseccomp
by default, based onseccomp(2)
, which is simpler, faster, completely agnostic to libc, and mostly agnostic to distribution (#1527). See §6.7.3 in thech-image
man page for details. The old mode is still available with--force=fakeroot
. Because it is prohibitively difficult to detect ifseccomp
mode would be useful, and we do not want to promote an obsolete mode,ch-image build
no longer recommends--force
on build failures, and the option--no-force-detect
is gone (#1563).Note that if
seccomp
mode proves successful, we plan to removefakeroot
mode (#1565) and possibly make--force
the default (#1572). -
Per our new one-year support policy, storage directories created by versions 0.24 and earlier can no longer be upgraded (#1588). Also, storage directories in the default location of 0.25 and earlier will no longer be moved to the new default location (#1243).
-
ch-test
now works with the maintainedbats-core
fork of Bats that is in most current distributions (#582). The old Bats 0.4.0 appears to still work but is no longer tested or supported. -
New examples to demonstrate use of
libfabric
(#1443).
What else is new:
ch-image
:ch-run
:ch-convert
:ch-fromhost
and examples are now more compatible with Flux (#1597)- miscellaneous bug fixes & improvements in:
More details: https://github.com/hpc/charliecloud/milestone/38?closed=1
Even more details: v0.31...v0.32
0.31
Highlights:
- The tutorial has been completely overhauled, and a new best practices section added to the docs (#536).
ch-run
can now run images in ch-image storage by name (e.g.,ch-run debian:11
), and running images inch-image
storage by path is now disallowed (e.g.,ch-run /var/tmp/charlie.ch/img/debian:11
) (#1058).ch-run
no longer bind-mounts home directories by default, i.e.,--no-home
is the default behavior (#1470). Your home directory can be bind-mounted with--home
, i.e., this is how to get the previous default behavior. Option--no-home
is still accepted as a no-op but will be removed in 0.32.ch-convert
now can convert to and from Podman (#1360).
What else is new:
ch-image
:- test suite now passes with ShellCheck 0.9.0 (#1510)
- miscellaneous bug fixes & improvements in:
More details: https://github.com/hpc/charliecloud/milestone/37?closed=1
Even more details: v0.30...v0.31
0.30
Highlights:
-
The build cache is now enabled by default if an appropriate Git is installed (#1344).
-
Significant build cache performance improvements (#1411, #1412, #1451, #1459, #1464).
What else is new:
ch-image
:build
:build-cache
:- more details printed in verbose mode (#1441)
delete
:list
:- new subcommand
undelete
(build cache enabled only) (#1438) - Lark module no longer enforces a maximum version (#1432)
- relative storage paths now rejected instead of breaking (#1403)
- new option
--no-lock
to disable storage collision checking (#1418) ch-run
:- error “can’t join user namespace of pid” fixed (#1270)
ch-convert
: obscure permissions bug fixed (#1484)- web docs: search no longer returns results that are 404 (#1461)
- test suite and examples:
- miscellaneous bug fixes & improvements in:
More details: https://github.com/hpc/charliecloud/milestone/36?closed=1
Even more details: v0.29...v0.30
0.29
Highlights:
-
All Charliecloud workflows are once again fully unprivileged end-to-end (#1415). Previously, the SquashFS workflow required a setuid executable (
fusermount3
) in order to initialize the FUSE mount. In 0.29,fusermount3
does not need to be setuid, and in fact Charliecloud actively suppresses its setuid bit if set. To our knowledge, Charliecloud is the first container implementation to deploy a release with a fully unprivileged SquashFS workflow. -
Build cache enabled by default has been postponed to 0.30 (#1421).
Known bugs of note:
- We are seeing intermittent problems with
ch-run --join
erroring “can’t join user namespace of pid” (#1270). Currently the workaround is to configure with--disable-syslog
.
What else is new:
ch-image
:- miscellaneous bug fixes & improvements in:
- documentation & logging/errors (#1402)
More details: https://github.com/hpc/charliecloud/milestone/35?closed=1
Even more details: v0.28...v0.29