Skip to content

Commit

Permalink
Fix NAR tests on Linux+ZFS+normalize
Browse files Browse the repository at this point in the history
A test added recently checks that when trying to deserialize a NAR with
two files that Unicode-normalize to the same result either succeeds on
Linux, or fails with an "already exists" error on Darwin. However,
failing with an "already exists" error can in fact also happen on Linux,
when using ZFS with the proper utf8 and Unicode normalization options
set.

This commit fixes the issue by not assuming the behavior from the
current system, but just by blindly checking that either one of the two
aforementioned possibilities happen, whether on Darwin or on Linux.

Additionally, we check that the Unicode normalization behaviour of
nix-store is the same as the host file system.
  • Loading branch information
yannham committed Sep 30, 2024
1 parent a5959aa commit 43bafd9
Showing 1 changed file with 45 additions and 7 deletions.
52 changes: 45 additions & 7 deletions tests/functional/nars.sh
Original file line number Diff line number Diff line change
Expand Up @@ -88,17 +88,55 @@ touch "$TEST_ROOT/case/xt_CONNMARK.h~nix~case~hack~3"
rm -rf "$TEST_ROOT/case"
expectStderr 1 nix-store "${opts[@]}" --restore "$TEST_ROOT/case" < case-collision.nar | grepQuiet "NAR contains file name 'test' that collides with case-hacked file name 'Test~nix~case~hack~1'"

# Deserializing a NAR that contains file names that Unicode-normalize
# to the same name should fail on macOS but succeed on Linux.
rm -rf "$TEST_ROOT/out"
if [[ $(uname) = Darwin ]]; then
expectStderr 1 nix-store --restore "$TEST_ROOT/out" < unnormalized.nar | grepQuiet "path '.*/out/â' already exists"
else
nix-store --restore "$TEST_ROOT/out" < unnormalized.nar
# Deserializing a NAR that contains file names that Unicode-normalize to the
# same name should fail on macOS and specific Linux setups (typically ZFS with
# `utf8only` enabled and `normalization` set to anything else than `none`). The
# deserialization should succeed on most Linux, where file names aren't
# unicode-normalized.
#
# We test that:
#
# 1. It either succeeds or fails with "already exists" error.
# 2. Nix has the same behavior with respect to unicode normalization than
# $TEST_ROOT's filesystem (when using basic Unix commands)
rm -rf "$TEST_ROOT/out"
set +e
unicodeTestOut=$(nix-store --restore "$TEST_ROOT/out" < unnormalized.nar 2>&1)
unicodeTestCode=$?
touch "$TEST_ROOT/unicode-â"
touch "$TEST_ROOT/unicode-â"
set -e

narFilesCount=$(find "$TEST_ROOT/out" -type f | wc -l)
touchFilesCount=$(find "$TEST_ROOT" -maxdepth 1 -name "unicode-*" -type f | wc -l)

if (( unicodeTestCode == 1 )); then
# If the command failed (MacOS or ZFS + normalization), checks that it failed
# with the expected "already exists" error, and that this is the same
# behavior as `touch`
echo "$unicodeTestOut" | grepQuiet "path '.*/out/â' already exists"

echo "narFilesCount: $narFilesCount"
echo "touchFilesCount: $touchFilesCount"

(( narFilesCount == 1 && touchFilesCount == 1 ))
elif (( unicodeTestCode == 0 )); then
# If the command succeeded, check that both files are present, and that this
# is the same behavior as `touch`
[[ -e $TEST_ROOT/out/â ]]
[[ -e $TEST_ROOT/out/â ]]
[[ -e $TEST_ROOT/unicode-â ]]
[[ -e $TEST_ROOT/unicode-â ]]

(( narFilesCount == 2 && touchFilesCount == 2 ))
else
# if the return code is neither 0 or 1, fail the test.
echo "NAR deserialization of files with the same Unicode normalization failed with unexpected return code $unicodeTestCode" >&2
exit 1
fi

rm -f "$TEST_ROOT/unicode-*"

# Unpacking a NAR with a NUL character in a file name should fail.
rm -rf "$TEST_ROOT/out"
expectStderr 1 nix-store --restore "$TEST_ROOT/out" < nul.nar | grepQuiet "NAR contains invalid file name 'f"
Expand Down

0 comments on commit 43bafd9

Please sign in to comment.