Skip to content

Commit

Permalink
chore: add hamt directory sharding test
Browse files Browse the repository at this point in the history
  • Loading branch information
hacdias committed Mar 14, 2023
1 parent 676e557 commit d23702b
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 3 deletions.
24 changes: 22 additions & 2 deletions test/sharness/t0115-gateway-dir-listing.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,34 @@ test_description="Test directory listing (dir-index-html) on the HTTP gateway"

test_expect_success "ipfs init" '
export IPFS_PATH="$(pwd)/.ipfs" &&
ipfs init --profile=test > /dev/null
ipfs init --empty-repo --profile=test > /dev/null
'

test_launch_ipfs_daemon_without_network

# HAMT-sharded directory test. We must execute this test first as we want to
# start from an empty repository and count the exact number of refs we want.
HAMT_CID=bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm
HAMT_REFS_CID=bafybeicv4utmj46mgbpamvw2k6zg4qjs5yvspfnlxz2y6iuey5smjfcn4a
HAMT_REFS_COUNT=963

test_expect_success "hamt: import fixture with necessary refs" '
ipfs dag import ../t0115-gateway-dir-listing/hamt-refs.car &&
ipfs refs local | wc -l | tr -d " " > refs_count_actual &&
echo $HAMT_REFS_COUNT > refs_count_expected &&
test_cmp refs_count_expected refs_count_actual
'

test_expect_success "hamt: fetch directory from gateway in offline mode" '
curl --max-time 1 -sD - http://127.0.0.1:$GWAY_PORT/ipfs/$HAMT_CID/ > hamt_list_response &&

This comment has been minimized.

Copy link
@lidel

lidel Mar 15, 2023

Member

@hacdias since we don't base test on time, but on correctness of ipfs refs local output, its fine to have this timeout after 15 or 30 seconds even – will make slow CI less flaky.

ps. is there PR which restores this commit? I've seen it was reverted?

This comment has been minimized.

Copy link
@hacdias

hacdias Mar 15, 2023

Author Member

@lidel Yes, this test is part of #9681. I pushed it to master by mistake.

ipfs refs local | wc -l | tr -d " " > refs_count_actual &&
echo $HAMT_REFS_COUNT > refs_count_expected &&
test_cmp refs_count_expected refs_count_actual
'

# Import test case
# See the static fixtures in ./t0115-gateway-dir-listing/
test_expect_success "Add the test directory" '
test_expect_success "add remaining test fixtures" '
ipfs dag import ../t0115-gateway-dir-listing/fixtures.car
'
DIR_CID=bafybeig6ka5mlwkl4subqhaiatalkcleo4jgnr3hqwvpmsqfca27cijp3i # ./rootDir/
Expand Down
23 changes: 22 additions & 1 deletion test/sharness/t0115-gateway-dir-listing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,12 @@

- fixtures.car
- raw CARv1
- hamt-refs.car
- raw CARv1 containing all the necessary blocks to present a directory listing
for `bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm`, which is
a directory containing over 10k items.

generated with:
fixutres.car generated with:

```sh
# using ipfs version 0.18.1
Expand All @@ -30,3 +34,20 @@ ipfs dag export ${DIR_CID} > ./fixtures.car
# FILE_CID=bafkreialihlqnf5uwo4byh4n3cmwlntwqzxxs2fg5vanqdi3d7tb2l5xkm # ./rootDir/ą/ę/file-źł.txt
# FILE_SIZE=34
```

hamt-refs.car generated with:

```sh
# using ipfs version 0.18.1
export IPFS_PATH=$(mktemp -d)
ipfs init --empty-repo
ipfs daemon &
curl http://127.0.0.1:8080/ipfs/bafybeiggvykl7skb2ndlmacg2k5modvudocffxjesexlod2pfvg5yhwrqm/
killall ipfs
ipfs refs local >> refs
ipfs files mkdir --cid-version 1 /Test
cat refs | xargs -I {} ipfs files cp /ipfs/{} /Test/{}
ipfs files stat /Test
# Grab CID: bafybeicv4utmj46mgbpamvw2k6zg4qjs5yvspfnlxz2y6iuey5smjfcn4a
ipfs dag export bafybeicv4utmj46mgbpamvw2k6zg4qjs5yvspfnlxz2y6iuey5smjfcn4a > ./refs.car
```
Binary file not shown.

0 comments on commit d23702b

Please sign in to comment.