Cargo confuses checksums of crate versions with + in them

[joshtriplett]

Steps to reproduce:

• cargo new foo
• Edit Cargo.toml to add lz4-sys = "=1.0.1+1.7.3"
• cargo fetch
• You'll get this error:

error: failed to verify the checksum of `lz4-sys v1.0.1+1.7.3`

• Look at the metadata for lz4-sys in Cargo.lock:

"checksum lz4-sys 1.0.1+1.7.3 (registry+https://github.com/rust-lang/crates.io-index)" = "00db259f5d4cd53491fc74379b3a68faaae357b64b61e2b1bdbb785b3e3301ee"

• Notice the checksum 00db259f5d4cd53491fc74379b3a68faaae357b64b61e2b1bdbb785b3e3301ee
• Look at lz/4-/lz4-sys in the crates.io-index repository:

{"name":"lz4-sys","vers":"1.0.1+1.7.3","deps":[{"name":"libc","req":"^0.2.17","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal"},{"name":"gcc","req":"^0.3.38","features":[],"optional":false,"default_features":true,"target":null,"kind":"build"}],"cksum":"7f1660dbe767c5b4bbbda783fce9e6a5ca8ae8f7b55605e4a404debc01e3359a","features":{},"yanked":false}
{"name":"lz4-sys","vers":"1.0.1+1.7.5","deps":[{"name":"libc","req":"^0.2.17","features":[],"optional":false,"default_features":true,"target":null,"kind":"normal"},{"name":"gcc","req":"^0.3.38","features":[],"optional":false,"default_features":true,"target":null,"kind":"build"}],"cksum":"00db259f5d4cd53491fc74379b3a68faaae357b64b61e2b1bdbb785b3e3301ee","features":{},"yanked":false}

• Observe that the checksum matches that of 1.0.1+1.7.5, not 1.0.1+1.7.3.

Cargo downloaded the .crate file for 1.0.1+1.7.3, and compared it to the checksum for 1.0.1+1.7.5 (which it put in Cargo.lock), resulting in a checksum mismatch.

Reproduced with cargo 1.36.0 and cargo 1.38.0-nightly (677a180f4 2019-07-08).

This seems related to #6504 , but that bug was closed with no fix. crates.io allows publishing multiple crates whose version numbers differ in metadata, such crates have been published, and cargo breaks when using them.

Also, despite #6806 , I didn't in practice get any warning messages when using such version numbers.

This bug led to a fair bit of consternation among the crates.io admin team, in which we were trying to figure out if we had some index/crate corruption or some other critical issue, before realizing that the issue was in cargo rather than crates.io.

Regardless of any future policy changes to crates.io, I don't think it makes sense for cargo to completely ignore build metadata like this.

[joshtriplett]

cc @sgrif @pietroalbini @rust-lang/crates-io

[alexcrichton]

I think this may be a dupe and/or boil down to rust-lang/crates.io#1059

[joshtriplett]

On July 26, 2019 7:56:40 AM PDT, Alex Crichton ***@***.***> wrote: I think this may be a dupe and/or boil down to rust-lang/crates.io#1059

While it's entirely possible that crates.io should prohibit new uploads of such versions, meanwhile real crates exist on crates.io with such versions, and cargo breaks when trying to use them. (And cargo also issues no warning, despite it being discussed and apparently implemented in one of those issues.) This has shown up in crater runs. I think this needs fixing in cargo, whether crates.io policy changes or not.

[ehuss]

And cargo also issues no warning

This is just a bug with cargo fetch not showing any warnings. The warning shows for any build command. We should decide which commands should and should not show warnings.

[sgrif]

I do think this is a bug in cargo and/or semver. Build metadata is ignored for precedence, but that does not mean it should be ignored for equality. I laid out some more reasoning on this in dtolnay/semver#107 (comment), but it sounds like Cargo is just picking the wrong row given a version in Cargo.lock

[Nemo157]

Another interesting error message I noticed, if you manually update the lockfile to have the correct checksum you get an error before cargo attempts to download the archive:

error: checksum for `lz4-sys v1.0.1+1.7.3` changed between lock files

this could be indicative of a few possible errors:

    * the lock file is corrupt
    * a replacement source in use (e.g., a mirror) returned a different checksum
    * the source itself may be corrupt in one way or another

unable to verify that `lz4-sys v1.0.1+1.7.3` is the same as when the lockfile was generated

[ogoffart]

This causes the build to fail randomly for anything that has ndk-sys in its dependency tree (eg, projects that depends on winit)

error: failed to verify the checksum of ndk-sys v0.4.0

[MarijnS95]

Maintainer of android-ndk-rs and ndk-sys here, and the one who - by accident through automated CI systems and crates.io still not denying this - published ndk-sys on the same version 0.4.0, but with metadata the second time around.

I have since published 0.4.1 to help users resolve this issue, but want to point out that yanking the broken publish with metadata (0.4.0+25.0.8775105) still results in failed downloads. After deleting ~/.cargo/registry/{cache,src}/github.com-1ecc6299db9ec823/ndk-sys-0.4.0{,.crate} (or after deleting all of ~/.cargo) I am still met with:

$ cargo build --target aarch64-linux-android
  Downloaded ndk-sys v0.4.0
error: failed to verify the checksum of `ndk-sys v0.4.0`

on a cargo build with a project that depends specifically on 0.4.0 (via Cargo.lock or "=0.4.0" semver specification). Typically running cargo build again magically solves this issue, but not always.

Timeline

1. (long ago) ndk-sys 0.4.0 was published;
2. ndk-sys 0.4.0+25.0.8775105 was published 2 days ago;
3. ndk-sys 0.4.0+25.0.8775105 was yanked about 14 hours later;
4. (ndk-sys 0.4.1+23.1.7779620 was released to represent the original ndk-sys 0.4.0 publish. Version metadata was added here just to follow a new standard of documenting what NDK revision it was generated from)

---

EDIT: This issue (also for ndk-sys) regarding yanked crates was also reported around the same date over at #11412.

[epage]

rust-lang/crates.io#6518 is merged so new metadata-different versions can't be published anymore.

cargo add lz4-sys@=1.0.1+1.7.3 strips the build metadata when creating the version requirement.

Cargo warns if the version requirement has build metadata.

I did

$ cargo new foo
$ cd foo
$ cargo add lz4-sys@=1.0.1+1.7.3
$ cargo check

and it worked.

Sounds like this is resolved and closing this. If there is anything I missed, let us know!