fix(db): use the correct state version for databases without a state version file

[teor2345]

Motivation

When Zebra opens a state cache that has never been upgraded, it was reporting the cache as a newly created state, then marking it with the current state version. This incorrectly skips all format upgrades.

Instead, all upgrades should be performed on the state.

Specifications

https://doc.rust-lang.org/std/fs/struct.Metadata.html#method.is_dir

Complex Code or Requirements

We missed this bug because there weren't enough tests for state version changes or state upgrades. But there are tests in PR #7379 now.

Solution

If there's a database directory wit no version file, report its version as 25.0.0. (Compatible but with no upgrades.)

Review

This bug fix blocks all the finalized state format changes.

Reviewer Checklist

• Will the PR name make sense to users?
  • Does it need extra CHANGELOG info? (new features, breaking changes, large changes)
• Are the PR labels correct?
• Does the code do what the ticket and PR says?
  • Does it change concurrent code, unsafe code, or consensus rules?
• How do you know it works? Does it have tests?

[upbqdn]

@Mergifyio update.

[mergify]

update .

✅ Branch has been successfully updated

[upbqdn]

This assert fails https://github.com/zcashfoundation/zebra/blob/f08cd9fbc6d01f1bae0d9e5b54047a053f36a4e7/zebra-state/src/service/finalized_state/disk_format/upgrade.rs#L481-L486.

It looks like database_format_version_on_disk returns None here https://github.com/zcashfoundation/zebra/blob/f08cd9fbc6d01f1bae0d9e5b54047a053f36a4e7/zebra-state/src/service/finalized_state/zebra_db.rs#L65-L66, but then Some here https://github.com/zcashfoundation/zebra/blob/f08cd9fbc6d01f1bae0d9e5b54047a053f36a4e7/zebra-state/src/service/finalized_state/disk_format/upgrade.rs#L477-L478, likely due to the new refactor. I don't see why this happens since database_format_version_on_disk should return consistent results.

[teor2345]

This assert fails zcashfoundation/zebra@f08cd9f/zebra-state/src/service/finalized_state/disk_format/upgrade.rs#L481-L486.

It looks like database_format_version_on_disk returns None here zcashfoundation/zebra@f08cd9f/zebra-state/src/service/finalized_state/zebra_db.rs#L65-L66, but then Some here zcashfoundation/zebra@f08cd9f/zebra-state/src/service/finalized_state/disk_format/upgrade.rs#L477-L478, likely due to the new refactor. I don't see why this happens since database_format_version_on_disk should return consistent results.

That's my mistake. As soon as RocksDB creates the database directory, we provide 25.0.0 as the database version, even though the database is empty and newly created. We need to open RocksDB first, because we depend on its lock to protect the version file.

We could delete the "newly created" code path entirely to simplify things, if you want. Because now it's equivalent to an upgrade from 25.0.0 to the current version, but without any data in the database. (We might want to keep logging it differently, to avoid confusing users.) But that's out of scope for this PR, so let's open a ticket if there's enough interest.

[upbqdn]

We could delete the "newly created" code path entirely to simplify things, if you want.

I'm fine with leaving it as it is.