Failed to open tomb - a troubleshooting journey with happy ending

[bugsug]

Problem

• I backed up my hard drive into a tomb on another drive at the beginning of the year
• my main drive crashed
• now I'm not able to open my tomb with the backup

More or less important additional information:

• I recovered the key from a picture
• System: Arch Linux 32 Bit
• Tomb 2.2
• cryptsetup 2.0.2
• pinentry-gtk2 (pinentry) 1.1.0
• Sudo version 1.8.22
• gpg (GnuPG) 2.2.5
• steghide version 0.5.1

Log

tomb [D] Identified caller: ___ (1000:985)
tomb [D] Tomb command: open _____
tomb [D] Caller: uid[1000], gid[985], tty[/dev/pts/1].
tomb [D] Temporary directory: /tmp/zsh
tomb . Commanded to open tomb _____
tomb [D] is_valid_tomb _____
tomb . Valid tomb file found: _____
tomb [D] load_key argument: / __ / __ /
tomb [D] load_key: / __ / __ /
tomb [D] is_valid_key
tomb . Key is valid.
tomb . Mountpoint not specified, using default: / __ / __ /
tomb (*) Opening ___ on / __ / __ /
tomb . This tomb is a valid LUKS encrypted device.
tomb . Cipher is "aes" mode "xts-plain64:sha256" hash "sha256"
tomb [D] dev mapper device: tomb.___.1522689414.loop0
tomb [D] Tomb key: / __ / __ /
tomb [D] Tomb name: ___ (to be engraved)
tomb . A password is required to use key / __ / __ /
tomb [D] asking password with tty=/dev/pts/1 lc-ctype=en_US.UTF8
tomb [D] using pinentry-gtk2

(process:1958): Gtk-WARNING **: 19:16:54.257: Locale not supported by C library.
Using the fallback 'C' locale.

(pinentry-gtk-2:1958): Gtk-WARNING **: 19:16:54.286: Unable to locate theme engine in module_path: "adwaita",

(pinentry-gtk-2:1958): Gtk-WARNING **: 19:16:54.294: Unable to locate theme engine in module_path: "adwaita",
tomb [D] get_lukskey
tomb [D] get_lukskey returns 0
tomb . Password OK.
tomb [E] Failure mounting the encrypted file.

[jaromil]

mount fails, anything related in dmesg? can't tell if the tomb was correctly formatted and how, however you can use devices appearing in /dev/mapper just like normal volumes, to check their integrity.

[bugsug]

Dmesg does not say something related about the failed mounting of the tomb. In /dev/mapper only control appears, which is not a valid block device.
However, when I try to lock a new tomb, it fails with following output:
tomb [D] Identified caller: ___ (1000:985)
tomb [D] Tomb command: lock Test123
tomb [D] Caller: uid[1000], gid[985], tty[/dev/pts/2].
tomb [D] Temporary directory: /tmp/zsh
tomb . Commanded to lock tomb Test123
tomb [D] Tomb found: Test123
tomb [D] Loop mounted on /dev/loop0
tomb . Checking if the tomb is empty (we never step on somebody else's bones).
tomb . Fine, this tomb seems empty.
tomb [D] load_key argument: Test123.key
tomb [D] load_key: Test123.key
tomb [D] is_valid_key
tomb . Key is valid.
tomb . Locking using cipher: aes-xts-plain64:sha256
tomb . A password is required to use key Test123.key
tomb [D] asking password with tty=/dev/pts/2 lc-ctype=en_US.UTF8
tomb [D] using pinentry-gtk2

(process:8007): Gtk-WARNING **: 16:28:15.941: Locale not supported by C library.
Using the fallback 'C' locale.

(pinentry-gtk-2:8007): Gtk-WARNING **: 16:28:15.972: Unable to locate theme engine in module_path: "adwaita",

(pinentry-gtk-2:8007): Gtk-WARNING **: 16:28:15.981: Unable to locate theme engine in module_path: "adwaita",
tomb [D] get_lukskey
tomb [D] get_lukskey returns 0
tomb . Password OK.
tomb (*) Locking Test123 with Test123.key
tomb . Formatting Luks mapped device.
tomb [W] cryptsetup luksFormat returned an error.
tomb [E] Operation aborted.

[jaromil]

I suspect is not a problem in tomb, you need to know what cryptsetup returns and if there are the filesystems needed (ext3/4).

[bugsug]

Where can I find the cryptsetup log?

[jaromil]

If you use your key in clear as indicated in Tomb's README you can get to the bare mount operation:

lo=$(losetup -f)
losetup -f secret.tomb
pass="$(gpg -d secret.key)"
echo -n -e "$pass" | cryptsetup --key-file - luksOpen $lo secret
mount /dev/mapper/secret /mnt
unset pass

then you can start adding arguments to cryptsetup according to its manpage, to have a verbose output about its problems. I hope you find out, can't help any further than this. I also recommend testing all on another system just to be sure, for instance a liveCD like heads.dyne.org

[bugsug]

Thanks!

[bugsug]

The output of cryptsetup:
echo -n -e "$pass" | sudo cryptsetup --debug --key-file - luksOpen $lo secret
cryptsetup 2.0.2 processing "cryptsetup --debug --key-file - luksOpen /dev/loop0 secret"
Running command open.
Locking memory.
Installing SIGINT/SIGTERM handler.
Unblocking interruption on signal.
Allocating context for crypt device /dev/loop0.
Trying to open and read device /dev/loop0 with direct-io.
Trying to open device /dev/loop0 without direct-io.
Initialising device-mapper backend library.
Trying to load any crypt type from device /dev/loop0.
Crypto backend (gcrypt 1.8.2) initialized in cryptsetup library version 2.0.2.
Detected kernel Linux 4.15.2-gnu-1 i686.
Loading LUKS2 header.
Opening lock resource file /run/cryptsetup/L_7:0
Acquiring read lock for device /dev/loop0.
Verifying read lock handle for device /dev/loop0.
Device /dev/loop0 READ lock taken.
Trying to read primary LUKS2 header at offset 0.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 8192.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 16384.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 32768.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 65536.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 131072.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 262144.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 524288.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 1048576.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 2097152.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
Trying to read secondary LUKS2 header at offset 4194304.
Opening locked device /dev/loop0
Veryfing locked device handle (bdev)
LUKS2 header read failed (-5).
Device /dev/loop0 READ lock released.
Releasing crypt device /dev/loop0 context.
Releasing device-mapper backend.
Unlocking memory.
Command failed with code -1 (wrong or missing parameters).

[bugsug]

How could the LUKS2 header be destroyed? This is a new hard drive which I only used for the backup.

[bugsug]

Is there any possibility to restore the LUKS2 header? I read that it is impossible to decrypt the tomb without the information out of the header but I still have a little bit hope.

[jaromil]

wow yes, you guess right the header is not there and cryptsetup seems to try reading at different offsets for repair. So that's the problem. Technically speaking and AFAIK the only thing you need from the header really is the Salt value (which is also a sort of secret) and a way to reconstruct the header.
So yes you have the key (which is symmetric so its enough) and that seems to be working. But I'm puzzled by this situation since tomb wouldn't allow you to copy things inside without a working header in the first place. The downside of all this is that even if you retrieve the Salt you may be discovering that the rest of the tomb is compromised in a similar way.
A further "forensic" approach: dump as possible all the data, compare the header sector to a normally formed LUKS header byte by byte (perhaps retrieved by a newly created tomb) and see what differs.

[bugsug]

Thanks for the explanation. I feel like doing something new every day. Yesterday I dealt the first time in more detail with encryption using dm_crypt, the next time I will have to dive deeper into the file system.
Later this week I will also try opening the tomb on a completely different system.
Any suggestions on how to compare the header sectors on byte level?

[jaromil]

using xxd or hexdump you can have a good view

[jaromil]

BTW your case confirms my plan to add a feature in tomb that makes a backup of the header, should call it "tombstone" perhaps.

[bugsug]

👍 Thank you very much. I will try it in the next time.

[bugsug]

That's why I love free (as in freedom) software: You can see how it works, where it fails, improve it and there's a great community around which helps you when you are in trouble.

[bugsug]

Source: Von Vernon Raineil Cenzon thevernon - https://unsplash.com/photos/bLpmMruu97Y, CC0, Link

A few minutes ago I tried to open it on a Open SUSE Leap 42.3 system with self-compiled tomb 2.5 and you have 3 tries to guess why I posted a firework picture:
It worked 😄!!!
Please do not immediately close this issue. I want to try to find out why there were errors on the other system.
Thank you very much for the great help!

[jaromil]

Lovely, I'm happy for you. You also keep good nerves and never gave up. Feel free to keep this post alive, I think is an informative journey into troubleshooting. Ciao.

[bugsug]

Giving up was no alternative: In the tomb was stored the only backup of a broken hard drive with a programmer's, a pro-user's and a part-time hobby photographer's work of seven months. I will change my backup strategy. The backup in the tomb will not be my only backup in the future, I will put a second unencrypted backup on another drive.
Thanks again for the great help!

[hyiltiz]

Does this mean LUKS2 is now supported? I am a bit confused as it is not mentioned in the docs and the few issues that mention LUKS2 are from a few years ago, e.g., #343. Does LUKS2 provide technical advantages over LUKS1 as far as security is concerned? Is LUKS2 the default in tomb now, or is there a way to use it? I dug up a tomb yesterday and had to patch the tomb bash script myself to _cryptsetup function specifies --type luks1 so tomb lock could work. (Maybe I should open another issue asking for LUKS2 support instead of bikeshedding here?)

[jaromil]

Does this mean LUKS2 is now supported?

Tomb volumes are LUKS1 only. This is hardcoded around line 1263 of the tomb script

Does LUKS2 provide technical advantages over LUKS1 as far as security is concerned?

Most advantages are about integrity, some improvement to brute-force protection, see the LUKS2 FAQ

Is LUKS2 the default in tomb now, or is there a way to use it?

No is not default and no there is no way to use it up until now with Tomb 2.x series.

I dug up a tomb yesterday and had to patch the tomb bash script myself to _cryptsetup function specifies --type luks1 so tomb lock could work.

You are patching an old version of the tomb script since this fix is now included. There should be no danger in doing so, however keep an eye on KNOWN BUGS to see if anything has been happening that affects your setup.

(Maybe I should open another issue asking for LUKS2 support instead of bikeshedding here?)

As you like, its on my mind anyway, but with low priority. Since this change will introduce multiple incompatible formats, I'm also considering wider support for tc-play and veracrypt and this may all converge in a new major Tomb v3 release some day.