Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--stdin-filename with path separator gives cryptic error message #2063

Closed
cdhowie opened this issue Oct 26, 2018 · 3 comments · Fixed by #2206
Closed

--stdin-filename with path separator gives cryptic error message #2063

cdhowie opened this issue Oct 26, 2018 · 3 comments · Fixed by #2206
Labels
state: need triaging need categorizing, labeling, next-step decision type: bug

Comments

@cdhowie
Copy link
Contributor

cdhowie commented Oct 26, 2018

Output of restic version

restic 0.9.3 compiled with go1.11.1 on linux/amd64

How did you run restic exactly?

RESTIC_REPOSITORY set to S3 bucket+prefix, and RESTIC_PASSWORD set appropriately.

# sudo -iu postgres pg_dumpall | restic backup -q --stdin --stdin-filename /root/postgresql-dump.sql
Fatal: unable to save snapshot: Lstat: file does not exist

What backend/server/service did you use to store the repository?

S3

Expected behavior

The backup should complete successfully, or at least a useful error message should be provided. I would not expect Restic to need to stat anything as part of this operation; the repository is in S3 and the file data is provided on standard input.

Actual behavior

Restic gives a "file does not exist" error message.

Steps to reproduce the behavior

Attempt to upload a file with --stdin-filename containing a path separator.

Do you have any idea what may have caused this?

It appears that Restic joins the provided filename to the current working directory Restic is run from; when I omit /root/ then the file is stored in the repository as /root/postgresql-dump.sql (cwd for the command is /root).

Do you have an idea how to solve the issue?

I would suspect this needs a patch to be fixed. A workaround is to cd before running Restic so that it's running from the directory where you want the backed-up file to appear to be stored.

Did restic help you or made you happy in any way?

Yes, this has become my go-to backup tool for all of my machines. My company is using it in production to backup multiple critical servers for quick recovery, while also providing inexpensive and easily-updated offsite backups.

@cdhowie
Copy link
Contributor Author

cdhowie commented Oct 29, 2018

As a follow up, I believe the stat errors are coming from restic trying to build the tree objects. I don't know for certain, but I suspect that with --stdin-filename /root/postgresql-dump.sql and a cwd of /root that restic is building the full path /root/root/postgresql-dump.sql. Then, when it tries to build the tree object for /root/root, it can't find that directory and chokes.

So I think the fix here is just to make restic able to handle absolute paths to --stdin-filename.

@fd0
Copy link
Member

fd0 commented Jan 6, 2019

Thanks for the report, it's a bug!

@fd0 fd0 added type: bug state: need triaging need categorizing, labeling, next-step decision labels Jan 6, 2019
@garrmcnu
Copy link
Contributor

As pointed out, the error is caused by the parent directories. The fs.Reader Lstat() method succeeds for the filename (e.g. /root/postgresql-dump.sql) but returns os.ErrNotExist for all of the parent directories (e.g. /root).
Lstat() currently only allows "/" and "." directories.

Updated Lstat() to also allow the parent directory components of the filename.

Not sure if it makes sense to also allow relative directory paths, given that the snapshot path is prepended with "/" so will always be absolute?

garrmcnu added a commit to garrmcnu/restic that referenced this issue Mar 11, 2019
garrmcnu added a commit to garrmcnu/restic that referenced this issue Mar 11, 2019
Return valid directory info from Lstat() for parent directories of the
specified filename. Previously only "/" and "." were valid directories.

Also set directory mode as this is checked by archiver.

Closes restic#2063
garrmcnu added a commit to garrmcnu/restic that referenced this issue Apr 28, 2019
garrmcnu added a commit to garrmcnu/restic that referenced this issue Apr 28, 2019
garrmcnu added a commit to garrmcnu/restic that referenced this issue Apr 28, 2019
Return valid directory info from Lstat() for parent directories of the
specified filename. Previously only "/" and "." were valid directories.

Also set directory mode as this is checked by archiver.

Closes restic#2063
fd0 pushed a commit to garrmcnu/restic that referenced this issue May 8, 2019
fd0 pushed a commit to garrmcnu/restic that referenced this issue May 8, 2019
fd0 pushed a commit to garrmcnu/restic that referenced this issue May 8, 2019
Return valid directory info from Lstat() for parent directories of the
specified filename. Previously only "/" and "." were valid directories.

Also set directory mode as this is checked by archiver.

Closes restic#2063
@fd0 fd0 closed this as completed in #2206 May 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
state: need triaging need categorizing, labeling, next-step decision type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants