-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Permission denied issue when writing to /tmp/.tensorboard-info
#2010
Comments
/tmp/.tensorboard-info
Hi @tete1030! Thanks for the clear report. This is a good point—I’d I think that the following patch should suffice, at least on Unices: diff --git a/tensorboard/manager.py b/tensorboard/manager.py
index a86c010b..92f7601f 100644
--- a/tensorboard/manager.py
+++ b/tensorboard/manager.py
@@ -235,6 +235,7 @@ def _get_info_dir():
The directory will be created if it does not exist.
"""
path = os.path.join(tempfile.gettempdir(), ".tensorboard-info")
+ old_umask = os.umask(0o000)
try:
os.makedirs(path)
except OSError as e:
@@ -242,6 +243,8 @@ def _get_info_dir():
pass
else:
raise
+ finally:
+ os.umask(old_umask)
return path
I’ll have to test this on Windows. If you’re looking for a quick fix, |
Sorry that I didn't reply. The patch works great and I have not encountered this problem again. |
Great—glad to hear that the patch is working, @tete1030 (and sorry for @hgadig: Yes, please keep this open. |
Hey there, just stumbled upon this, I just thought I'd mention that this patch doesn't fix the issue from the non-sudo user perspective. I added the patch first, but It did not fix the problem, as I do not have write access on that directory anyway. I'm gonna try and change the ".tensorboard-info" name to something unique, and hope this works. I'm thinking it might be nice to be able to customize this location, on a per-user basis ? |
@lebrice, could you clarify what you mean by the “non-sudo user (To be clear, the patch needs to be applied before the info directory |
My workaround while this issue is resolved involves setting the TMPDIR environment variable. Since tensorboard uses tempfile which will respect user set environment variables (TMPDIR, TMP, etc.), this is possible. Make sure that the directory which TMPDIR points to exists! You can change and test the new temp directory by running the following (which sets it to /tmp/$USER instead of /tmp). export TMPDIR=/tmp/$USER; mkdir -p $TMPDIR; python -c "import tempfile; print(tempfile.gettempdir())" A safe tensorboard invocation is: export TMPDIR=/tmp/$USER; mkdir -p $TMPDIR; tensorboard --logdir $LOGDIR |
Also, for @wchargin, the issue being referenced as the multi-user scenario is:
|
Right, I understand this; thanks. (That’s what this issue is about.) And |
@wchargin I have encountered this problem and used the method you provided. I still can't solve it. |
Make sure you have deleted the old /tmp/.tensorboard-info, or use Be aware if you have only patched tensorboard installed in one environment (e.g. a conda env, or locally installed package), other user with unpatched env could also cause this problem |
Summary: Fixes #2010. This patch is primarily directed at Unices. On Windows, the underlying problem should not be an issue, because the default temporary directory seems to be user-scoped (e.g., `C:\Users\wchargin\AppData\Local\Temp`). Test Plan: Unit tests added. Before this change, the first test has an assertion error (`'0o755' != '0o777'`) and the second test has an I/O error trying to write to a non-writable directory. After this change, all tests pass. Tested only on Linux. wchargin-branch: info-dir-mode
…eful (#2131) Summary: Fixes #2010. This patch is primarily directed at Unices. On Windows, the underlying problem should not be an issue, because the default temporary directory seems to be user-scoped (e.g., `C:\Users\wchargin\AppData\Local\Temp`). Test Plan: Unit tests added. Before this change, the first test has an assertion error (`'0o755' != '0o777'`) and the second test has an I/O error trying to write to a non-writable directory. After this change, all tests pass. Tested only on Linux. wchargin-branch: info-dir-mode
So does it means that in one Linux Server, only one user can use tensorboard freely? And if others want to use it, he/she must get root privilege to fix the problem? |
@minygd: This issue has been fixed, and the fix will be in the next |
@wchargin I am using ubuntu with tb-nightly 1.14.0a20190506 and I still have the issue.
|
@reactivetype: Could you please run
and post the full output here? |
@wchargin thanks for replying. here is the output:
|
@reactivetype: Thanks. It looks like your
|
@wchargin Makes sense. Thank you very much! |
Summary: Users often report problems that depend on environment-specific configuration. Rather than asking them to find all this information and enter it into an issue template manually, we can ask them to run a script and paste its output verbatim into the issue. Furthermore, we can detect and suggest fixes to common problems, such as #1907 and #2010. The script can grow as we see fit to add new diagnoses and suggestions. Test Plan: Open to suggestions about how much automated testing there should be. The structure of the script makes it robust to errors in each individual diagnosis (in the worst case, it prints a stack trace and continues to the next one), and I tested that the main framework works in Python 2 and 3 on Linux and in Python 3 on Windows. To simulate a bad hostname, add ```python socket.getfqdn = lambda: b"\xc9".decode("utf-8") ``` to the top of the script. To simulate a bad `.tensorboard-info` directory and test the quoting behavior, run export TMPDIR="$(mktemp -d)/uh oh/" && mkdir -p "${TMPDIR}/.tensorboard-info" && chmod 000 "${TMPDIR}/.tensorboard-info" && python ./tensorboard/tools/diagnose_me.py wchargin-branch: diagnose-me
Summary: Users often report problems that depend on environment-specific configuration. Rather than asking them to find all this information and enter it into an issue template manually, we can ask them to run a script and paste its output verbatim into the issue. Furthermore, we can detect and suggest fixes to common problems, such as #1907 and #2010. The script can grow as we see fit to add new diagnoses and suggestions. Test Plan: The script is designed to be robust to errors in each individual diagnosis: in the worst case, it prints a stack trace and continues to the next one. I’ve manually tested that the main framework works in Python 2 and 3 on Linux and in Python 3 on Windows. Automated testing of this script is possible, but would take a fair number of CPU cycles to run tests (setting up virtualenvs and Conda, installing and importing TensorFlow many times). Given that this script is never a production dependency and is explicitly designed to be run in a discussion context, light testing seems reasonable. To simulate a bad hostname, add ```python socket.getfqdn = lambda: b"\xc9".decode("utf-8") ``` to the top of the script. To simulate a bad `.tensorboard-info` directory and test the quoting behavior, run ```shell export TMPDIR="$(mktemp -d)/uh oh/" && mkdir -p "${TMPDIR}/.tensorboard-info" && chmod 000 "${TMPDIR}/.tensorboard-info" && python ./tensorboard/tools/diagnose_tensorboard.py ``` To cross-check the autoidentification logic: ``` $ python tensorboard/tools/diagnose_tensorboard.py | > awk '/version / { print $NF }' e093841ffaea564cb2410e0b430bd0c552ada208 $ git hash-object tensorboard/tools/diagnose_tensorboard.py e093841ffaea564cb2410e0b430bd0c552ada208 $ git rev-parse HEAD:tensorboard/tools/diagnose_tensorboard.py e093841ffaea564cb2410e0b430bd0c552ada208 $ git cat-file blob e093841ffaea564cb2410e0b430bd0c552ada208 | > diff -u tensorboard/tools/diagnose_tensorboard.py - | wc -l 0 ``` wchargin-branch: diagnose-me
Just try
|
hey @wchargin I have installed tensorboard 1.14 but I face the same issues. Seems like I have a |
@alwynmathew: Who owns the |
@alwynmathew: I have encountered the same problem while working on a shared machine. Since the admin is on christmas break, I had to find another solution. Turns out, tensorboard uses It is possible to specify the temp directory returned by Now we can simply force tensorboard to use another directory then mkdir /tmp/.selgs
env TMPDIR=/tmp/.selgs tensorboard |
Yep, using (You probably want to use either |
Sorry @SimonSelg , I was in Christmas break too. @wchargin latest solution seems elegant with just one command. I had a tough time fixing my issues, I tracked down the owner of the dir with |
tensorboard
)1.13.1
1.13.1
Ubuntu 16.04
2.7
Please describe the bug as clearly as possible, and if possible provide a minimal example (code, data, and/or command line) to reproduce the issue. Thanks!
The writing of tensorboard info files introduced in #1806 can cause permission problem under multi-user scenario. It directly create
.tensorboard-info
directory under/tmp
as intensorboard/tensorboard/manager.py
Line 237 in 5fc3c8c
The text was updated successfully, but these errors were encountered: