-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MYST_RETAIN_SYMBOLS env var to retain symbols #1229
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. For my understanding, do the replays not themselves invoke the load and unload symbol paths?
I've tried to explain this in the PR description. Can you let me know if that is not clear so that I can reword it if needed? |
Normally, when run under a debugger, Mystikos writes out each loaded shared library to /tmp/myst<abcdef>/ so that the debugger can load symbols from these libraries. At the end of the execution these files are deleted. [rr debugger](https://rr-project.org/) allows `record and replay` debugging. You record a failure once, then debug the recording, deterministically, as many times as you want. The same execution is replayed every time. rr also provides efficient reverse execution under gdb. Set breakpoints and data watchpoints and quickly reverse-execute to where they were hit. At a very high-level, rr works by capuring all kernel interactions, including thread scheduling, and then re-applying all the interactions during playback. The playback is not a real-execution, and does not produce most side effects (e.g writing files, opening sockets etc). Thus, during playback, the shared libraries necessay for the debugger are not written out. To make rr work for Mystikos, the symbol files need to be retained during execution. That is the purpose of the MYST_RETAIN_SYMBOLS flag. ------------------------------------------------------------------------------------------ Prerequisites: - Install rr from https://github.com/rr-debugger/rr/releases - Read up https://rr-project.org/ - Try out rr on a simple hellworld program outside mystikos $ rr record helloworld $ rr replay rr will suggest the necessary configuration settings that need to be made. Do as suggested. Usage: For a test, doing `make TARGET=linux` would print out the actual Mystikos command. Execute `MYST_RETAIN_SYMBOLS=1 rr record <mystikos command>` to create a recording. E.g: $ MYST_RETAIN_SYMBOLS=1 rr record myst exec-linux rootfs /bin/hello red green blue rr: Saving execution to trace directory `/home/user/.local/share/rr/myst-35'. Hello world! I received: argv[0]={/bin/hello}, argv[1]={red}, argv[2]={green}, argv[3]={blue} === passed test (/bin/hello) To replay and debug, do `rr replay -d /path/to/myst-gdb` E.g: $ rr replay -d ../../../build/bin/myst-gdb GNU gdb (Ubuntu 11.1-0ubuntu2) 11.1 0x00007fe413b720d0 in _start () from /lib64/ld-linux-x86-64.so.2 (rr) Most GDB commands are supported; but keep in mind that this is a replay, not an actual execution. It is like watching a video where you can easily skip forward and backward, but not actually waching something actually happen. Put a breakpoint: (rr) b main Breakpoint 2 at 0x10000024e32: file host.c, line 527. (rr) c Continuing. Breakpoint 2, main (argc=7, argv=0x7ffda41c9688, envp=0x7ffda41c96c8) at host.c:527 527 int ec = _main(argc, argv, envp); (rr) c Continuing. oegdb: Loaded enclave module /home/anand/msft/mystikos/build/lib/libmystkernel.so oegdb: analyzing symbols for module /home/anand/msft/mystikos/build/lib/libmystkernel.so oegdb: Loaded enclave module ./.mystX68Tlv/libmystcrt oegdb: analyzing symbols for module ./.mystX68Tlv/libmystcrt oegdb: Loaded enclave module ./.mystX68Tlv/hello oegdb: analyzing symbols for module ./.mystX68Tlv/hello Breakpoint 2, main (argc=4, argv=0x7fe412cf2010) at hello.c:10 10 assert(argc == 4); (rr) Step over, debug as before. (rr) n 11 assert(strcmp(argv[0], "/bin/hello") == 0); (rr) n 12 assert(strcmp(argv[1], "red") == 0); (rr) n 13 assert(strcmp(argv[2], "green") == 0); (rr) n 14 assert(strcmp(argv[3], "blue") == 0); Reverse execution works, but seems flaky. Needs more investigation. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
d820ece
to
45226df
Compare
Makes sense, the symbol loading and unloading syscalls are not performed during replay. |
@vtikoo Feel free to merge this PR if it is useful. |
@anakrish yes would be great to get this merged. Its complaining about rebase conflicts, could you take a look? |
I don't have the original branch and I'm no longer set up to work on Mystikos. |
Changes commited by #1449 |
Normally, when run under a debugger, Mystikos writes out each loaded shared library
to /tmp/myst/ so that the debugger can load symbols from these libraries.
At the end of the execution these files are deleted.
rr debugger allows
record and replay
debugging.You record a failure once, then debug the recording, deterministically, as
many times as you want. The same execution is replayed every time.
rr also provides efficient reverse execution under gdb. Set breakpoints and
data watchpoints and quickly reverse-execute to where they were hit.
At a very high-level, rr works by capuring all kernel interactions,
including thread scheduling, and then re-applying all the interactions during playback.
The playback is not a real-execution, and does not produce most side effects
(e.g writing files, opening sockets etc).
Thus, during playback, the shared libraries necessay for the debugger are not written out.
To make rr work for Mystikos, the symbol files need to be retained during execution.
That is the purpose of the MYST_RETAIN_SYMBOLS flag.
Prerequisites:
$ rr record helloworld
$ rr replay
rr will suggest the necessary configuration settings that need to be made.
Do as suggested.
Usage:
For a test, doing
make TARGET=linux
would print out the actual Mystikos command.Execute
MYST_RETAIN_SYMBOLS=1 rr record <mystikos command>
to create a recording.E.g:
$ MYST_RETAIN_SYMBOLS=1 rr record myst exec-linux rootfs /bin/hello red green blue
rr: Saving execution to trace directory `/home/user/.local/share/rr/myst-35'.
Hello world!
I received: argv[0]={/bin/hello}, argv[1]={red}, argv[2]={green}, argv[3]={blue}
=== passed test (/bin/hello)
To replay and debug, do
rr replay -d /path/to/myst-gdb
E.g:
$ rr replay -d ../../../build/bin/myst-gdb
GNU gdb (Ubuntu 11.1-0ubuntu2) 11.1
0x00007fe413b720d0 in _start () from /lib64/ld-linux-x86-64.so.2
(rr)
Most GDB commands are supported; but keep in mind that this is a replay, not an actual
execution. It is like watching a video where you can easily skip forward and backward,
but not actually waching something actually happen.
Put a breakpoint:
(rr) b main
Breakpoint 2 at 0x10000024e32: file host.c, line 527.
(rr) c
Continuing.
Breakpoint 2, main (argc=7, argv=0x7ffda41c9688, envp=0x7ffda41c96c8) at host.c:527
527 int ec = _main(argc, argv, envp);
(rr) c
Continuing.
oegdb: Loaded enclave module /home/anand/msft/mystikos/build/lib/libmystkernel.so
oegdb: analyzing symbols for module /home/anand/msft/mystikos/build/lib/libmystkernel.so
oegdb: Loaded enclave module ./.mystX68Tlv/libmystcrt
oegdb: analyzing symbols for module ./.mystX68Tlv/libmystcrt
oegdb: Loaded enclave module ./.mystX68Tlv/hello
oegdb: analyzing symbols for module ./.mystX68Tlv/hello
Breakpoint 2, main (argc=4, argv=0x7fe412cf2010) at hello.c:10
10 assert(argc == 4);
(rr)
Step over, debug as before.
(rr) n
11 assert(strcmp(argv[0], "/bin/hello") == 0);
(rr) n
12 assert(strcmp(argv[1], "red") == 0);
(rr) n
13 assert(strcmp(argv[2], "green") == 0);
(rr) n
14 assert(strcmp(argv[3], "blue") == 0);
Reverse execution works, but seems flaky. Needs more investigation.
Signed-off-by: Anand Krishnamoorthi anakrish@microsoft.com