Merge eugeneia/snabb:timeline-raptorjit into Vita #65
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a very useful instruction for self-benchmarking programs that want to read the CPU timestamp counter efficiently. See Intel whitepaper for details: http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-32-ia-64-benchmark-code-execution-paper.pdf
Use 'double' instead of 'uint64_t' for values in the timeline file. This change is motivated by making timeline files easier to process by R. In the future we may switch back to uint64_t for the TSC counter and/or argument values for improved precision. The major_version file header field can be used to avoid confusion. The obvious downside to using doubles is that the TSC value will lose precision as the server uptime increases (the TSC starts at zero and increases at the base frequency of the CPU e.g. 2GHz.) The impact seems to be modest though. For example a 2GHz CPU would start rounding TSC values to the nearest 128 (likely quite acceptable in practice) after approximately 2 years of operation (2^53 * 128 cycles.) So - storing the TSC as a double-float is definitely a kludge - but unlikely to cause any real harm and expedient for the short-term goal of putting this code to use without getting blocked due to e.g. my lack of sophisticated as an R hacker.
Resolved conflict in app.lua between adding timeline events and the new breath topological-sort machinery.
Simplify the code and eliminate unwanted branches from the engine loop
by drawing a random timeline level from a log-uniform distribution that
mathematically favors higher log levels over lower ones.
Plucked log5() out of the air i.e. each log level should be enabled for
5x more breaths than the one below.
Here is how the distribution of log level choice looks in practice using
this algorithm:
> t = {0,0,0,0,0,0,0,0,0}
> for i = 1, 1e8 do
local n = math.max(1,math.ceil(math.log(math.random(5^9))/math.log(5)))
t[n] = t[n]+1
end
> for i,n in ipairs(t) do print(i,n) end
1 560
2 2151
3 10886
4 55149
5 273376
6 1367410
7 6844261
8 34228143
9 171120244
Note: Lua provides only natural logarithm functions but it is easy to
derive other bases from this (google "log change of base formula").
I suspect that it is a misfeature for the timeline to sample the contents of packets. Do we really want user data potentially appearing in debug logs? Removed for now.
Cleanup timeline integration in core.app a little along the merge.
…les" This reverts commit 0e52b24.
This fixes a bug where timeline log level was rerolled between end of breaths but before before post-breath events, causing sampling to affect the event lag of the polled_timers event.
Changes the syntax of event specs to
<level>,<rate>|<eventname>: ...
The previous level digit becomes the event’s "rate" and retains its semantics
with regard to the logging frequency of the specified event. The "stack depth"
of the event is now decoupled as the new, leading level digit and specified
independently. The new level semantics are as follows:
- level ranges from 0-9 (10 levels in total)
- 0 is the top most level while 9 in the lowest
- levels 0-4 are reserved for use by the engine
- user applications can use levels 5-9 to create hierarchy in their events
Caveat: users should avoid defining events with a higher level and a lower
event rate than an enclosed event if the higher level event is supposed to
serve as a latency anchor for the lower level event.
RIGHT WRONG
5,3|op_start: 5,2|op_start:
6,2|op_iter: 6,3|op_iter:
5,3|op_end: 5,2|op_end:
In the left most WRONG example, the anchor of the op_inter event depends on the
log rate at the time of sampling.
randomize_log_rate will then set the rate to values between 1 and 6, enabling the respective events.
Realized: 9 is the rate that means "log always". Elevate some rare but important events to rate 9.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This builds on @lukego’s work on the timeline log, a probabilistic flight recorder for Snabb, see: snabbco#849 snabbco#873 snabbco#916 snabbco#973 snabbco#1011 snabbco#1098 snabbco#1112
My current working branch for the timeline is at eugeneia/snabb:timeline-raptorjit, this branch merges this feature into Vita and adds some application specific events (user events).
See #58 for some example plots.