Better understanding transitive sizes

[wmertens]

Would it be possible to track 3 sizes: the package size and closure size like now, and the "single-owner" closure size?

Right now, if two packages depend on the same thing, the ownership is muddled. So I wonder if it would be more obvious if we could see the closure size of things that are not shared with other closures at the same or closer distance from the root?

Given this tree:

A -> D
B -> D
C -> A, E
D
E -> D
F -> A,C
root -> F,B

the sizes would be

pkg	size	closure	single-owner
A	A	AD	0
B	B	BD	B
C	C	ACDE	E
D	D	D	0
E	E	ED	0
F	F	ACDEF	ACEF
root	0	ABCDEF	ABCDEF

This would show us that F is the biggest package, and while C has a big closure, removing it will only remove E

[utdemir]

Hey, thank you for creating the issue!

If I understood you correctly, this feature is already implemented (but it's totally undocumented, so I don't expect anyone to figure it out by themselves) :). On nix-tree panes, every store path has two sizes, something like:

sudo-1.9.5p2                      50.17 MiB (4.65 MiB)

Using your terminology first size there is the "closure size", and the second parenthesised size is "single-owner" size.

It's called "added size" in the codebase; as in "how much size does depending on that store path add on top of everything else", or alternatively "how much size do I save if I were to stop depending on that specific derivation". To be honest I think, both "added size" and "single owner size" is a bit arbitrary, but I don't have anything better.

But as you can imagine, I have no idea how to make it clearer on the application that it is what it is. Your explanation on this issue is pretty accurate, so I think we should add it to README (probably rendering that dependency tree as a graph so it's easier to see). But I wish there was a way to present that number in the app in a way that is obvious without looking at the documentation. Do you have any idea on how to do this?

[wmertens]

I'm thinking that it would be most obvious to repeat the added size in the status line. There's enough room there to write Size this package is solely responsible for: x.

Could I also ask for the option to sort the entries by added size?

[utdemir]

I'm thinking that it would be most obvious to repeat the added size in the status line. There's enough room there to write Size this package is solely responsible for: x.

Thanks for the suggestion. That makes sense, but it was a bit wordy, so I added "Added size" to the status line, and added a "Glossary" to the README.md. Let's see if we'll get the same question again :).

Could I also ask for the option to sort the entries by added size?

This is a similar question to #5. Since it seems like a common request, I implemented this functionality on df01088 (tl;dr: the key s toggles the order, and there are somes visual cues).

---

I'll cut a release tomorrow (after looking at your other issue #15), meanwhile I'm closing this issue, but please do let me know what you think!

[hraban]

What if you changed it to:

A -> D
B -> D
C -> A, E
D
E -> D
F -> A,C
G -> A,C
root -> F,G,B

That is: introduce another package G which is essentially a copy of F

Do I understand correctly that it would effectively obscure the size of both?

pkg	size	closure	single-owner
A	A	AD	0
B	B	BD	B
C	C	ACDE	E
D	D	D	0
E	E	ED	0
F	F	ACDEF	F
G	G	ACDEG	G
root	0	ABCDEFG	ABCDEFG

And so on for 3, 4, 5 etc? It feels like "1" is only a special case of a more general problem. Although it does seem like this problem is generally intractable.

The problem this tool tries to solve reminds me of performance profiling, and flame graphs. But the top comment on a HN thread about them is poignant and seems to ask a similar question. They do mention pprof which can output a dotfile with weighted nodes--maybe that could be a nice way to solve this for nix-tree?

• https://www.brendangregg.com/flamegraphs.html
• https://news.ycombinator.com/item?id=40798828
• https://slcjordan.github.io/images/pprof/graphviz.png

[utdemir]

Do I understand correctly that it would effectively obscure the size of both?

Yes! I think your graph summarizes the meaning of the single-owner thing well.

I know that it does not feel very consistent. This is another form of issues caused by nix-tree projecting a DAG to a tree. It means something like "how much the total closure size reachable from the roots would change, if I were to remove the edges between this derivation and its parents.".

I'm sure we can tackle/workaround the performance issue with it. Worst case we'll run it in a background thread or something. But my problem is more about its UX.

I found it useful in some cases, but you're right that it's not clear at all. This is pretty much the only part people are confused about nix-tree. On the other hand, there are also multiple people who has asked for this exact feature without figuring out it already exists, so it is somehow useful.

These days I keep thinking about the "marking" feature described here #15. Maybe we need to be able to:

• Mark all downstream (or upstream) dependencies of a specific derivation, maybe with different colors
• And show aggregates for the marked derivations.

So, when someone wants to replicate the added size, they can go to the derivation, "mark" all the derivations downstream, and the total size of the "unmarked" closures would be the added size. But they can also mark multiple derivations to answer some more questions.