Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

performance comparison and readme clarifications #138

Closed
matu3ba opened this issue Sep 12, 2023 · 4 comments
Closed

performance comparison and readme clarifications #138

matu3ba opened this issue Sep 12, 2023 · 4 comments

Comments

@matu3ba
Copy link

matu3ba commented Sep 12, 2023

You call it "Blazing Fast", but do not show benchmarks or comparisons to other terminal file managers.

Some brief design questions:

  • How will you ensure stuff works, ie to limit the scope, as you seem to integrate all functionality for what I believe is reasonably optimal performance?
  • "Vim-like Input component, and Select component" is also very open-ended, since vim has a lot of functionality.

Some brief comments on README:

task priority assignment

Internal one or also by the Kernel scheduler? Use "internal task priority assignment", if you dont't use sched_* class functions ie on Linux etc.

All I/O operations are asynchronous

Are they using blocking syscalls or non-blocking ones (io_uring)? Would sounds nicer, if you mention asynchronous executed and using non-blocking syscalls, if available.

pre-caching mechanism

How much additionally memory does this use? Is this configurable? Overall memory consumption would be also nice to know and track to estimate what kind of low(er) end devices can be used with yazi.

@sxyazi
Copy link
Owner

sxyazi commented Sep 12, 2023

You call it "Blazing Fast", but do not show benchmarks or comparisons to other terminal file managers.

I don't understand the real significance of this comparison.

The functionality offered by each file manager varies significantly, and when categorized by language, currently only Joshuto is implemented using Rust.

However, it doesn't provide out-of-the-box image preview but rather relies on users to implement it themselves, which depends on how users choose to do so. As an old user, I followed the official document using the combination of Joshuto + icat and found it to be practically unusable.

When navigating quickly with the cursor, all the images stack up together. I believe this is more of a distinction between "usable" and "unusable" rather than a performance issue.

If we must compare performance, I can say that Yazi is much better because:

  • It has a pre-caching mechanism (explained below)
  • Preview tasks are cancellable. When performing rapid file navigation, if the previous task is still unfinished, it will be directly discarded)
  • Chunk loading for large directories (perf: load large folders in chunks #117)
  • For code highlighting, if your terminal's height is 10, only the first 10 lines will be highlighted. This is different from other file managers that rely on external programs (like bat), which highlight the entire content and then only display the first 10 lines.

How will you ensure stuff works, ie to limit the scope, as you seem to integrate all functionality for what I believe is reasonably optimal performance?

The term "stuff" here is too broad, and I'm not sure how to answer it. However, I will still attempt to write an article later to introduce the internal workings of Yazi from my perspective.

"Vim-like Input component, and Select component" is also very open-ended, since vim has a lot of functionality.

keymap = [
{ on = [ "<C-q>" ], exec = "close", desc = "Cancel input" },
{ on = [ "<Enter>" ], exec = "close --submit", desc = "Submit the input" },
{ on = [ "<Esc>" ], exec = "escape", desc = "Go back the normal mode, or cancel input" },
# Mode
{ on = [ "i" ], exec = "insert", desc = "Enter insert mode" },
{ on = [ "a" ], exec = "insert --append", desc = "Enter append mode" },
{ on = [ "v" ], exec = "visual", desc = "Enter visual mode" },
{ on = [ "V" ], exec = [ "move -999", "visual", "move 999" ], desc = "Enter visual mode and select all" },
# Navigation
{ on = [ "h" ], exec = "move -1", desc = "Move cursor left" },
{ on = [ "l" ], exec = "move 1", desc = "Move cursor right" },
{ on = [ "0" ], exec = "move -999", desc = "Move to the BOL" },
{ on = [ "$" ], exec = "move 999", desc = "Move to the EOL" },
{ on = [ "I" ], exec = [ "move -999", "insert" ], desc = "Move to the BOL, and enter insert mode" },
{ on = [ "A" ], exec = [ "move 999", "insert --append" ], desc = "Move to the EOL, and enter append mode" },
{ on = [ "<Left>" ], exec = "move -1", desc = "Move cursor left" },
{ on = [ "<Right>" ], exec = "move 1", desc = "Move cursor right" },
{ on = [ "b" ], exec = "backward", desc = "Move to the beginning of the previous word" },
{ on = [ "w" ], exec = "forward", desc = "Move to the beginning of the next word" },
{ on = [ "e" ], exec = "forward --end-of-word", desc = "Move to the end of the next word" },
# Deletion
{ on = [ "d" ], exec = "delete --cut", desc = "Cut the selected characters" },
{ on = [ "D" ], exec = [ "delete --cut", "move 999" ], desc = "Cut until the EOL" },
{ on = [ "c" ], exec = "delete --cut --insert", desc = "Cut the selected characters, and enter insert mode" },
{ on = [ "C" ], exec = [ "delete --cut --insert", "move 999" ], desc = "Cut until the EOL, and enter insert mode" },
{ on = [ "x" ], exec = [ "delete --cut", "move 1 --in-operating" ], desc = "Cut the current character" },
# Yank/Paste
{ on = [ "y" ], exec = "yank", desc = "Copy the selected characters" },
{ on = [ "p" ], exec = "paste", desc = "Paste the copied characters after the cursor" },
{ on = [ "P" ], exec = "paste --before", desc = "Paste the copied characters before the cursor" },
# Undo/Redo
{ on = [ "u" ], exec = "undo", desc = "Undo the last operation" },
{ on = [ "<C-r>" ], exec = "redo", desc = "Redo the last operation" },
# Help
{ on = [ "~" ], exec = "help", desc = "Open help" }
]

These are the Vim keybindings currently supported by Yazi. Yes, it's open-ended, and over time, Yazi will integrate more Vim functionalities, but certainly not all of them, as that would be creating a new text editor.

Internal one or also by the Kernel scheduler? Use "internal task priority assignment", if you dont't use sched_* class functions ie on Linux etc.

Are they using blocking syscalls or non-blocking ones (io_uring)? Would sounds nicer, if you mention asynchronous executed and using non-blocking syscalls, if available.

Yazi doesn't use sched_* and io_uring, and the reason is here: https://www.reddit.com/r/rust/comments/16fxr58/comment/k066gmh/

"Internal task priority assignment" sounds good, I will update it.

How much additionally memory does this use? Is this configurable? Overall memory consumption would be also nice to know and track to estimate what kind of low(er) end devices can be used with yazi.

If you are referring to transient memory usage, it depends on the user's file type and file size. For image files, it only consumes the memory necessary for image decoding and downscaling, then writes the result to disk and promptly releases the memory.

If you are talking about resident memory, for images, it is not applicable because they are reading cache from disk. For regular files, the MIME type of the file is pre-cached, and a BTreeMap<Url, String> is used to store the mapping between files and their MIME types.

The amount of memory consumed in this case also depends on the number of files and user actions -- pre-caching is lazily triggered only when the user scrolls over pages.

Given the implementation mentioned above, I don't believe there's a significant need to make it configurable. Therefore, it's non-configurable.

@sxyazi
Copy link
Owner

sxyazi commented Sep 12, 2023

Hi, I wrote a new article to explain the inner workings of Yazi: #143

And I have changed the README in c3bc872

@matu3ba
Copy link
Author

matu3ba commented Sep 12, 2023

Wow, thanks alot. That clarifies alot and sounds very convincing, so I'll close this issue.

I'll add my only nits regarding relevant rough upper bound memory usage to the (first design draft?) article.

@matu3ba matu3ba closed this as completed Sep 12, 2023
@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days. ⏳ This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 27, 2023
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants