Content from other tmux-clients is missing

[balta2ar]

@roxma Hi! It looks like currently the plugin lists only panes in the current session (client). I usually have multiple sessions with multiple windows (and usually one pane per window), so I'm missing out on a lot of tmux content from other sessions. tmux-complete (for deoplete) works fine, though: https://github.com/wellle/tmux-complete.vim/blob/master/sh/tmuxcomplete#L57
I noticed that it has integrations with many plugins. Maybe it's better to contribute ncm2 support to tmux complete?

deoplete Source there looks dead simple: https://github.com/wellle/tmux-complete.vim/blob/master/rplugin/python3/deoplete/sources/tmuxcomplete.py

[roxma]

Unfortunately, The deoplete integration (vim.call) may block the editor. I don't like this implementation.

[roxma]

In most cases I have a single tmux session, that's why this source doesn't grab contents from other sessions.

I'll try add other sessions when I have some free time. Though, this is kind of low priority.

[roxma]

It's kind of interesting to add on_completed support based on ncm2/ncm2#28

[balta2ar]

The deoplete integration (vim.call) may block the editor

That's just one way to implement it, you don't have to do the same.

It's kind of interesting to add on_completed support

Sorry, I'm not quite following... Will this allow writing async sources? If not, is there a way to do it now? That's treading on a thin ice, though. deoplete had (and still has, as far as I understand) multiple buts related to async sources and multiprocessing though...

[roxma]

That's just one way to implement it, you don't have to do the same.

The tmux-complete.vim uses vimscript to launch tmux sub-process, and wait for the completion results. Blocking wait in vimscript means blocking wait of the editor. It's impossible to be async without change the main code of tmux-complete.vim. Which will require more time to debug and test.

Sorry, I'm not quite following... Will this allow writing async sources?

I mean another feature. There's gif in the PR.

To explain with more detail:

1. The user gets a popup list of completions, say, one of them is hello from a sentence hello world. This is my first ...
2. the user select hello
3. the user press <c-y> to accept the completion and close the popup
4. ncm2 gets CompleteDone, forward the event to the ncm2-bufword source
5. ncm2-bufword handle the event, send the next word of the sentence, which is world in hello world. This is my first ..., back to ncm2 for completion
6. If the user select and accept the completion too, we'll go to step 4 and get more and more results.

I think this feature should be useful, since the next popup is more context-aware.

[balta2ar]

The tmux-complete.vim uses vimscript to launch tmux sub-process, and wait for the completion results. Blocking wait in vimscript means blocking wait of the editor. It's impossible to be async without change the main code of tmux-complete.vim. Which will require more time to debug and test

When you say "uses vimscript" you mean only the following single line, right (https://github.com/wellle/tmux-complete.vim/blob/master/rplugin/python3/deoplete/sources/tmuxcomplete.py#L16)?

command = self.vim.call('tmuxcomplete#getcommand', '', 'words')

So because we have this call to vim, it may get stuck there and may block the editor (please confirm my understanding here). But in practice this call is only to grab the option value, and should be blazing fast. It extracts the command to run, and I actually see no reason to call it every time in gather_candidates -- it should have been called once during the initialization and then reused. However, the blocking line is the next one (https://github.com/wellle/tmux-complete.vim/blob/master/rplugin/python3/deoplete/sources/tmuxcomplete.py#L17):

words = check_output(['sh', '-c', command]).decode('utf-8').splitlines()

But it's not resorting to vim, so there is no vimscript, right? In fact, it depends on how the Source caller is implemented. If it's the same process with vim, then it's a problem. However, for deoplete it's not, because it creates each source in a separate process (Shougo/deoplete.nvim#602), thus source can block as long as it wants without affecting the editor and deoplete core.

And even before that transition, there was async source model (https://github.com/Shougo/deoplete.nvim/blob/master/doc/deoplete.txt#L1071-L1082) when source is repeatedly called (say, every 50ms) until certain flag is not set. In this model it was easy to offload computations into a background thread, e.g.: https://github.com/balta2ar/webcomplete.vim/blob/master/rplugin/python3/deoplete/sources/webcomplete.py#L27

Either of this can be applied in tmux-complete given proper support from completion framework side. Thus I'm asking what's the source model in ncm2? Can it do any of the following?

1. either run each source in a different process
2. or support non-blocking async model, when source is repeatedly polled for new matches

[roxma]

But it's not resorting to vim, so there is no vimscript, right?

You're right. I misunderstood the source code. I thought it was running command in vimscript.

Either of this can be applied in tmux-complete given proper support from completion framework side. Thus I'm asking what's the source model in ncm2? Can it do any of the following?

Unlike the old ncm, ncm2 no longer manage processes anymore. There's no strict source model in ncm2. All it does is to notify the sources (on_complete) for completion when it's appropriate. The source sends notification back to ncm2 when the completions is available (ncm2#complete).

Currently most sources is based on nvim-yarp. In this way, each source is a standalone process.

Techniquely it is possible to write a nvim-yarp wrapper to easily group sources into a process using multi-threading, which is already proven in the old ncm. But there's no need for that at current state, since the only benifit is to save memory, and generally speeking it's not a good thing to suffer from GIL.

[balta2ar]

Unlike the old ncm, ncm2 no longer manage processes anymore. There's no strict source model in ncm2. All it does is to notify the sources (on_complete) for completion when it's appropriate. The source sends notification back to ncm2 when the completions is available (ncm2#complete).

Thanks, now it's clear to me.

Currently most sources is based on nvim-yarp. In this way, each source is a standalone process

I think this detail is very important to source authors and it affects heavily how a source is implemented. It basically means that on_complete handler is synchronous, but it's isolated from 1) the editor 2) ncm2 core 3) other sources. So if you have sources A, B, and C, and it's taking 3 seconds for source A to produce matches, sources B and C won't be affected and their results will appear in completion menu as soon as they are available. If this understanding is right, it's very worth putting this clarification into the documentation (and maybe comparing to other completion frameworks, such as deoplete, so that when they see a description, they could immediately understand the model of operations: "Aha, it's the same with deoplete, now I get it").