Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP/RFC] allow polling on a provided fd #72

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bfredl
Copy link
Member

@bfredl bfredl commented Dec 21, 2014

This allows a plugin to poll for readability and/or writability on a specified fd.
here is an example integrating with a zeromq socket.

However, I'm not actually sure we want to spawn a greenlet for every poll event, because this event doesn't imply that there actually is useful data that needs to be handled. ( For instance, zmq uses the same fd also for internal communication with its worker threads and so at times there could be lots of spurious poll events) Perhaps its better to spawn a greenlet only when there is message, for instance:

     def _on_poll(self, fd, read, write):
        while self.sock.getsockopt(zmq.EVENTS) & zmq.POLLIN:
            self.vim.session.schedule(lambda: self.on_msg( self.sock.recv()))

One might even want back to yield back to an existing greenlet that is blocking on receiving (or sending) on the fd.
Or it could be that spawning new greenlets is so cheap, that its not worth bothering avoiding it, I don't know.

@tarruda
Copy link
Member

tarruda commented Dec 22, 2014

However, I'm not actually sure we want to spawn a greenlet for every poll event, because this event doesn't imply that there actually is useful data that needs to be handled

I've been thinking that it would be best to add automatic greenlet creation as a session filter that is disabled by default and enabled with something like: greenlet_vim = vim.with_filter(GreenletWrap). Plugins running in the host would enable greenlets with per-method decorators

@bfredl
Copy link
Member Author

bfredl commented Dec 22, 2014

A constraint to consider might be making it convenient for plugins to use greenlets to handle messaging to other sources. Right now in my ipython plugin I do this to wait for a reply asynchrously:

def waitfor(self, msg_id):
    gr = greenlet.getcurrent()
    self.pending_shell_msgs[msg_id] = gr
    return gr.parent.switch()

and then in an event callback:

def on_shell_msg(self, m):
    msg_id = m['parent_header']['msg_id']
    try:
        handler = self.pending_shell_msgs.pop(msg_id)
    except KeyError:
        debug('unexpected shell msg: %r', m)
        return
    handler.parent = greenlet.getcurrent()
    handler.switch(m)

Which "just happens" to work given how the python client library uses greenlets. Perhaps it would be a cleaner abstraction if there was a session.schedule_greenlet method (i e switch to this greenlet as soon as the event loop is idle)

@tarruda
Copy link
Member

tarruda commented Dec 22, 2014

Which "just happens" to work given how the python client library uses greenlets. Perhaps it would be a cleaner abstraction if there was a session.schedule_greenlet method (i e switch to this greenlet as soon as the event loop is idle)

greenlet wrapping can be implemented as decorators, for example:

@neovim.plugin
class Plugin(object):
    def __init__(self, nvim):
        self.nvim = nvim
        self.greenlet_nvim = nvim.with_hook(GreenletWrap)

    @neovim.greenlet
    def handler1(self):
        # runs in a greenlet
        print(self.greenlet_nvim.eval('1'))

    def handler2(self):
        # doesnt run in a greenlet
        def cb(value):
            print value
         self.nvim.eval('1', cb)

I dont think the greenlet hook is even required, session.request can simply detect if it is being called inside a greenlet and act accordingly(eg: receive a callback as last argument if not inside a greenlet)

@bfredl
Copy link
Member Author

bfredl commented Dec 22, 2014

Yeah, I think it would be cleaner API if one doesn't have to use two different nvim objects. A possible interface could be:

nvim.eval('1')  ==> yield from this greenlet (if running inside one)
nvim.eval('1', cb) ==> use callback
nvim.eval('1', None) ==> ignore reply

@bfredl
Copy link
Member Author

bfredl commented May 14, 2015

Updated, now it has simpler vim.session.poll_fd(fd, on_readable=callback) api, and a greenlet is not automatically spawned.

@bfredl
Copy link
Member Author

bfredl commented Aug 28, 2016

Updated. Goes back to spawn a greenlet per default, to behave like a an ordinary (async) neovim command handler or async_call callback (i e api calls can be done in the poll callback, principle of least surprise).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants