REPL appears not to start

[propensive]

A command invocation such as bloop console kaleidoscope/core appears to work right up until it starts the REPL. The last lines I see are (including a blank one),

Starting scala interpreter...

The invocation does not appear to terminate, and the cursor is left "hanging".

There is no output at all on the server until I hit Ctrl+C, then:

Client in <path> has disconnected with a 'SocketError' event. Cancelling tasks...

This is the beaviour for the first five invocations of bloop console. On the sixth attempt, no output is shown on the frontend at all.

Bloop-frontend: 1.0.0-M7
Zinc: 1.1.0+9-57ca9e0f
Scala: 2.12.4

[propensive]

Update: running bloop console on a freshly-started bloop server, it gets a bit further: It reports the intro message from the REPL (Welcome to Scala 2.12.4...), and sends an ANSI code to turn my text pink, but it doesn't work interactively. This appears to only be the case if it's the first thing bloop does.

I wonder if the JLine interactivity in the REPL is failing due to some sort of buffering going on. I have vague memories of having to send some sort of stty -icanon command in my terminal on some project I worked on in the past, though I can't remember the details. But it's something to Google for, maybe.

[propensive]

Still not working with M8, including the behavior described in my "update" comment.

[tues]

Seems like there are three issues at play here:

1. The REPL starts correctly only if it's the first command after starting bloop server
2. Bloop doesn't send Nailgun's Start-reading-input chunk to the client so stdin doesn't work
3. Nailgun buffers IO per line, so the REPL's scala> prompt isn't displayed until sending SIGINT to the client (e.g. by pressing Ctrl+C). Related to The python nailgun client doesn't seem to like system processes facebookarchive/nailgun#115 @jvican?

I've captured some TCP traffic between the client and the server and decoded it:

[A   10] jsonConfig
[E   23] NAILGUN_FILESEPARATOR=/
[E   23] NAILGUN_PATHSEPARATOR=:
[E   15] NAILGUN_TTY_0=1
[E   15] NAILGUN_TTY_1=1
[E   15] NAILGUN_TTY_2=1
[E   26] TERM=screen.xterm-256color
[E 1357] TERMCAP=SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:\
        :DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:UP=\E[%dA:bs:bt=\E[Z:\
        :cd=\E[J:ce=\E[K:cl=\E[H\E[J:cm=\E[%i%d;%dH:ct=\E[3g:\
        :do=^J:nd=\E[C:pt:rc=\E8:rs=\Ec:sc=\E7:st=\EH:up=\EM:\
        :le=^H:bl=^G:cr=^M:it#8:ho=\E[H:nw=\EE:ta=^I:is=\E)0:\
        :li#61:co#257:am:xn:xv:LP:sr=\EM:al=\E[L:AL=\E[%dL:\
        :cs=\E[%i%d;%dr:dl=\E[M:DL=\E[%dM:dc=\E[P:DC=\E[%dP:\
        :im=\E[4h:ei=\E[4l:mi:IC=\E[%d@:ks=\E[?1h\E=:\
        :ke=\E[?1l\E>:vi=\E[?25l:ve=\E[34h\E[?25h:vs=\E[34l:\
        :ti=\E[?1049h:te=\E[?1049l:us=\E[4m:ue=\E[24m:so=\E[3m:\
        :se=\E[23m:mb=\E[5m:md=\E[1m:mh=\E[2m:mr=\E[7m:\
        :me=\E[m:ms:\
        :Co#8:pa#64:AF=\E[3%dm:AB=\E[4%dm:op=\E[39;49m:AX:\
        :vb=\Eg:G0:as=\E(0:ae=\E(B:\
        :ac=\140\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:\
        :po=\E[5i:pf=\E[4i:Km=\E[M:k0=\E[10~:k1=\EOP:k2=\EOQ:\
        :k3=\EOR:k4=\EOS:k5=\E[15~:k6=\E[17~:k7=\E[18~:\
        :k8=\E[19~:k9=\E[20~:k;=\E[21~:F1=\E[23~:F2=\E[24~:\
        :F3=\E[1;2P:F4=\E[1;2Q:F5=\E[1;2R:F6=\E[1;2S:\
        :F7=\E[15;2~:F8=\E[17;2~:F9=\E[18;2~:FA=\E[19;2~:\
        :FB=\E[20;2~:FC=\E[21;2~:FD=\E[23;2~:FE=\E[24;2~:kb=:\
        :K2=\EOE:kB=\E[Z:kF=\E[1;2B:kR=\E[1;2A:*4=\E[3;2~:\
        :*7=\E[1;2F:#2=\E[1;2H:#3=\E[2;2~:#4=\E[1;2D:%c=\E[6;2~:\
        :%e=\E[5;2~:%i=\E[1;2C:kh=\E[1~:@1=\E[1~:kH=\E[4~:\
        :@7=\E[4~:kN=\E[6~:kP=\E[5~:kI=\E[2~:kD=\E[3~:ku=\EOA:\
        :kd=\EOB:kr=\EOC:kl=\EOD:km:
[D   20] /home/tues/dev/bloop
[C    7] console
[H    0]
[1   29] Starting scala interpreter...
[1    1]

[1    1]

[H    0]
[H    0]
[H    0]

First character after [ is the chunk type, next is chunk length, then follows raw chunk data after ] . I've omitted irrelevant E lines. Chunks of type 1 are the only ones sent by the server, the rest is sent by the client.

As you can see, there is no S (Start-reading-input) chunk sent by the server.

Reference (probably outdated): http://www.martiansoftware.com/nailgun/protocol.html

[propensive]

Thanks for the detailed analysis, @tues! Hopefully gets us a step closer to a fix. :)

[tues]

I've tested this with the newest master and it seems that the first issue has been fixed:

The REPL starts correctly only if it's the first command after starting bloop server

The other two issues are still present:

[C    7] console
[H    0]
[1   29] Starting scala interpreter...
[1    1]

[1    1]

[1  118] Welcome to Scala 2.12.6 (OpenJDK 64-Bit Server VM, Java 1.8.0_162).
Type in expressions for evaluation. Or try :help.

[1   17]
scala>
[H    0]

I was typing :help\n but nailgun client didn't even send it to the server.

I've been experimenting with Tasks.console a bit:

    val foo = scala.io.StdIn.readLine()
    scala.io.StdIn
    println(s"foo: $foo")

    state.logger.warn(s"*** stdin: ${System.in}")
    state.logger.warn(s"*** stdout: ${System.out}")

    compiler.console(classpathFiles, project.scalacOptions, classpathOptions, "", "", state.logger)(
      Some(loader))

and readLine seems to be working perfectly fine:

[1    4] foo?
[1    1]

[S    0]
[H    0]
[0    4] bar

[1    8] foo: bar
[1    1]

[1   87] [W] *** stdin: com.martiansoftware.nailgun.ThreadLocalInputStream@12256b78
[1    1]

[1   88] [W] *** stdout: com.martiansoftware.nailgun.ThreadLocalPrintStream@6b7a0a22
[1    1]

[1   29] Starting scala interpreter...
[1    1]

so it seems that sbt.internal.inc.AnalyzingCompiler doesn't inherit correct stdin for some reason.

It's worth noting that stdin is working for the run command which is using the new Forker class (#449). I'm still learning Bloop's codebase, but my wild guess is that maybe we should implement a ConsoleServer similar to the TestServer being used currently and start forking new processes for the REPL?

[tues]

Turned out to be much easier to fix. I was fooled by xsbt.ConsoleInterface overriding ILoop.createInterpreter and forcefully setting its own input reader, but somehow passing -Xnojline (which should affect only ILoop's preLoop reader) worked.

[C    7] console
[H    0]
[1   29] Starting scala interpreter...
[1    1]

[1    1]

[1  118] Welcome to Scala 2.12.6 (OpenJDK 64-Bit Server VM, Java 1.8.0_162).
Type in expressions for evaluation. Or try :help.

[1   17]
scala>
[S    0]
[0   13] val foo = 13

[1   39] foo: Int = 13
[1    1]

[1   17]
scala>
[S    0]
[0    6] :quit

[X    1] 0
[X    1]

[X    1] 0
[X    1]

[folex]

The REPL starts correctly only if it's the first command after starting bloop server

I'm experiencing this on M11, is that expected?
If I restart bloop via brew services stop/start, then bloop console projectName works fine.

    Bloop-frontend version    `1.0.0-M11`
    Zinc version     `1.1.7+62-0f4ad9d5`
    Scala version    `2.12.6`

macOS 10.13.5, installed via brew

[tues]

Hi @folex 🙂 No, it's not expected, it's a bug: please see #497. Unfortunately I haven't managed to find the cause yet 🙁