Loading SRFI 169 will automatically turn it on

[jpellegrini]

Hi @egallesio

A on-liner patch...

Currently,

stklos> (accept-srfi-169-numbers #f)
stklos> 1_2
**** Error:
%execute: symbol `1_2' unbound in module `stklos'

stklos> (import (srfi 169))  ;; This should turn SRFI 196 on...

stklos> 1_2                  ;; But it didn't!
**** Error:
%execute: symbol `1_2' unbound in module `stklos'

This adds a line to the SRFI implementation file, that just changes the parameter to #t.

[lassik]

AFAIK it is unconventional for (import ...) to change the lexical syntax.

Usually it's done with directives like #!r6rs or #!fold-case. Unfortunately, I think we didn't standardize a directive for SRFI 169. #!srfi-169 would be a natural one.

Gauche supports SRFI 169, but I think the underscore syntax is always on.

[jpellegrini]

AFAIK it is unconventional for (import ...) to change the lexical syntax.

Yes... But it seems to make sense that "if it isn't working and I load the SRFI, it should immediately start working".

Anyway, I have no strong preference here. I just realized it could make sense (but then, I'm not really sure it does make sense :)

[lassik]

It makes more sense from the REPL. But how is a source file like this interpreted:

(define-library (srfi 3_4_5)
  (import (scheme write) (srfi 1_6_9))
  (begin (write 4_5_6)))

What about this:

(define-library (example)
  (import (srfi 169) (srfi 1_2_3))
  (import (srfi 2_3_4))
  ...)

[jpellegrini]

Hi @lassik .
I'm not sure what the problem would be with the first example.

Your second example is interesting. (import lib-A lib-B) loads libraries libA and libB, but I guess nobody thought of loading libraries as having side effects. What if libA has side-effects that affect the loading of libB? It could even be "changing libB.so to a different file in the filesystem".

I suppose you're right, and we'd better not letting that happen...

[lassik]

It's even worse than I considered...

(define-library ...) is a top-level datum, so the complete datum is read before any part of the library definition is interpreted.

This underscores (no pun intended) that #! directives are the consistent way to solve lexical syntax problems, and import causes several problematic situations.

[lassik]

On side effects generally:

• (begin ...) blocks in libraries can have side effects. But they may not be executed every time the library is imported.

• The effect of (import A B) can depend on whether A has already been loaded or not.

R7RS section 5.6.1 Library Syntax specifies this.

[lassik]

Concretely, #! directives would work like this:

#!srfi-169
(define-library (lib 2_3_4)
  (import (scheme write) (lib 3_4_5))
  (begin (write 4_5_6)))

Or this:

(define-library (lib 234)
  #!srfi-169
  (import (scheme write) (lib 3_4_5))
  (begin (write 4_5_6)))

Even this would work:

(define-library (lib 234)
  (import (scheme write) #!srfi-169 (lib 3_4_5))
  (begin (write 4_5_6)))

Alll of the above examples work in a consistent way because #! is handled already at read time.

[lassik]

In STklos, that would be implemented by keeping a table of all known #! keywords in the main executable. And some of those keywords would cause an .so library to be loaded.

The numeric vector SRFIs could be handled using this approach, too. so #!srfi-160 would load the same module as (import (srfi 160)). But:

• Only #!srfi-160 would change the current lexical syntax.
• Only (import (srfi 160)) would import identifiers into the current environment.

[jpellegrini]

a table of all known #! keywords in the main executable

Actually, that sounds like a very nice idea! And if we can get this to be configurable at runtime (a dynamic table), we could even do things like

(define-shebang disable-peephole
  (compiler:peephole-optimizer #f))

(define-shebang enable-peephole
  (compiler:peephole-optimizer #t))

Then

(define (a-function x)
  ...
  ...
  #!disable-peephole
  ... ;; this code will not be optimized
  ...
  #!enable-peephole)

See, it would not work to change the parameter inside the function, because it won't change the compiler behavior when the function is being compiled. This could perhaps help in debugging and benchmarking. @egallesio what do you think?

[lassik]

a table of all known #! keywords in the main executable

Actually, that sounds like a very nice idea! And if we can get this to be configurable at runtime (a dynamic table), we could even do things like

(define-shebang ...)

That's a good idea.

I recommend Common Lisp's readtables as a model. A readtable is a special type of object that stores settings for the reader. In CL, the special variable *readtable* controls the current readtable. (Special variables are similar to a parameter objects in Scheme.) But since Scheme does not have the baggage of backward compatibiilty, it would probably be cleaner if the current readtable is a property of each port object.

Then

(define (a-function x)
  ...
  ...
  #!disable-peephole
  ... ;; this code will not be optimized
  ...
  #!enable-peephole)

See, it would not work to change the parameter inside the function, because it won't change the compiler behavior when the function is being compiled. This could perhaps help in debugging and benchmarking.

This will not have the intended effect. It's best to have a clean separation between read time (what affects lexical syntax) and expansion/compilation/evaluation time.

I recommend this instead:

(define (a-function x)
  ...
  ...
  (with-declare ((optimize (peephole #f)))
    ... ;; this code will not be optimized
    ...))

This is patterned after CL's (declare ...) form, which I hope will be copied into RnRS at some point.

[lassik]

The CL standard lets you write things like this:

(defun fast ()
  (declare (optimize (speed 3) (safety 0)))
  (let ((product 1))
    (dotimes (i 10000 product)
      (setf product (* (the fixnum product) (1+ i))))))

Compare the result to the non-broken version:

(defun slow ()
  (let ((product 1))
    (dotimes (i 10000 product)
      (setf product (* product (1+ i))))))

[jpellegrini]

This will not have the intended effect

Yes, obviously, and I don't know what I was thinking! 😁 You're right! And I like your suggestion!

[jpellegrini]

The CL standard lets you write things like this:

Yes, I have programmed in Common Lisp ~20 years ago. It's a really fun language to use!

[egallesio]

Very interesting discussion

@lassik: STklos has already two forms when-compile and when-load-and-compile (not very fan of the names, especially the latter one). So, it is already possible
to have something like

(define (a-function x)
  ...
  ...
  (when-compile (compiler:peephole-optimizer #f))
  ... ;; this code will not be optimized
  ...
  (when-compile (compiler:peephole-optimizer #t)))

@jpellegrini: I have hacked a form similar to your define-shebang
This form must be evaluated in the compiler too to permit the introduction of new #!keywords in the code.
It almost works, but I need also to change the metadata of the .ostk files since new keywords must be also redefined in the compiler when the file which defines them is imported. In some extent, the new keywords should be always "exported".
It needs still some work, before I can release it.

About SRFI-169, the problem is that the default is to accept numbers with underscores, and we need a way to forbid _ in numbers. So we probably need to define two keywords #!srfi-169 and #!no-srfi-169 (if you have a better name ...). The problem, is that we'll need to import (srfi 169) to disable SRFI-169 numbers 👎. A more logical solution would be to add both keywords in core STklos and let lib/srfi/169.stk as is.

About, changing the syntax when importing a file, we also have this problem, at least, with SRFFI-4 and SRFI-178. However, in these SRFIs, we extend the syntax, we do not modify it (as in SRFI-169 where symbols become numbers).