Issues with the hygienic macro system

[jpellegrini]

Hi @egallesio !
I'm opening this issue so we can track the problems with the hygienic macro system implementation.

This is what I can see...

Macros should remember their environment

(define-syntax f
  (syntax-rules ()
    ((f) a)))

(f)

The above should trigger an error, because the macro did not remember its environment. The "a" that it was referencing was "a in current module", and the local a included by the let should not be visible (this is a new local environment that did not exist a the time the macro was created).

Interestingly, Scheme9, Gambit, Cyclone and Bigloo also behave like that (but it's not according to the standard).

(define-syntax f
  (syntax-rules ()
    ((f a b) (- a b))))

(let ((f +)) (f 2 3))  ;; => should be 5;  *** STklos returns -1

Now Gambit does return 5... Cyclone and Bigloo return -1 (but the correct is 5).

(let ((x 'outer))
  (let-syntax ((m (syntax-rules ()
                    ((m a) (list x a)))))
    (let ((x 'inner))
      (cons x (m 17)))))

;; Should be (inner outer 17)
;; STklos result is (inner inner 17)

I think this problem is related to the eval issue (#410), since we'd have to eval the macro expander in the module where the macro was created.

STklos does not accept internal define-syntax

This is easy to fix, I suppose, since let-syntax already works.

Hygiene

This can be dealt with later, but I think it would be possible to either implement ER macros (there are issues with it, but I think we can circumvent them), or syntax-case (but I think a lexically-scoped define-macro along with gensym, syntax-rules and hygiene is as powerful as syntax-case, and perhaps easier -- then syntax-case can be implemented on top of this?)

The syntax-rules matcher

It's quite old (from Macros by Example), and there are some glitches. I can re-write it if it's ok.

In particular, the current implementation doesn't handle improper lists as patterns correctly (it complains about the length not being calculable). I think this is not hard to re-implement in a better way.

Aliases, for SRFI 212

That SRFI specifies an alias macro that will work for any symbol - even those bound to macros.

STklos already has %symbol-alias, which does the right thing for global symbols (including macros!)

stklos> (define-macro (f . whatever) -1)
;; f
stklos> (f 1)
-1
stklos> (%symbol-alias 'g 'f)
stklos> (g 2)
-1

But it would be nice to be able to alias local symbols. For that, the scope structure in the compiler would have to be slightly adapted (I can do that):

Keep variable scopes and macro scopes separated as they are currently, but also create a third type of scope for aliases;

Both locals and mlocals in structure scope would be association lists:

• ALIASES: the CAR is the symbol, and the CDR points to the location

  (a . LOCATION)

  where location is either some other entry for a variable in this scope (down the scope stack), or a binding to a specific symbol in a specific module. This works for both macros and variables.
• MACROS: the CDR is the syntax object:

  (s . [#syntax ...])

• LOCAL VARS: no need for a CDR

  (v)

This would give us a way to implement that SRFI, and would repect lexical scope for both variables, macros and aliases. :)

This is not difficult to do, I can help (I can make a PR soon if it's ok).

Tests for all the above

I have some tests... I'll organize them and make a draft PR for reference.

[jpellegrini]

@egallesio is it OK if I try to implement SRFI 212 (aliases)? (I won't if you're working on the same ting, or in parts of the compiler that would conflict with it)

[egallesio]

I have already implemented this SRFI (and not committed it yet, since the tests used in the SRFI use free-identifier? and bound-identifier? ....). Anyway, a simple implementation could be:

(define-module srfi/212
  (import (only SCHEME %symbol-alias))
  (export alias)

  (define-macro (alias var1 var2)
    `(%symbol-alias ',var1 ',var2)))

(provide "srfi/212")

The first example of the SRFI document:

stklos> (define *important-global-variable* '())
    (define (setup!)
      (alias ls *important-global-variable*)
      (set! ls (cons 1 ls))
      (set! ls (cons 2 ls)))
    (setup!)
;; *important-global-variable*
;; setup!
stklos> *important-global-variable* 
(2 1)

[jpellegrini]

I have already implemented this SRFI (and not committed it yet,

That's great! :)
Did you make it work for local symbols and macros also?

[jpellegrini]

Hi @egallesio !
I think the implementation, as you showed, would only work for globals. The following would be necessary for it to actually work, no? (And it may also be useful to implement hygiene for local variables later!)

(let ((a 10))
  (alias b a)
  (set! a 20)
  b) ;; => 20
b ;; => error (not bound)

(define a 10)
(let ((b 20))
  (alias c a)
  (display 'hello)
  (display c)) ;; => hello10
c ;; => error (not bound)

(define a 10)
(let ((b 20))
  (alias a b)
  (display 'hello)
  (display a)) ;; => hello20
a ;; => 10

(define a 10)
(define d 30)
(let ((b 20))
  (alias a d)
  (display 'hello)
  (display a)) ;; => hello30
a ;; => 10

(define-syntax f
  (syntax-rules ()
    ((f) 10)))
(let ((g (lambda () 2)))
  (alias g f)
  (f)) ;; => 10

(let ((a 10)
      (b 20))
  (let-syntax ((f (syntax-rules ()
                    ((f) b))))
    (alias g f)
    (g))) ;; => 20

And the following one, which would need the work on macros remembering their environment;

(let ((a 10)
      (b 20))
  (let-syntax ((f (syntax-rules ()
                    ((f) b))))
    (let ((b 30))
      (alias g f)
      (g)))) ;; => 20 (the 'b' from f's definition env)