Optimize optional and keyword arguments (calls to procedures using these become twice as fast)

[jpellegrini]

Hi @egallesio !

What

While looking at the code generated for optional and keyword arguments, I thought that perhaps they could be optimzied. I did a small experiment, and it's close to actually becoming ready. These are the commits I have made, in order:

1. Make closures remember their optional and argument list. It doesn't work automatically; only creates the fields in the closure strutcure.
2. Create a %%set-procedure-optionals-and-keys! so we can test these procedures that know their optionals and keyword arguments. This is a temporary hack only to build the procedures with the new optionals and keyword fields.
3. Create %%set-procedure-arity!, to also change the arity of procedures. This is a temporary hack only to build the procedures with the correct arity.
4. Change the VM. adjust_arity becomes adjust_stack_get_arity, because it really reorganizes the stack, putting the arguments in proper order. For the usually-created lambdas, no changes. But if the closure being called has a key or opt field different from STk_nil (and that only happens if we explicitly use the procedures in steps 2 and 3), then those fields are used. I have tried to thoroughly document this in comments in the VM code.
5. Change %procedure-arity so that it returns three values: arity as usual; number of optional arguments; number of keyword arguments (this isn't strictly necessary, but I thought it'd be nice.

Why?

The benefits are speed (it's almost twice as fast to use the new in-VM version of opts-and-keys parsing); and also convenience :) because then we'd be able to ask %procedure-optionals...

How do I build a procedure that uses the new parsing?

The last step would be to change the compiler, but that seems harder than what I thought, so I decided to show what I've done so far.

;; Define a procedure. The arguments are:
;; 1. the mandatory ones
;; 2. the optional ones
;; 3. the tests for the optionals
;; 4. the keyword ones
;; 5. the tests for keywords
;; 6. the rest list
(define (g a b ;; mandatory
           c d ;; optionals
           c?  ;; tests for optionals
           e f ;; keyword arguments
           e?  ;; tests for keywords
           r)  ;; rest list
  (list a b c c? d e e? f r))

;; Now we set the arity to `-3` (because we want the two first args, a and b,
;; to be mandatory)
(%%set-procedure-arity! g -3)
;; And we also set the optionals and keywords list of the procedure
(%%set-procedure-optionals-and-keys! g '( (c 10 c?) (d 20) ) '( (:e -1 e?) (:f -2) ) )

;; And we can call the procedure!
(g 'x 'y)
 => (x y 10 #f 20 -1 #f -2 ())
(g 'x 'y -10)
 => (x y -10 #t 20 -1 #f -2 (-10))
(g 'x 'y -10 -20)
 => (x y -10 #t -20 -1 #f -2 (-10 -20))
(g 'x 'y -10 -20 -30)
 => error: bad list of keywords `(-30)`
(g 'x 'y :f 200)
 => (x y 10 #f 20 -1 #f 200 (#:f 200))

and so on.

Rudimentary benchmarks

I have tried some very rudimentary timings:

;; A usual procedure definition with two 
;; optional and two keyword arguments
(let ((x 0)
      (y 0))

  (define (G a b :optional (c 10 c?) (d 20)
                 :key      (e -1 e?) (f -2))
    (set! x (+ x c f))
    (set! y (+ y e d)))

  (time (repeat 10_000_000
                (G 0 0 2 :e 1 :f -2)))
  (list x y))
Elapsed time: 16716.973 ms                <== Almost 17 seconds
(0 210000000)


;; Now, instead of making a usual STklos definition,
;; we programatically build a procedure that will work 
;; as the previous one, but using the new method.
(let ((x 0)
      (y 0))

  (define (g a b   c d c?   e f e?  r)
    (set! x (+ x c f))
    (set! y (+ y e d)))

  (%%set-procedure-arity! g -3)
  (%%set-procedure-optionals-and-keys! g '( (c 10 c?) (d 20) ) '( (:e -1 e?) (:f -2) ) )

  (time (repeat 10_000_000
                (g 0 0 2 :e 1 :f -2)))
  (list x y))
Elapsed time: 9332.507 ms                <== A bit more than 9 seconds
(0 210000000)

Do we keep trying? :)

Do you think it would be worth changing the compiler in order to take advantage of this? I came up with a new version of rewrite-params-and-body but it didn't work properly.

[jpellegrini]

PS: a simple grep shows that more than 60 lines of code in STklos use optional and keyword arguments, and I suppose that for some of them would the speedup would be really nice. For example (selected output from grep),

(define (delete  x l :optional (comp equal?))
(define (delete! x l :optional (comp equal?))
(define (hash-table-update! hash key func :optional (thunk #f thunk?))
(define (hash-table-get ht key :optional (default #f default?))
(define (member x y :optional (compar %equiv?)) (member-simple x y compar))
(define (assoc  x y :optional (compar %equiv?)) (assoc-simple  x y compar))
(define (mutex-lock! mtx :optional timeout (thread (current-thread)))
(define (mutex-unlock! mtx :optional condv timeout)

[jpellegrini]

For reference, this was my idea for rewrite-params-and-body:

(define (rewrite-params-and-body method? formals body)
  (let* ((params (parse-parameter-list method? formals))
         (req    (car params))
         (opt    (or (cadr params)   '()))
         (key    (or (caddr params)  '()))
         (rest   (cadddr params)))
    (let* ((opt-names (map car opt))
           (opt-tests (filter symbol? (map caddr opt)))
           (key-names (map make-keyword (map car key)))
           (key-tests (filter symbol? (map caddr key))))
      (values (if (or opt key)
                ;; We have a :optional or a :key keyword
                (let ((new-formals(append req
                                          opt-names
                                          opt-tests
                                          key-names
                                          key-tests
                                          (if rest (list rest) '()))))
                  (cons new-formals body))
                ;; "Normal" lambda
                (begin (if rest
                           (if (null? req)
                               (set! req rest)
                               (set-cdr! (last-pair req) rest)))
                       `(,req  ,@body)))
            opt key))))

But it needs to return opt and key also, because they'll be used in compile-user-lambda to call the new VM instruction. Also, macros use closures, and I didn't know how to work this out for them.

[egallesio]

Hi @jrpellegrini,

This work is really interesting. I have already thought to a similar thing before (it is quite similar to the way Bigloo implement DSSSL keywords). However, I gave up since I didn't find a good way to use it when you are in a multi files context. When you want to call (g 1 2 :f 10), you must know that there is a potential keyword e that must be passed before the value of f. Since you must know, the “holes” that must be fitted, you need to have stored some meta information somewhere. For .ostk, this could be done in the header file, but if the file is not compiled, it is more difficult (one way could that the import clause gives the signature of the function, but this is nor R7RS and quite a burden imo) . If you have an idea on this point, I'll happily take it.

About macros, the lambda used can be an extended lambda, so it should not be a problem:

stklos> (define-macro (foo a b :optional x) `(list ',a ',b ',x))
;; foo
stklos> (foo x y)
(x y #f)
stklos> (foo x y z)
(x y z)

[jpellegrini]

When you want to call (g 1 2 :f 10), you must know that there is a potential keyword e that must be passed before the value of f. Since you must know, the “holes” that must be fitted, you need to have stored some meta information somewhere. For .ostk, this could be done in the header file, but if the file is not compiled, it is more difficult (one way could that the import clause gives the signature of the function, but this is nor R7RS and quite a burden imo) .

I'm not sure I understand -- do you mean when I create a procedure g in one file and use it in another?

;; file A.stk
(define (g a b :key (e 100) (f 200)) (list a b e f))

;; file B.stk
(include "A.stk")
(display (g 1 2 :f 10))

If A is included, the definition of g is textually included in B, and the default 100 value for e would be used... So I'm not sure I understood.

(The code I was working on would keep the defaults inside the closure object, so the VM only needs to check what's missing, and reorder the keyword arguments.)

[jpellegrini]

As to the compiled file, I suppose it would work if the compiler issues the CLOSURE-SET-OPTS-KEYS instruction: it'd go with the code that defines the closure, and the list of keywords and optionals would automatically be put in the header. Or did I misunderstood or forget something?

[jpellegrini]

The code I was working on would keep the defaults inside the closure object

So, since the closure object knows its optional and keyword arguments, no bookkeeping is necessary, I suppose. (no?)

[egallesio]

In fact, I was thinking of something like

;; file A.stk
(define (g a b :key (e 100) (f 200)) (list a b e f))

and

;; file B.stk
(require "A")
(display (g 1 2 :f 10))

Id A is compiled, we can have some information embedded in the header of the .ostk file. If this not the case, we probably cannot know that we have to generate a call such as (g 1 2 100 10).

Furthermore, we'll need some kind of static analysis even if everything is a single file:

(define (f a b) (list a b))
(define (g a b :key (e 100) (f 200)) (list a b e f))

(set! f g)
(f 1 2)     ;; Here we must compile a call such as (f 1 2 100 200)

And static analysis can be not sufficient, since the value of f cannot be known at compile time

(when (foo) (set! f g))
(f 1 2)     ;; do we call original f or original g?

In fact, I think that the thing you try to do can be done at compile time using macros. For the example you gave before we can implement:

(define x 0)
(define y 0)

(define-macro (G a b
                 :optional (c 10 c?) (d 20)
                 :key      (e -1 e?) (f -2))

  (define (%real-G a b ;; mandatory
                   c d ;; optionals
                   c?  ;; tests for optionals
                   e f ;; keyword arguments
                   e?  ;; tests for keywords
                   r)  ;; rest list
    (set! x (+ x c f))
    (set! y (+ y e d)))
  `(,%real-G ,a ,b ,c ,d ,c? ,e ,f ,e? '()))

(time (repeat 10_000_000
              (G 0 0 2 :e 1 :f -2)))
(print (list x y))

Here the call that must be generated is determined at compile time (and it can be in another file as macros)
This code is 4 times faster to an implementation of G with an extended lambda (1.5s vs 6s on my laptop). The main drawback I see here, is that we cannot pass G to another function (OK we can, but we'll pass a syntax which cannot be used in the function).

So, a possible way to implement faster extended functions could be (perhaps) to use a new type of object which is macro expanded when used in a call and compiled as actually if passed as a parameter or when we cannot prove that that we can do an efficient call in sequences such as (begin (set! f g) (f 1 2) as seen before.

It is a bit complicated, and we would need two implementations for the extended function (one normal lambda and one which correspond to the actual extended lambda). I need to think more about it, but it should work.

I'm not sure to have been clear.

[jpellegrini]

If this not the case, we probably cannot know that we have to generate a call such as (g 1 2 100 10).

Oh, but we don't :)
The procedure knows it needs more parameters, and it provides its own defaults.
If you call (g 1 2 :f 10), the VM will see exactly 1 2 :f 10 as arguments, the arity will be negative, and the VM will then supply the missing elements on the stack by asking the procedure what its defaults are.

If the procedure was compiled, then the compiler generated something like

000:  CREATE-CLOSURE       3 -3	;; ==> 005
003:  IM-MINUS1           
004:  RETURN              
005:  CLOSURE-SET-OPT-KEY       0 1         <=== it already included the optionals and key arguments in the closure
007:  DEFINE-SYMBOL        2

If it was not compiled, then when the source is evaluated the optionals and keywords will be read...

Your example:

(when (foo) (set! f g))
(f 1 2)     ;; do we call original f or original g?

would still work, because g carries with it its optionals and keyword arguments and defaults, and f will have those too (it's the same closure_obj object). Its arity is negative (-3 in this case), so the compiler knows there are 2 mandatory args, and the rest is decoded by the VM by asking the closure object...

This is the change to struct closure_obj:

struct closure_obj {
  stk_header header;
  int16_t arity;
  uint16_t code_size;
  SCM formals;
  SCM opt_args; /* a list of default values for optional arguments */  <= here
  SCM key_args; /* an assoc list; keyword -> default */  <= and here
  SCM env;
  ... (etc)
};

key_args has the names, defauls, and test names for each argument, so the VM can take proper action. That list is exactly what %procedure-keys returns:

stklos> (%procedure-keys f)
((#:e -1 e?) (#:f -2))

The VM (in adjust_stack_get_arity) does call STk_key_get in order to find all missing arguments and push them. It also counts the optionals and brings the missing defaults, which are in the opt_args field of the closure_obj structure.

This is what my implementation currently does:

stklos> (define (g a b ;; mandatory
           c d ;; optionals
           c?  ;; tests for optionals
           e f ;; keyword arguments
           e?  ;; tests for keywords
           r)  ;; rest list
  (list a b c c? d e e? f r))
;; g
stklos> (define f #f)
;; f
stklos> (set! f g)    ;; Let's assign g to f, even before changing its arity and args
;; g
stklos> (%%set-procedure-arity! g -3)
#t
stklos> (%%set-procedure-optionals-and-keys! g '( (c 10 c?) (d 20) ) '( (:e -1 e?) (:f -2) ) )
#t
stklos> (%procedure-arity f) ;; the arity and args of f were changed, because its eq? to g...
-3
2
2
stklos> (%procedure-optionals f)
((c 10 c?) (d 20))
stklos> (%procedure-keys f)
((#:e -1 e?) (#:f -2))                   <= f (which is eq? to g) knows that it needs "e" and "f" 
                                            keyword arguments, and also knows the defaults,
                                            and also knows that e has a test, "e?". The VM will
                                            get that information and use it to reorganize the stack,
                                            "as if" f was called with all its arguments!
stklos> (f 1 2)
(1 2 10 #f 20 -1 #f -2 ())     <= The call to f worked correctly!

I think I didn't understand yet the problem.

About the macro implementation,

The main drawback I see here, is that we cannot pass G to another function

By storing the arguments and defaults in the closure, we can pass it along freely, and the VM will always be able to decode it. I thought about using macros, but then thought that maybe with procedures it would be easier, because the closure_obj would carry all information needed to decode any calls to it, so it can be passed around...

[egallesio]

The procedure knows it needs more parameters, and it provides its own defaults.
But when you compile, you may not have the procedure (you have it at runtime, not at compile time).

I don't understand how you can compile the call of bar in foo. You have to produce a call which is (bar 1 2 #f 100 1). Whence you produce the code for foo you don't know that bar must be called with 5 parameters.

(define (foo)
  (bar  1 2))  

(define (bar x y :optional z :key  (a 100) (b 200))
  (list  x y z a b r))

By storing the arguments and defaults in the closure, we can pass it along freely, and the VM will always be able to decode it.

No problem for the VM since at runtime you have the closure. At compile time, the closure may not exist.

[jpellegrini]

I don't understand how you can compile the call of bar in foo

I thought of working as if bar had variable arguments. In fact, the idea was to make its arity negative.
The call is compiled as if there were only two arguments. When the closure is available, it will know (and provide) the missing arguments. Maybe I'm missing something -- I'll later dig into this more (and do some more coding) so I understand what the problem is.

[jpellegrini]

@egallesio

I think I got it almost working.

There is a sequence of six commits in this PR. The last one is a temporary proof-of-concept change to the compiler, which introduces a new special form Lambda (with a capital L), that uses the new code.

You can define a Lambda in a file, compile it, and load from another.

;; A.stk
(define f
  (Lambda (a :optional (b 10 b?) :rest r :key (c 20 c?))
          (list a '-- b b? '--  c c? '-- r)))

Compile A.stk into A.ostk...

;; B.stk
(load "A.ostk")
(define g f)
(format #t "result: ~w~%" (g -1 :c -2))

If you load B.stk, it will work fine and print the expected result.

This is because the Lambda in A.stk is compiled into:

000:  IM-VOID             
001:  CREATE-CLOSURE       18 -2	;; ==> 021
004:  LOCAL-REF            5
006:  PUSH                
007:  CONSTANT-PUSH        0
009:  LOCAL-REF4-PUSH     
010:  LOCAL-REF3-PUSH     
011:  CONSTANT-PUSH        0
013:  LOCAL-REF2-PUSH     
014:  LOCAL-REF1-PUSH     
015:  CONSTANT-PUSH        0
017:  LOCAL-REF0-PUSH     
018:  IN-LIST              9
020:  RETURN              
021:  CLOSURE-SET-OPT-KEY  1 2       <== HERE! It sets the optionals and keywords!
024:  DEFINE-SYMBOL        3
026:  END-OF-CODE         

CLOSURE-SET-OPT-KEY will set the optinals and keyword lists, so the closure will be created with the proper argument names, optionals and test names.

But I noticed that there is some small glitch somewhere (the test value is being used instead of the arg value in one specific situation).

Still working on this

[jpellegrini]

It seems to work properly. But if I turn this Lambda into the standard lambda in the compiler, it fails to compile the boot files.
I have added one VM instruction and also changed the standard LET in STklos. Does this break the boot files? How do we revocer from this?

[jpellegrini]

I've resolved the conflicts for merging, but I'll have to check again the working of this. Keeping information (including keyword and optional args) in the plist is great!

Anyway -- when the procedure is created, these

CLOSURE_OPT_ARGS(z) = STk_nil;
CLOSURE_KEY_ARGS(z) = STk_nil;

were done in the C code. I've just removed them, and this is a reminder for myself that the equivalent using plists needs to be put there.

[jpellegrini]

I adapted the code to using plists.
Since the plists are consulted at each procedure call that has optionals and keywords, it got slightly slower, but still much faster than using the original (now it goes from 17 to 10 instead of 17 to 9 seconds).

So... This is the state of this PR:

• The example with files A.stk and B.stk works
• It's still much faster than the original
• The last commit adds a Lambda special form that can be used to test the new code
• I still have to find a way to incorporate this into the standard lambda

[jpellegrini]

I think the only problem with this PR now is that I don't know how to bootstrap it... Everything works, but generating the boot file crashes.

[jpellegrini]

@egallesio -- is there some special trick to bootstrap STkos when you change the bytecode in a fundamental way, like this? (It changes the way procedures with optionals & keywords work).

I thought of going a radical route and try to temporarily change all opts&keys from the core STklos code, then generate the boot images, and then change back the procedures. Is there another way?

[egallesio]

I thought of going a radical route and try to temporarily change all opts&keys from the core STklos code, then generate the boot images, and then change back the procedures. Is there another way?

Not really; You need to rewrite some code using the new feature, test, peak some other code and use the new feature, test and so on. When you have all base code converted, you're' done (work by small steps, otherwise it'll break, and you'll don't know why).

For now, I have seen that Lambda,

• doesn't work when there are no optional or keyword.
• doesn't accept function names as keyword (this is undocumented)
  • (%procedure-name (lambda () "doc" :name 1)) => "name"
  • (%procedure-doc (lambda () "doc" :name 1)) => "doc"
• is not compatible with actual implementation of lambda. See below

(define f1
  (Lambda (a :optional (b 10 b?) :rest r :key (c 20 c?))
          (list a '-- b b? '--  c c? '-- r)))

(define f2
  (lambda (a :optional (b 10 b?) :rest r :key (c 20 c?))
          (list a '-- b b? '--  c c? '-- r)))

stklos> (f1 1 2)
(1 -- 2 #t -- 20 #f -- (2))
stklos> (f2 1 2)
(1 -- 2 #t -- 20 #f -- ())
stklos>

Anyway, it seems promising (even if I don't see improvements as big as you).

In any case, I will not try to integrate this PR for this release, since it changes too many things.

[jpellegrini]

In any case, I will not try to integrate this PR for this release, since it changes too many things.

Of course, it's a big change, I wouldn't expect it to be included that soon.

Ok, I'll check the problems that you have described!

[egallesio]

Ok, I'll check the problems that you have described!

If you have time, and want to, it would be great.

[jpellegrini]

is not compatible with actual implementation of lambda. See below

I believe this one is fixed... Will work on the others later.

[jpellegrini]

• doesn't work when there are no optional or keyword.
• doesn't accept function names as keyword (this is undocumented)

He he. These two are related. An #f is being left on the stack instead of the documentation (actually wether there is documentation or not!) I'll work on them later.

[jpellegrini]

I believe I've fixed all those problems. I'll see if I can make STklos bootstrap with the changes later this week.

[jpellegrini]

• But :doc doesn't work in plain STklos either, when it's not inside a named function:

 (%procedure-name (lambda () "y" :doc 1)) => "doc"     ;; should have been "y"