[New Feature] Allow spread operator in list-constructor-expr

[lochana-chathura]

This is to discuss and give updates on implementing the spread operator support in list-constructor-expr.

Spec Issue: ballerina-platform/ballerina-spec#52
Lang issue: #35178
Upstream branch: spread-operator

Samples:

int[] nums = [3,4];

var v1 = [1, 2, ...nums];
io:println(v1); // @output [1,2,3,4];

var v2 = [1, 2, ...[3,4]];
io:println(v2); // @output [1,2,3,4];

var v3 = [1, 2, ...nums, 5, 6];
io:println(v3); // @output [1,2,3,4,5,6];

var v4 = [1, 2, ...[4,5], 5, 6];
io:println(v4); // @output [1,2,3,4,5,6];

[int, string] t = [5, "five"];
var v5 = [1, 2, ...nums, ...t];
io:println(v5); // @output [1,2,3,4,5,"five"];

var v6 = [1, 2, ...[...nums, 5, 6]];
io:println(v6); // @output [1,2,3,4,5,6];

[lochana-chathura]

Update on syntax-related changes.
CC: @rdulmina @sameerajayasoma @hasithaa

Current syntax:
list-constructor-expr := [ [ expr-list ] ]
expr-list := expression (, expression)*

New syntax:
list-constructor-expr := [ [ list-members ] ]
list-members := list-member (, list-member)*
list-member := expression | spread-member
spread-member := ... expression

Current syntax tree structure.

        {
            "name": "ListConstructorExpressionNode",
            "base": "ExpressionNode",
            "kind": "LIST_CONSTRUCTOR",
            "attributes": [
                {
                    "name": "openBracket",
                    "type": "Token"
                },
                {
                    "name": "expressions",
                    "type": "Node",
                    "occurrences": "MULTIPLE_SEPARATED"
                },
                {
                    "name": "closeBracket",
                    "type": "Token"
                }
            ]
        }

Currently, for the expressions we allow comma separated expression list. Notice the type is Node. Not ExpressionNode. This allows us to allow the new node say SpreadMemberNode to the existing list without changing the type of the list. Yet referring it to as expressions is not good, as it now includes the spread operator.

So the proposed new structure.

        {
            "name": "ListConstructorExpressionNode",
            "base": "ExpressionNode",
            "kind": "LIST_CONSTRUCTOR",
            "attributes": [
                {
                    "name": "openBracket",
                    "type": "Token"
                },
                {
                    "name": "listMembers", // A member will be either an ExpressionNode or an SpreadMemberNode
                    "type": "Node",
                    "occurrences": "MULTIPLE_SEPARATED"
                },
                {
                    "name": "closeBracket",
                    "type": "Token"
                }
            ]
        },
        {
            "name": "SpreadMemberNode",
            "base": "Node",
            "kind": "SPREAD_MEMBER",
            "attributes": [
                {
                    "name": "ellipsis",
                    "type": "Token"
                },
                {
                    "name": "expression",
                    "type": "ExpressionNode"
                }
            ]
        }

Notice spread operator syntax will be ... expression (not just ... variable-reference) similar to spread-field in mapping-constructor-expr

[rdulmina]

+1 for the design, btw ... expression is not just an operator so the name SpreadOperatorNode wont be suitable for this. How about SpreadMemberNode?

[lochana-chathura]

My bad. +1. Details updated accordingly.

[sameerajayasoma]

Looks good

[hasithaa]

After discussion with @lochana-chathura, we decided to keep, expressions instead of renaming to listMembers. The main reason for this is to keep backward compatibility, in the SyntaxAPI. We need to find a proper solution to deprecate Syntax APIs in such cases.

[lochana-chathura]

Update on the semantics.

CC: @hasithaa @MaryamZi @gimantha @pcnfernando

The static type of the expression in a spread-member must allow only list values, i.e. subtype of [(any|error)...]. All the member values of the list that results from evaluating that expression are included in the list value being constructed. When the applicable contextually expected type of the list constructor expression has a fixed length, the static types of all the spread member expressions should also have a fixed length.

Implemented an inner class named BLangListConstructorSpreadOpExpr inside BLangListConstructorExpr to represent the spread operator.

Inner Class (click to expand)

    /**
     * Implementation of list constructor spread operator.
     *
     * @since 2201.1.0
     */
    public static class BLangListConstructorSpreadOpExpr extends BLangExpression {
        
        private BLangExpression expr;

        public BLangExpression getExpression() {
            return expr;
        }

        public void setExpression(ExpressionNode expr) {
            this.expr = (BLangExpression) expr;
        }

        @Override
        public NodeKind getKind() {
            return NodeKind.LIST_CONSTRUCTOR_SPREAD_OP;
        }

        @Override
        public void accept(BLangNodeVisitor visitor) {
            visitor.visit(this);
        }

        @Override
        public <T> void accept(BLangNodeAnalyzer<T> analyzer, T props) {
            analyzer.visit(this, props);
        }

        @Override
        public <T, R> R apply(BLangNodeTransformer<T, R> modifier, T props) {
            return modifier.transform(this, props);
        }
    }

[lochana-chathura]

Elaborating further on the spread operator compatibility:

When the applicable contextually expected type of the list constructor is,

1. a fixed-length list, only spread member expressions having fixed-length types are allowed.
   e.g.

int[] a1 = [1, 2];
int[2] _ = [...a1]; // @error invalid usage of spread operator; length of 'a1' is not fixed
[int, any] _ = [... a1]; // @error invalid usage of spread operator; length of 'a1' is not fixed

[(int|string)...] a2 = [1, "s"];
(int|string)[2] _ = [...a2]; // @error invalid usage of spread operator; length of 'a1' is not fixed
[int, any] _ = [... a2]; // @error invalid usage of spread operator; length of 'a1' is not fixed

int[2] a3 = [1, 2];
int[2] _ = [...a3]; // @output [1,2]
[int, any] _ = [...a3]; // @output [1,2]

[(int|string), "s"] a4 = [1, "s"];
(int|string)[2] _ = [...a4]; // @output [1, "s"]
[int|string, any] _ = [...a4]; // @output [1, "s"]

int[2] a5 = [];
string[1] a6 = [];
[int, int, string] _ = [...a5, ...a6]; // @output [0, 0, ""]

2. an open array, both spread member expressions having fixed and non-fixed length types are allowed.
   e.g.

int[] a1= [1, 2];
int[] _ = [...a1]; // @output [1,2]
int[] _ = [3, ...a1, 4, ...a1]; // @output [3,1,2,4,1,2]

[int, string...] a2 = [];
(int|string)[] _ = [...a2]; // @output [0]
(int|string)[] _ = [2, ...a2, "s"]; // @output [2,0,"s"]

a2[2] = "s";
(int|string)[] _ = [...a2]; // @output [0,"","s"]
(int|string)[] _ = [2, ...a2, "s"]; // @output [2,0,"","s","s"]

[int, string] a3 = [];
int[*] a4 = [1,2];
(int|string)[] _ = [...a3, ...a4]; // @output [0,"",1,2]

3. a tuple with rest descriptor, both spread member expressions having fixed and non-fixed length types are allowed.
   e.g.

string[] a1 = ["s"];
[int, string...] _ = [1, ...a1, ...a1]; // @output [1,"s","s"]

string[2] a2 = ["s"];
[(int|string), string, string...] _ = [...a2, ...a2]; // @output ["s","","s",""]

string[1] a3 = ["s"];
[(int|string), int, string...] _ = [...a3]; // @output ["s",0]
[(int|string), (string|int), string...] _ = [...a3]; // @error no filler value for type '(int|string)'

string[] a4 = ["s"];
[string, string, string...] _ = [...a4, ...a4]; // @output ["s","s"]
[(int|string), string, string...] _ = [...a4, ...a4]; // @error no filler value for type '(int|string)'
[string, int...] _ = [...a4]; // @error incompatible types: expected 'int', found 'string'

[int, string] a5 = [1, "s"];
[int, string...] _ = [...a5, "x"]; // @output [1,"s,"x"]

[string...] a6 = ["s"];
[string, int, string...] _ = [...a6, 1]; // @error incompatible types: expected 'int', found 'string'

[rdulmina]

@lochana-chathura Why do we need a class for spread members? can't we just have an expression field in BLangListConstructorExpr to hold it?

[lochana-chathura]

Having just one field is not enough right? We can have multiple spread members.
So could have a separate list like List<BLangExpression> spreadMemberExprs; for spread members. Yet we need to maintain the order of all list constructor members. So we have to add the spread members also to the existing public List<BLangExpression> exprs;.

[rdulmina]

Okay +1