Implementability of joins and lazy query expressions

[grainier]

Consider the following examples for join and lazy query-expr;

join sample (ref[1])

stream<DeptPerson> outputStream = 
    from var person in personStream 
    join var dept in departmentStream
    on person.id == dept.id
    select {
        firstName: person.firstName,
        lastName: person.lastName,
        dept : dept.name
    };

lazy query-expr sample (ref[2])

stream<DeptPerson> outputStream = 
    stream from var person in personStream 
    from var dept in deptList   
    where person.id == dept.id       
    select {
        firstName: person.firstName,
        lastName: person.lastName,
        dept : dept.name
    };

In such examples, to evaluate on, where conditions and move further (i.e to select) we'll have to compare each and every member of the first stream (i.e personStream) with all the members of the second stream/list (i.e departmentStream, deptList).

This will lead to two issues;

1. When we evaluate the 1st event of StreamA, it gets consumed and won't be available for the second pass for it to be evaluated with the second event of StreamB.
2. Even if we maintain a state of 1st event of StreamA to address above, and move to the 2nd event of StreamA to be evaluated with StreamB, the StreamB will be fully consumed at this point.

As I understand, this will be an issue when proceeding with the implementation. What would be the best approach to overcome this? And since the [1], [2] are open, it would be great if you could consider this case for them as well.

Related Issues:
[1] join: #435
[2] query-expr: #443 / #436

[jclark]

Unbounded streams and streaming join is part #440.

At this stage we are not doing unbounded streams. So the implementation for [1] would read the entire departmentStream once and build a hash table, exactly as with deptList [2].

This is specified in #435.

[jclark]

Note the syntax uses equals (like LINQ), not ==.

[sanjiva]

James wouldn't "where x = y" be very confusing for mort?

We have let x = y too, so I think we should stick to where x == y.

[jclark]

The syntax is not where x = y. It is on L equals R, where L and R are separate expressions which are evaluated at different times with different things in scope. See #435. Using == instead of equals would be bad because it has quite different semantics from a == expression (which is why LINQ uses equals not ==).