Language changes

[cueckoo]

Originally opened by @myitcv in cuelang/cue#339

Overview

The purpose of this Issue is to inform of the language changes that are currently being contemplated. These can be split into backwards-incompatible changes and strict extensions.

The goal is to resolve the changes that are backwards-incompatible as quickly as possible so that we can reach a point to guarantee backwards compatibility as soon as. A general guarantee is foreseen for v1.0.0, but it could be that we fix the language beforehand.

Backwards-incompatible changes

Many of these are proposals and still require a decision.

Int vs Float vs Number

There is an outstanding proposal to simplify CUE's numbering system by removing the distinction between the various number types and have an independent constraint to require that a value is an integral number. A major consequence and objective of this change is that an integer constant can be assigned to a floating point value.

A major reason to have distinct number types is to allow round trips between different formats without loss of numeric precision. However, CUE's arbitrary precision decimals may already be good enough to achieve this without the need for special types. Other variants are possible.

See #253.

Use new "front-style" comprehensions for lists as well

Issue #112 proposes use the same style comprehensions for lists as are currently used for fields. This results indeed looks cleaner and it simplifies the language. It seems like a worthwhile change.

Change syntax for definitions

Issue #307 proposes changing the syntax for definitions from

Foo :: bar

to

#Foo: bar

This removes the syntax for ::. The main appeal, though, is that it puts definitions and normal fields in a different name space (using the same trick as older hidden fields).

One major benefit of this is that it makes it much easier to model other formats like JSON Schema and makes mapping of other formats more reliable. See #307 for more discussion.

Semantics of [foo]: bar fields

CUE currently defines the constraints of bulk optional fields to be applied to any field in a struct that matches it. This is a different from how patternProperties or additionalProperties work in JSON Schema. It turns out it is cumbersome to express the JSON Schema interpretation in CUE. Conversely, it is quite trivial to express the CUE interpretation in JSON Schema. It seems that CUE would be better served adopting the JSON Schema semantics.

It has to be investigated, though, if this generally feasible. To be able to punt on this decision for now, we propose allowing build optional fields to be only specified in a struct by itself. This means that the majority of cases can stay as is. Cases like

a : {
  b: 3
  [string]: int
}

can trivially be rewritten to

a : {
  b: 3
  { [string]: int }
}

Even if we decide to change to JSON Schema semantics, this restriction would have to be introduced for a while to allow for a bug-free transition.

Remove deprecated syntax

Space-separated labels and old-style field comprehensions will be removed imminently.

Simplified closedness rules

Closed structs are a bit tedious to use in practice. This is partly because the evaluator has some bugs, and partly because the definition is too restrictive.

A good API design allows for extensibility. CUE's closed struct seem to fly in the face of this concept. The main purpose of closed structs, however, is to catch typos in field names. Experience with GCL and other languages indicate this is a common mistake and can give quite a headache.

How do we reconcile these two goals? A good analogy can be drawn to Protocol Buffers, which by design allows the addition of arbitrary fields, but once the respective definitions are converted to typed languages like Go, a misspelled field will cause a compile error. Closed structs should be seen in this context.

Closed structs do play another role: the presence of fields may disambiguate a disjunction. That is a disjunct that would cause a "compilation error" because of the presence of a field in a struct with which it is unified is eliminated from a disjunction.

A goal of the evaluator redesign is to relax the restrictions of closed structs to make them more convenient and find a definition that does not conflict with the concept of open APIs, while being formally rigorous enough to serve as a disambiguator of disjunctions.

Reintroduce slices

Slices were removed from the spec, but never from the language implementation. With the foreseen query extension, it seems proper to reintroduce them. That may give an opportunity to reevaluate their semantics, for instance whether Python-style slices would be more appropriate.

Consider changing token for regexp matching

CUE uses the =~ operator for regexp matching. With the introduction of bulk optional fields this turns out to be somewhat awkward. Using an alias in such a field, for instance [X= =~"^P{Lu}"]: foo requires an awkward space or additional parentheses. It's clumsy and common enough to be annoying. Using the AWK-style ~ token instead may be better in the future.

Use of ? and/or ! for optional fields

In proper API design, most fields should probably be optional. This requires an additional ? in CUE, meaning that the majority of fiels will have such a question mark. It maybe worth evaluating if a GraphQL-style ! for required fields makes more sense.

There are other option would be a hybrid mode: when exporting, export only concrete fields and drop non-concrete fields. The ? could then be used to explicitly mark a field as optional. The ! could be used to explicitly make a field undefined, but required. This would eliminate the need for either marker in most cases, while allowing specifying combinations that currently are sometimes needed, but not possible. For instance, this would allow a!: 1 (field a must be specified by a user and must be 1), and a!: [...int] (if specified, a must be a list of ints).

Language extensions

Issue #165 proposes a query extension to the language.

Other contemplated language extensions include:

• embedded scalar values (like top-level, but then for any struct)
• type casts, including struct casts
• a{b} operator for modeling closed structs
• topological sort of fields, providing some guarantees on order

Links

• All issues specific to this roadmap issue
• Open issues specific to this roadmap issue