[Spec Deviation] Support StringNumericEscape

[MaryamZi]

Description:

Updated 2020R1 Spec:

The syntax for Unicode escapes in strings has changed from \u[CodePoint] to \u{CodePoint} so as to align with ECMAScript. Although this is an incompatible change, the previous syntax was not implemented.

Related commit - ballerina-platform/ballerina-spec@eb2fa56

------Old Content-------

From 2019R1 Spec :

As per the spec:

...
string-literal :=
DoubleQuotedStringLiteral | symbolic-string-literal
DoubleQuotedStringLiteral := " (StringChar | StringEscape)* "
...
StringEscape := StringSingleEscape | StringNumericEscape
...
StringNumericEscape := \u[ CodePoint ]
CodePoint := HexDigit+

"In a StringNumericEscape, CodePoint must valid Unicode code point; more precisely, it
must be a hexadecimal numeral denoting an integer n where 0 <= n < 0xD800 or 0xDFFF <
n <= 0x10FFFF."

Following needs to be supported:

string s1 = "\u0";
string s2 = "\uD799";
string s3 = "\uEFFF";
string s4 = "\u10FFFF";

[rdhananjaya]

Current compiler implementation uses apache commons' StringEscapeUtils.unescapeJava method to unescape unicode literals and it only support \uXXXX (where X is a hex digit).

Since our current supported range is a subset of allowed range and using this supported range would not impede future expansion of this range, shall we lower the priority of this task.

[hasithaa]

Related to ballerina-platform/ballerina-spec#390

[jclark]

You have misunderstood the spec: there is a literal [ and ] in there. It's \u[D799] not \uD799. See https://ballerina.io/spec/lang/2019R3/#StringNumericEscape

Anyway, we decided to change to e.g. \u{D799} for compatibility with ECMAScript.

[jclark]

If you are currently supporting \uXXXX syntax, you should up the priority of this, since that is not the right syntax.

[hasithaa]

@jclark yes. It seems like a bug and we support \uXXXX syntax. I have increased priority for this issue.