Split Cairo out into separate Cairo and Cairo Zero languages

[mkaput]

This PR is updating the situation around the Cairo language to match the current state of things.

Disclaimer: I am involved in the development of the Cairo compiler and its dev tooling.

Description

Historical context

Cairo is a programming language that allows writing zero-knowledge-provable programs. Cairo is also the smart-contract language of Starknet.

In Jan 2023, Cairo 1.0 was released (blog post). Cairo 1.0 (and all versions upwards) is a completely new language built from scratch.

A couple of weeks later, we have decided a new naming strategy has been established:

• Cairo 1.0 and upwards is now called Cairo proper.
• The old Cairo is now called Cairo Zero.
• A tiny, strict subset of Cairo Zero, the assembly language of the Cairo virtual machine became CASM - Cairo Assembly.

These naming changes got community approval, but everything happened on an invite-only Telegram group, so I am unable to provide a link.

Cairo is not backwards-compatible with Cairo Zero. Cairo Zero development is frozen and the language will not receive any new features or updates.

As a side note: Cairo also brings another language - Sierra, which is an IR between Cairo and CASM. This PR is not concerning about Sierra at all.

What this PR changes

1. Renamed existing Cairo language to Cairo Zero.
2. Added a new language Cairo that represents the new Cairo.
3. Updated grammar sources to the new location that provides grammars for both languages: https://github.com/software-mansion-labs/cairo-tm-grammar
   • I am maintaining this repository. It is just a repackaging of grammars that this repo pulls from upstream repositories. I made sure to have everything documented and properly attributed there.
   • The grammar for Cairo is new, and I made it for the official Visual Studio Code extension for Cairo.
   • The grammar for Cairo Zero is different from the one that was used here before.
     • The new grammar comes from the old, official Visual Studio Code extension for Cairo Zero.
     • The old grammar was written by @0xChqrles. It has bugs that make it stop highlighting on some files (example). The new grammar does not exhibit this problem. I have talked with @0xChqrles about this, and he approves stopping using his grammar.
4. I have added heuristics for Cairo Zero that help disambiguate it from Cairo (as both languages use the same extension). With these heuristics and newly added samples, the classifier seems to identify the correct languages well, according to my testing.
5. Some samples of CASM language can appear on GitHub as *.casm. I have added two as *.cairo to help the classifier, but this PR is not doing anything more, as CASM is not significant enough.

Checklist:

• I am adding a new language.
  • The extension of the new language is used in hundreds of repositories on GitHub.com.
    • Search results for each extension:
      • https://github.com/search?type=code&q=NOT+is%3Afork+path%3A*.cairo+starknet
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • https://github.com/starkware-libs/cairo/blob/157b38ea28250bf88bdca6faf565935c1e57ff1e/corelib/src/boolean.cairo
      • https://github.com/cairo-book/cairo-book/blob/99673ae213e7ada059e3bc73251391461970b094/listings/ch11-advanced-features/no_listing_13_cust_struct_vect/src/lib.cairo
      • https://github.com/cairo-book/cairo-book/blob/99673ae213e7ada059e3bc73251391461970b094/listings/ch17-starknet-smart-contracts-security/listing_02_pizza_factory_snfoundry/src/pizza.cairo
      • https://github.com/keep-starknet-strange/alexandria/blob/0aa7cca8a1acb1f35cff32420614f7f58e38de5e/packages/merkle_tree/src/tests/merkle_tree_test.cairo
      • https://github.com/HerodotusDev/cairo-lib/blob/a0c0ee4feeb4edb61e7a35d29595ad8647c3d469/src/encoding/rlp.cairo
    • Sample license(s):
      • Apache-2.0 for starkware-libs/cairo
      • MIT for cairo-book
  • I have included a syntax highlighting grammar: https://github.com/software-mansion-labs/cairo-tm-grammar
  • I have added a color
    • Hex value: #ff4a48
    • Rationale: Brand color, per Media Kit.
  • I have updated the heuristics to distinguish my language from others using the same extension.
    • Not for Cairo language itself as it is still evolving, but added heuristics (& tests) that help to disambiguate Cairo Zero which is frozen.

[lildude]

See inline.

Please can you replace your new samples which are not real-world files with real-world examples.

[mkaput]

Please can you replace your new samples which are not real-world files with real-world examples.

Do you mean the CASM ones (1, 2)? Yes, they are auto-generated, but I used them because:

1. they do appear on GitHub already, and
2. they help the classifier distinguish Cairo 0 from Cairo, as they are full of Cairo 0-specific syntax; without these, the classifier fails to differentiate remaining samples

wdyt given this context?

[lildude]

Please can you replace your new samples which are not real-world files with real-world examples.

Do you mean the CASM ones (1, 2)? Yes, they are auto-generated, but I used them because:

1. they do appear on GitHub already, and
2. they help the classifier distinguish Cairo 0 from Cairo, as they are full of Cairo 0-specific syntax; without these, the classifier fails to differentiate remaining samples

wdyt given this context?

Yes, those files. The fact you can only get the classifier to match our current samples by fudging things kinda speaks for itself… either the current samples aren't good illustrations of the language or the new ones aren't. Or you're trying to match things which look completely different which normally indicates a whole different syntax and thus commonly a whole different language.

[mkaput]

Yes, those files. The fact you can only get the classifier to match our current samples by fudging things kinda speaks for itself… either the current samples aren't good illustrations of the language or the new ones aren't. Or you're trying to match things which look completely different which normally indicates a whole different syntax and thus commonly a whole different language.

I removed the bigger file, but left the smaller one as it's a good representation and actually hand-writable for some example purposes. I added more real-world Cairo and Cairo 0 samples to provide more data for the classifier. PR description is updated with source links.

[mkaput]

@lildude is there anything I have to add here? :)

[lildude]

LGTM. Thanks.

Note: this PR will not be merged until close to when the next release is made. See here for more details.

[mkaput]

Awesome! 😍 Thank you for review!

I saw this section in docs; thought you'd merge this now and will wait, just releasing from main. But it's fine whatever you do.

Are you able to give an order of magnitude, when do you plan to do a next release? I bet my community will love to hear about it, but it's also fine if you can't tell it here.

[lildude]

I make a release approx every 3-4 months. The last one was on 7 Jun so it won't be for a few months... probably late August.