Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly support position encodings #5778

Open
mkaput opened this issue Jun 11, 2024 · 1 comment
Open

Properly support position encodings #5778

mkaput opened this issue Jun 11, 2024 · 1 comment
Labels
bug Something isn't working ide This issue refers to CairoLS or editor extensions

Comments

@mkaput
Copy link
Contributor

mkaput commented Jun 11, 2024

LS is not aware of and thus is not performing position encoding kind negotiation, which means the language client will assume UTF-16. Actually, LS is using UTF-8 because that's the native encoding of Rust strings. This means bad things can happen if files contain non-ASCII characters, especially ones like Emoji, which span multiple Unicode codepoints.

Things to implement:

  • Properly negotiate position encoding with the language client. Prefer UTF-8 to avoid re-encoding files, but fall back to UTF-16 as this is the only encoding guaranteed to be universally supported by clients.
  • When converting Cairo positions to LSP ones, take into account encoding differences. This will require knowing file source at conversion time, which will be a large refactoring.
  • Enforce UTF-8 encoding in E2E tests (add appropriate asserts in MockClient).
@mkaput mkaput added bug Something isn't working ide This issue refers to CairoLS or editor extensions labels Jun 11, 2024
@mkaput
Copy link
Contributor Author

mkaput commented Jul 9, 2024

Using https://docs.rs/line-index/latest/line_index/ for line counting instead of custom code may enable this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ide This issue refers to CairoLS or editor extensions
Projects
None yet
Development

No branches or pull requests

1 participant