Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove language group recursion #5987

Merged
merged 1 commit into from
Jul 22, 2022
Merged

Remove language group recursion #5987

merged 1 commit into from
Jul 22, 2022

Conversation

lildude
Copy link
Member

@lildude lildude commented Jul 22, 2022

After over 11 years within Linguist, and over 5 year of direct usage, it's been discovered Linguist::Language has a recursion problem that doesn't play nicely with Rails' .to_json:

[1] pry(main)> require 'linguist'
=> true
[2] pry(main)> require 'active_support/all'
=> true
[3] pry(main)> ld = Linguist::Language.find_by_name("Markdown")
=> #<Linguist::Language name=Markdown>
[4] pry(main)> ld.to_json
SystemStackError: stack level too deep
from /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:180:in `initialize_dup'
[5] pry(main)> wtf -v
[...]
9152: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:179:in `as_json'
9153: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:63:in `as_json'
9154: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:180:in `block in as_json'
9155: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:179:in `each'
9156: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:179:in `as_json'
9157: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:63:in `as_json'
9158: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/json/encoding.rb:38:in `encode'
9159: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/json/encoding.rb:23:in `encode'
9160: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:42:in `to_json'
9161: (pry):4:in `__pry__'
9162: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/pry-0.13.1/lib/pry/pry_instance.rb:290:in `eval'
9163: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/pry-0.13.1/lib/pry/pry_instance.rb:290:in `evaluate_ruby'
9164: /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/pry-0.13.1/lib/pry/pry_instance.rb:659:in `handle_line'
[...]
[6] pry(main)>

Closer examination (ie doing what Rails does) shows where the recursion is:

[7] pry(main)> ld.respond_to?(:to_hash)
=> false
[8] pry(main)> ld.instance_values.as_json
SystemStackError: stack level too deep
from /workspaces/github/vendor/gems/3.1.2/ruby/3.1.0/gems/activesupport-7.1.0.alpha.bb68040de4/lib/active_support/core_ext/object/json.rb:100:in `as_json'
[9] pry(main)> ld.instance_values
=> {"name"=>"Markdown",
 "fs_name"=>nil,
 "type"=>:prose,
 "types"=>[:data, :markup, :programming, :prose],
 "color"=>"#083fa1",
 "aliases"=>["markdown", "pandoc"],
 "tm_scope"=>"source.gfm",
 "ace_mode"=>"markdown",
 "codemirror_mode"=>"gfm",
 "codemirror_mime_type"=>"text/x-gfm",
 "wrap"=>true,
 "language_id"=>222,
 "extensions"=>[".md", ".livemd", ".markdown", ".mdown", ".mdwn", ".mdx", ".mkd", ".mkdn", ".mkdown", ".ronn", ".scd", ".workbook"],
 "interpreters"=>[],
 "filenames"=>["contents.lr"],
 "popular"=>false,
 "group"=>#<Linguist::Language name=Markdown>}
[10] pry(main)> 

💥 there's our recursion: "group"=>#<Linguist::Language name=Markdown>

We don't really need this as we have a group() method which gets the same information as and when it is needed so all we really need is the name.

This PR removes the recursive assignment.

@lildude lildude requested a review from a team as a code owner July 22, 2022 08:48
Copy link
Collaborator

@Alhadis Alhadis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11 years? Damn.

@lildude lildude merged commit 1f307eb into master Jul 22, 2022
@lildude lildude deleted the lildude/remove-recursion branch July 22, 2022 09:02
@lildude lildude mentioned this pull request Jul 22, 2022
2 tasks
@Alhadis
Copy link
Collaborator

Alhadis commented Aug 7, 2022

@lildude One of the tests has started failing for an unfamiliar reason:

  1) Failure:
TestLanguage#test_all_languages_have_a_valid_ace_mode [/home/runner/work/linguist/linguist/test/test_language.rb:420]:
The following languages do not have an Ace mode listed in languages.yml. Please add an Ace mode for all new languages.
If no Ace mode exists for a language, mark the language with `ace_mode: text` in lib/linguist/languages.yml.
Toggle listABAP abap AGS Script c_cpp API Blueprint markdown ATS ocaml ActionScript actionscript Ada ada Alpine Abuild sh Altium Designer ini ApacheConf apache_conf Apex java Apollo Guidance Computer assembly_x86 AppleScript applescript AsciiDoc asciidoc Assembly assembly_x86 Asymptote c_cpp AutoHotkey autohotkey AutoIt autohotkey Batchfile batchfile Beef csharp BibTeX tex Bluespec verilog C c_cpp C# csharp C++ c_cpp C-ObjDump assembly_x86 C2hs Haskell haskell COBOL cobol CODEOWNERS gitignore Cabal Config haskell Nunjucks nunjucks OCaml ocaml ObjDump assembly_x86 Objective-C objectivec Objective-C++ objectivec OpenCL c_cpp OpenRC runscript sh OpenSCAD scad PLSQL sql PLpgSQL pgsql Pascal pascal Perl perl PicoLisp lisp Pod perl Pod 6 perl PowerShell powershell Procfile batchfile Prolog prolog Protocol Buffer protobuf Pug jade PureScript haskell Python python R r RDoc rdoc RMarkdown markdown RPC c_cpp Racket lisp Raku perl ReScript rust Reason rust ReasonLIGO rust Ren'Py python Rouge clojure Ruby ruby Rust rust SCSS scss SQL sql SQLPL sql SRecode Template lisp SWIG c_cpp Sage python Sass sass Scala scala Scheme scheme Shell sh ShellCheck Config ini ShellSession sh Smarty smarty Squirrel c_cpp Starlark python Stylus stylus SystemVerilog verilog TOML toml TSQL sql Tcl tcl Tcsh sh TeX tex TextMate Properties properties Textile textile Twig twig TypeScript typescript Unified Parallel C c_cpp Unix Assembly assembly_x86 Uno csharp UnrealScript java V golang VHDL vhdl Vala vala Velocity Template Language velocity Verilog verilog Volt d WebAssembly lisp Win32 Message File ini Windows Registry Entries ini X BitMap c_cpp X PixMap c_cpp XC c_cpp XS c_cpp edn clojure wisp clojure

Would this have been caused by the changes made in this PR, you think?

@lildude
Copy link
Member Author

lildude commented Aug 8, 2022

Huh‽ 😱 I've never seen that either. I don't see how my changes could have caused this as the test passed in that PR and on master since the PR was merged. I'll dig into it this week.

@lildude
Copy link
Member Author

lildude commented Aug 8, 2022

Yup, this is definitely not my fault 😁 The cause is the upstream Ace repo has had a bit of restructuring in ajaxorg/ace#4851 which means we're looking in the wrong place now. They appear to have missed two files as part of this move for some reason: jsoniq.js and xquery.js our tests will fail on these two.

I've started a PR to fix this in #6002 however I'm investigating if we need to keep the ace_mode property as GitHub hasn't use the Ace editor in a very long time.

@github-linguist github-linguist locked as resolved and limited conversation to collaborators Jun 17, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants