Add BQN language #6623

skykanin · 2023-12-04T14:29:58Z

Add support for the BQN language as it's now passed the 2k unique file threshold.

Checklist:

I am adding a new language.
- The extension of the new language is used in hundreds of repositories on GitHub.com.
  - Search results for each extension:
    - https://github.com/search?type=code&q=NOT+is%3Afork+path%3A*.bqn
- I have included a real-world usage sample for all extensions added in this PR:
  - Sample source(s):
    - https://github.com/Brian-ED/rayed-bqn
  - Sample license(s):
- I have included a syntax highlighting grammar: [URL to grammar repo]
- I have added a color
  - Hex value: #21584F
  - Rationale: This is the main colour associated with BQN
- I have updated the heuristics to distinguish my language from others using the same extension.

lildude · 2023-12-04T15:07:15Z

Add support for the BQN language as it's now passed the 2k unique file threshhold.

Not quite. 1 repo alone accounts for 380 of those files.

skykanin · 2023-12-04T17:42:46Z

Add support for the BQN language as it's now passed the 2k unique file threshhold.

Not quite. 1 repo alone accounts for 380 of those files.

So how many files can a single repository maximally account for as a percentage of the threshold?

lildude · 2023-12-04T17:59:37Z

So how many files can a single repository maximally account for as a percentage of the threshold?

It's not about how many files a repo has but rather how widely used the extension is. If there is an indication of a single user or repo having a disproportionate impact on the usage, I'll exclude it... 2000 files spread across 200 repos is a lot more indicative of widespread usage than 2000 files in a single repo.

lildude

Along with addressing the failing tests, we need the cached license file from the grammar. When you ran the script/add-grammar script, it should have downloaded a copy.

If it didn't (I've seen this a few times but haven't had the chance to look into it), you can get the file by running this line from the script:

linguist/script/add-grammar

Line 134 in 7ca3799

bundle exec licensed cache -c vendor/licenses/config.yml

You'll also need to run script/list-grammars also run at the bottom of that script.

skykanin · 2023-12-05T08:59:41Z

You'll also need to run script/list-grammars also run at the bottom of that script.

When I try to run script/list-grammars I get the following error:

./script/list-grammars
Traceback (most recent call last):
        9: from ./script/list-grammars:4:in `<main>'
        8: from ./script/list-grammars:4:in `require'
        7: from /home/skykanin/Projects/linguist/lib/linguist.rb:5:in `<top (required)>'
        6: from /home/skykanin/Projects/linguist/lib/linguist.rb:5:in `require'
        5: from /home/skykanin/Projects/linguist/lib/linguist/language.rb:9:in `<top (required)>'
        4: from /home/skykanin/Projects/linguist/lib/linguist/language.rb:9:in `require'
        3: from /home/skykanin/Projects/linguist/lib/linguist/classifier.rb:1:in `<top (required)>'
        2: from /home/skykanin/Projects/linguist/lib/linguist/classifier.rb:1:in `require'
        1: from /home/skykanin/Projects/linguist/lib/linguist/tokenizer.rb:2:in `<top (required)>'
/home/skykanin/Projects/linguist/lib/linguist/tokenizer.rb:2:in `require': cannot load such file -- linguist/linguist (LoadError)

Seems that it's unable to load linguist/linguist from lib/linguist/tokenizer.rb for some reason.

lildude · 2023-12-05T09:32:34Z

You need to install the linguist gem... or use the Codespace we recommend.

skykanin · 2023-12-05T12:41:06Z

You need to install the linguist gem... or use the Codespace we recommend.

Is that different from the github-linguist gem? Cause I have that one installed and I can't see any gem simply called "linguist"

lildude · 2023-12-05T13:43:44Z

Yes, it's the github-linguist gem. If you've got it installed, try running bundle exec ./script/list-grammars else try the Codespace.

lildude · 2023-12-05T14:43:02Z

Oh yes, and you'll need to run it after you've sorted out the other issues reported in the test failures as the vendor/README.md file which script/list-grammars will only be updated once you've added the missing tm_scope field.

RocketRace · 2023-12-07T19:55:41Z

Add support for the BQN language as it's now passed the 2k unique file threshhold.

Not quite. 1 repo alone accounts for 380 of those files.

Now, this obviously doesn't discount the issue that many of the files are concentrated in one repository. However, I think it's worth clarifying that the linked repository automatically mirrors solutions to Rosetta Code programming tasks implemented by various users of the site, as opposed to a single power user.

mlochbaum · 2023-12-08T15:05:02Z

There are quite a lot of non-indexed repositories with BQN code. I found this while looking for Advent of Code repositories for this page and this year's chart, and as I investigate it keeps getting worse...

I check whether a repository is indexed by finding a BQN file and searching for its path or a few keywords in it. For indexed repositories I'm always able to find the file. For others I can't. I've checked some of the Advent of Code repos by searching in the repo, and it says it needs to index it and then a quarter hour later or so I can find files. Today I found these on pages 5 and 6 of a repository search; these are the only ones I tried and none are indexed!

List of 35 AoC repos I believe aren't indexed, out of <90 checked (but as I'm using manual searches, there's probably an error or two in here). These account for 308 bqn files based on local clones. Biased for 2023 because I can only find non-indexed repos by searching repo names/descriptions and commit messages, and it doesn't include the two or three I force-indexed.

Non-indexed Advent of Code

So I'm fairly certain we could go past the 2k requirement by searching in some repositories. I'm also sure there are over 200 repositories containing BQN code and could probably come up with a list if that's acceptable. Thoughts?

mlochbaum · 2024-01-08T17:13:32Z

I've indexed the following 28 Advent of Code repositories, which brings this search up to 2k. I did find that some repositories I listed earlier were actually deleted or renamed, which I hadn't thought to check, and have updated that list to indicate them. Three of the 2023 repositories had also been indexed since I made that list.

repo:AlexDikelsky/puzzles OR repo:alephno/aoc2021 OR repo:alvinvoo/aoc2021 OR repo:arwn/aoc2021 OR repo:dancek/bqn-advent2021 OR repo:frasiyav/AoC2021 OR repo:m-lima/advent-of-code-2021 OR repo:AugustUnderground/AoC2022 OR repo:Choram/AoC2022BQN OR repo:Nhlest/AoC2022 OR repo:SamuelSarle/advent OR repo:akamayu-ouo/AoC OR repo:bddean/aoc OR repo:dancek/bqn-advent2022 OR repo:dankeyy/aoc22 OR repo:dgkf/advent-of-code OR repo:eissplitter/aoc-2022 OR repo:extorious/aoc2022 OR repo:felixr/advent-of-code OR repo:juriad/advent2022 OR repo:mpizzzle/AdventOfCode OR repo:saltysylvi/bqn-aoc2022 OR repo:RKBethke/aoc23-bqn OR repo:azenna/advent-of-code-2023 OR repo:icendoan/aoc OR repo:qqii/advent-of-code-2023 OR repo:timmarinin/aoc2023 OR repo:dlozeve/aoc2023

I note that file totals frequently don't add up right: the full search of added files gives 286, while this half returns 150 and this half returns 157, totalling 307 (after updating local clones I see 308). And these "286" files were needed to increase BQN's total from 1.8k to 2k, as earlier 255 wasn't enough.

lildude · 2024-01-08T17:24:19Z

Popularity really isn't the priority right now... passing tests is 😉

mlochbaum · 2024-01-08T17:31:59Z

Apologies, I'd just talked to skykanin who believed popularity was the only remaining issue. I'm asking around to see if I can find someone to help.

skykanin · 2024-01-10T18:16:26Z

@lildude could you approve the workflow? I was able to get rid of all the test failures locally 🙂

skykanin · 2024-03-24T21:23:35Z

@lildude Github's search indexing now shows that BQN has 3.4k unique files so I think this PR should be good to merge after the tests are rerun.

add BQN

45015e0

skykanin requested a review from a team as a code owner December 4, 2023 14:29

fix colour

9aa0c5e

lildude requested changes Dec 4, 2023

View reviewed changes

add submodule license

f0625e2

lildude changed the title ~~add BQN language~~ Add BQN language Dec 5, 2023

skykanin added 2 commits December 5, 2023 17:21

add missing fields to languages.yml

6261b2a

update vendor readme

6a88ce8

skykanin requested a review from lildude December 5, 2023 19:57

lildude added the Pending Popularity label Dec 6, 2023

skykanin added 2 commits January 10, 2024 17:19

fix test failures

b5687f6

Merge remote-tracking branch 'upstream/master' into add-bqn

969653c

skykanin and others added 3 commits February 11, 2024 10:26

Merge branch 'master' into add-bqn

3d96d2d

Merge branch 'master' into add-bqn

1ea0ac0

Merge branch 'master' into add-bqn

1c7f126

Merge branch 'master' into add-bqn

b34a1fc

lildude removed the Pending Popularity label Jun 7, 2024

lildude approved these changes Jun 7, 2024

View reviewed changes

lildude added this pull request to the merge queue Jun 7, 2024

Merged via the queue into github-linguist:master with commit c808119 Jun 7, 2024
5 checks passed

skykanin deleted the add-bqn branch June 7, 2024 12:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add BQN language #6623

Add BQN language #6623

skykanin commented Dec 4, 2023 •

edited

Loading

lildude commented Dec 4, 2023

skykanin commented Dec 4, 2023 •

edited

Loading

lildude commented Dec 4, 2023

lildude left a comment •

edited

Loading

skykanin commented Dec 5, 2023

lildude commented Dec 5, 2023

skykanin commented Dec 5, 2023

lildude commented Dec 5, 2023

lildude commented Dec 5, 2023

RocketRace commented Dec 7, 2023 •

edited

Loading

mlochbaum commented Dec 8, 2023 •

edited

Loading

mlochbaum commented Jan 8, 2024 •

edited

Loading

lildude commented Jan 8, 2024

mlochbaum commented Jan 8, 2024

skykanin commented Jan 10, 2024

skykanin commented Mar 24, 2024

Add BQN language #6623

Add BQN language #6623

Conversation

skykanin commented Dec 4, 2023 • edited Loading

Checklist:

lildude commented Dec 4, 2023

skykanin commented Dec 4, 2023 • edited Loading

lildude commented Dec 4, 2023

lildude left a comment • edited Loading

Choose a reason for hiding this comment

skykanin commented Dec 5, 2023

lildude commented Dec 5, 2023

skykanin commented Dec 5, 2023

lildude commented Dec 5, 2023

lildude commented Dec 5, 2023

RocketRace commented Dec 7, 2023 • edited Loading

mlochbaum commented Dec 8, 2023 • edited Loading

mlochbaum commented Jan 8, 2024 • edited Loading

lildude commented Jan 8, 2024

mlochbaum commented Jan 8, 2024

skykanin commented Jan 10, 2024

skykanin commented Mar 24, 2024

skykanin commented Dec 4, 2023 •

edited

Loading

skykanin commented Dec 4, 2023 •

edited

Loading

lildude left a comment •

edited

Loading

RocketRace commented Dec 7, 2023 •

edited

Loading

mlochbaum commented Dec 8, 2023 •

edited

Loading

mlochbaum commented Jan 8, 2024 •

edited

Loading