Exclude old releases from GitHub's search/jump-to-definition functionality

[cincodenada]

Is your feature request related to a problem? Please describe.
Yes - trying to find and follow code in this repository using the GitHub interface is very difficult. If I try to search for something, search results are obscured by the same code block duplicated across dozens of old releases, which makes them less useful to me - I generally want to find code in the most recent state of the project, and in any case that's what I expect.

For context: my use case is that I very often use GitHub as the first place to dig into a dependency's source code to understand what's going on before submitting a PR or try to fix a bug or contribute something. I could dig through node_modules but maybe my issue has been fixed in a more recent version, so I want to look at the most recent code, and GitHub is a convenient way to do so without having to clone the project locally just to read a couple files. For small changes, I often even edit and submit a PR entirely through GitHub.

The docs folder makes this especially frustrating when using GitHub's "jump to definition" functionality, which is very useful for tracing through code to understand it. When I click on a function in this repo, I am presented with dozens, even hundreds of "definitions", one for each version in the docs directory, which makes this very useful feature basically unusable.

This is less of a concern, but it's also a lot of space to take up to just to check out and contribute to the repo. the docs/_releases folder is 14MB and the docs/releases folder is 256MB - the whole repo (.git directory included) is 336MB, so that's a lot of weight to be carrying around that very few people need. This is not a huge concern because it's all in the .git directory anyway (albeit highly compressed), so it doesn't affect download time, and space is "cheap", my node_modules is 700MB...but 1/3 of node_modules is still a lot!

Describe the solution you'd like
I realize this is a relatively small problem with no easy solution I'm aware of, and I don't know enough of what the infrastructure looks like for maintaining the docs site and such to offer useful suggestions - it seems that it's built out of this repository, is all I could tell so far.

Anything that prevents the docs directory from showing up in search results or the jump-to feature I'm happy with, but I couldn't find any mechanism to exclude a directory from those features without excluding them from the repo altogether somehow, I searched but didn't find any github-specific exclusion file or anything. It's hard to search for because you just get .gitignore over and over, but I did some pretty specific searching and didn't find anything.

One suggestion would be to move historical releases to a branch that's maintained in parallel, where part of the release process is to merge the release branch into the website branch (or whatever), then copy the releases folder into the archival folder as usual, and then use that branch for deploying the site itself. That way, the archival releases are segmented away from the default branch (which I believe should resolve the problem), but are still easily accessible for anyone who wants them, and hopefully easily accessible for whatever site deployment setup there is. It's a small additional lift to the release process, but since this is the only thing the branch would be used for and merges are always one-way, there wouldn't be any conflicts to deal with. The initial setup would be a little annoying, deleting and re-introducing everything, but that's a one-time lift and I'd be happy to do it.

Obviously moving the site deployment/docs to a totally separate repo would also solve the problem, but that's a much bigger lift. And without knowing more about what the deployment process is like, I don't know how big a lift either would be, and I'm not sure how I would go about making a PR for this if it is feasible.

I'm interested in trying to assist here, but I'm not sure where I would even start, because of how it intersects with the site deploy process and additional branches and all.

Describe alternatives you've considered
For code search, a partial solution is to search within a path using advanced search parameters - this works okay, but requires users to know about the advanced search parameters, and does nothing for the code-jumping features.

Additional context
A screenshot of trying to look up a method's definition from GitHub, resulting in 101 definitions of the method, making this usually very useful feature significantly less useful:

A screenshot of a search for "calledWithExactly", which results in 7 pages of javascript file results alone, most of which are just old releases:

A screenshot showing a similar search for calledWithExactly on my computer, showing the vast majority of results in docs (within docs, it's approx 2:1 in _releases vs releases):

[fatso83]

Ooh, good issue. We have been talking about a new homepage for the last three years without things really progressing on this matter, but this seems like a obvious candidate that should attack. None of us are very content with the current homepage with regards to all the versions (one per major should suffice).

The deploy process is quite simple, so moving the produced releases into another branch should absolutely be doable.

[cincodenada]

Great, thanks for the pointer to the release process! It is indeed pretty simple. I've been working on a proposal here and have a PR just about ready, but have a question about how the website is deployed.

There are files that are required for the docs website to function (release-source/release/examples/node_modules, assets/js/sinon.js) that aren't (and shouldn't be) committed to Git - how does the website deploy end up with those files in its directory tree? Is the whole directory tree copied over after running npm version or is there some sort of other doc build process?

I thought it was build-docs but I wasn't able to get those commands working - for one, the script was renamed to .cjs but the script command wasn't updated and the script assumes your node is called nodejs, which mine isn't. But after fixing that and trying to run bundler to install jekyll, it complains about a dependency it can't find.

I ask because in the setup with a separate release branch, I'm checking out the release branch, doing the release things, and then switching back to master - but currently, this results in the above files being left over as untracked files. I've got it set up to clean them up, but I'm not sure if that will break whatever process is involved in actually serving the docs.

[cincodenada]

Okay, put up the PR for this - depending on the answers to my above comment, it may need some adjusting, but I think the general approach is reasonable.

[fatso83]

The node_modules folder comes from running tests on the example codes. It is this NPM script that does it:

  "test-runnable-examples": "docs/release-source/release/examples/run-test.sh"

That in itself is a super simple bash script:

$ cat    docs/release-source/release/examples/run-test.sh
#!/bin/bash
# Link 'sinon' to local development dir
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
SINON_ROOT="$SCRIPT_DIR/../../../.."

cd "$SINON_ROOT"
npm link

# Install examples project and link to local sinon folder
cd "$SCRIPT_DIR"
rm -r node_modules 2>/dev/null
npm install --ignore-scripts
npm link sinon

# Lint
$(npm bin)/eslint .

# Make sure all examples are still runnable
set -e
for f in *.test.js; do
    node $f
done
set +e

# clean up to avoid circular links confusing watchers
npm unlink sinon
git checkout -- package.json
npm install --ignore-scripts
cd "$SCRIPT_DIR/.."
npm unlink .

it should probably have a final step that just does rm -r node_modules.

[cincodenada]

Okay, great - that makes sense. I originally thought the examples were runnable on the sinon docs page, and was confused about how the node_modules made it to the docs server!

But now I think I understand - the node_modules are only installed to run the examples as part of the release process to make sure they actually run (which is great!), but they aren't interactively runnable on the docs site once it's deployed, so the node_modules aren't needed after the tests are run. Thanks!

[fatso83]

They are actually interactively runnable on the docs site, which is why I took extra interest in making sure they actually do run 🙂 They run inside a Runkit interactive iframe that has a Node environment, but that prevents something like Mocha from running. It all has to be plain, single Node scripts. So the local tests just require the same modules as Runkit will.

[cincodenada]

Oh, ha! I just hadn't waited long enough for the RunKit environment to load, I see the runnable bits now. So, to see if I'm tracking now: the runnable examples do run, but the docs site couldn't use a node_modules folder if it did exist - on the website, their deps are supplied separately via the Runkit setup?

Either way the end result is the same, the site doesn't need the node_modules around, but I appreciate you explaining the difference!

Edit: Ah, and now actually looking into the package.json, I see the latest deps and the note about RunKit there. It's all making sense now!