Skip to content

Commit

Permalink
perf(lockfiles): Eagerly parse berry resolutions (#5417)
Browse files Browse the repository at this point in the history
### Description

Each time we were resolving a dependency in the berry lockfile we were
creating an `Identifier` by parsing the contents of each resolution
entry. This resulted in us parsing these strings `|resolutions| *
|lockfile entries|` times causing a noticeable slowdown if either number
get large. To avoid this we eagerly parse the resolution entries upon
construction. This eagerness will be a speed up in all cases except for
if we parse the lockfile, but never use it.

I also removed some allocations that could be avoided in
`reduce_dependency` since it gets called as much as it does.

### Testing Instructions

Added an example with a large enough resolution list and lockfile to
incur poor performance.

Inspect the profile from running the example before and after the
change: `cargo instrument --example berry_resolutions`. (Requires
installation of
[cargo-instruments](https://crates.io/crates/cargo-instruments) along
with XCode dev tools)

Before: Overall execution of the example took 341ms and
`Resolution::reduce_dependency` was 245ms of that
After: Overall execution of the example took 62ms and
`Resolution::reduce_dependency` was 3ms of that

---------

Co-authored-by: Chris Olszewski <Chris Olszewski>
  • Loading branch information
chris-olszewski committed Jun 28, 2023
1 parent fb823ac commit 7252924
Show file tree
Hide file tree
Showing 3 changed files with 4,485 additions and 73 deletions.
33 changes: 33 additions & 0 deletions crates/turborepo-lockfiles/examples/berry_resolutions.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
use turborepo_lockfiles::{BerryLockfile, BerryManifest, Lockfile, LockfileData};

fn main() {
let manifest = generate_manifest("foobar", 100);
let lockfile_bytes = include_bytes!("yarn.lock");
let data = LockfileData::from_bytes(lockfile_bytes.as_slice()).unwrap();
let lockfile = BerryLockfile::new(&data, Some(&manifest)).unwrap();
let key = "debug@npm:3.2.7";
println!(
"Dependencies of {key}: {}",
lockfile
.all_dependencies(key)
.unwrap()
.unwrap()
.into_iter()
.map(|(k, v)| format!("{k}@{v}"))
.collect::<Vec<_>>()
.join(", ")
);
}

fn generate_manifest(prefix: &str, size: usize) -> BerryManifest {
let mut count = 0;
BerryManifest::with_resolutions(std::iter::from_fn(move || {
let cont = count < size;
count += 1;
if cont {
Some((format!("{prefix}{count}"), "1.0.0".to_string()))
} else {
None
}
}))
}
Loading

0 comments on commit 7252924

Please sign in to comment.