subdir issue

[ericphanson]

julia> analyze_from_registry(find_package("SnoopCompileAnalysis"))
ERROR: IOError: readdir("/var/folders/jb/plyyfc_d2bz195_0rc0n_zcw0000gp/T/jl_jE3Zv2/9ea4277c-da97-4c3a-afb0-537c066769de/SnoopCompileAnalysis"): no such file or directory (ENOENT)
Stacktrace:
  [1] uv_error
    @ ./libuv.jl:97 [inlined]
  [2] readdir(dir::String; join::Bool, sort::Bool)
    @ Base.Filesystem ./file.jl:830
  [3] readdir
    @ ./file.jl:825 [inlined]
  [4] readfiles
    @ ~/.julia/packages/LicenseCheck/OJxVg/src/find_licenses.jl:24 [inlined]
  [5] find_licenses(dir::String; allow_brute::Bool, max_bytes::Int64)
    @ LicenseCheck ~/.julia/packages/LicenseCheck/OJxVg/src/find_licenses.jl:110
  [6] find_licenses
    @ ~/.julia/packages/LicenseCheck/OJxVg/src/find_licenses.jl:110 [inlined]
  [7] analyze(dir::String; repo::String, reachable::Bool, subdir::String)
    @ AnalyzeRegistry ~/.julia/dev/AnalyzeRegistry/src/AnalyzeRegistry.jl:360
  [8] analyze_from_registry!(root::String, dir::String)
    @ AnalyzeRegistry ~/.julia/dev/AnalyzeRegistry/src/AnalyzeRegistry.jl:229
  [9] #42
    @ ~/.julia/dev/AnalyzeRegistry/src/AnalyzeRegistry.jl:277 [inlined]
 [10] mktempdir(fn::AnalyzeRegistry.var"#42#43"{String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:729
 [11] mktempdir
    @ ./file.jl:727 [inlined]
 [12] analyze_from_registry(p::String)
    @ AnalyzeRegistry ~/.julia/dev/AnalyzeRegistry/src/AnalyzeRegistry.jl:276
 [13] top-level scope
    @ REPL[6]:1

The problem is that the code for the package was removed in timholy/SnoopCompile.jl#127, so AnalyzeRegistry can't find it from cloning the repo (and it's not Pkg.dev'able). It's still Pkg.add'able though, so we could get the code that way if we wanted.

[ericphanson]

the simplest fix is to call isdir(pkgdir) in analyze and if it doesn't exist, bail out instead of calling find_license on it. But theoretically we could Pkg.add it download the code and then analyze that...

If we just check isdir(pkgdir) we get

julia> analyze_from_registry(find_package("SnoopCompileAnalysis"))
Package Invalid Project.toml:
  * repo: https://github.com/timholy/SnoopCompile.jl.git
  * uuid: 00000000-0000-0000-0000-000000000000
  * is reachable: true
  * no license found
  * has documentation: false
  * has tests: false
  * has continuous integration: true
    * GitHub Actions

which isn't ideal since we don't even store what package it was.

[giordano]

I wanted to avoid adding packages if possible, also because not all packages are installable in any given version of Julia

[ericphanson]

Ok, makes sense. Maybe we should just go back to storing the name and UUID from the registry, and then we can have a separate field valid_project field which just checks if the Project.toml is parsable and yields the same name and UUID. And if we don't have the registry info (i.e. analyze(pkgdir(AnalyzeRegistry)) then we fill in "Invalid Project.toml" just the same as now if we can't parse the Project.

That way we can just give up here and at least have the right name of the package.

(edit: though actually we should still at least have the repo and subdir.. so maybe we don't need the name)

[giordano]

But yeah, I was aware that git cloning the latest version of the repository is dumb, as the repository may have changed layout since last released version.

One thing we could look into is how Pkg gets the package. However I don't want to use the resolver for the reason above (we don't care about compatibility, also we don't need the dependencies etc...)

[ericphanson]

Apparently Pkg.add gets the code by asking git for the tree hash that is stored in the registry (not totally sure how that works). So it's pretty robust bc the tree just has to exist somewhere in the repo history.

[ericphanson]

One advantage of cloning the repo is we get access to the toplevel too, which often is where the CI scripts are, and sometimes the license.