Skip to content

Commit

Permalink
WIP systematize builtins.fetchTree docs
Browse files Browse the repository at this point in the history 
The renaming task is splatting everything together into markdown lists.
I think I want to do that with the lowdown AST; I opened
kristapsdz/lowdown#131 for this purpose

PR #9273 should be able to improve upon this a good bit, but I think it
is still a useful stepping stone.
  • Loading branch information
@Ericson2314
Ericson2314 committed Jan 12, 2024
1 parent c3a97e7 commit 10d663b
Show file tree
Hide file tree
Showing 10 changed files with 427 additions and 205 deletions.
125 changes: 0 additions & 125 deletions src/libexpr/primops/fetchTree.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -227,131 +227,6 @@ static RegisterPrimOp primop_fetchTree({
document) if `fetchTree` was a curried call with the first paramter for
`type` or an attribute like `builtins.fetchTree.git`! -->
- `"file"`
Place a plain file into the Nix store.
This is similar to [`builtins.fetchurl`](@docroot@/language/builtins.md#builtins-fetchurl)
- `url` (String, required)
Supported protocols:
- `https`
> **Example**
>
> ```nix
> fetchTree {
> type = "file";
> url = "https://example.com/index.html";
> }
> ```
- `http`
Insecure HTTP transfer for legacy sources.
> **Warning**
>
> HTTP performs no encryption or authentication.
> Use a `narHash` known in advance to ensure the output has expected contents.
- `file`
A file on the local file system.
> **Example**
>
> ```nix
> fetchTree {
> type = "file";
> url = "file:///home/eelco/nix/README.md";
> }
> ```
- `"tarball"`
Download a tar archive and extract it into the Nix store.
This has the same underyling implementation as [`builtins.fetchTarball`](@docroot@/language/builtins.md#builtins-fetchTarball)
- `url` (String, required)
> **Example**
>
> ```nix
> fetchTree {
> type = "tarball";
> url = "https://github.com/NixOS/nixpkgs/tarball/nixpkgs-23.11";
> }
> ```
- `"git"`
Fetch a Git tree and copy it to the Nix store.
This is similar to [`builtins.fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit).
- `url` (String, required)
The URL formats supported are the same as for Git itself.
> **Example**
>
> ```nix
> fetchTree {
> type = "git";
> url = "git@github.com:NixOS/nixpkgs.git";
> }
> ```
> **Note**
>
> If the URL points to a local directory, and no `ref` or `rev` is given, Nix will only consider files added to the Git index, as listed by `git ls-files` but use the *current file contents* of the Git working directory.
- `ref` (String, optional)
A [Git reference](https://git-scm.com/book/en/v2/Git-Internals-Git-References), such as a branch or tag name.
Default: `"HEAD"`
- `rev` (String, optional)
A Git revision; a commit hash.
Default: the tip of `ref`
- `shallow` (Bool, optional)
Make a shallow clone when fetching the Git tree.
Default: `false`
- `submodules` (Bool, optional)
Also fetch submodules if available.
Default: `false`
- `allRefs` (Bool, optional)
If set to `true`, always fetch the entire repository, even if the latest commit is still in the cache.
Otherwise, only the latest commit is fetched if it is not already cached.
Default: `false`
- `lastModified` (Integer, optional)
Unix timestamp of the fetched commit.
If set, pass through the value to the output attribute set.
Otherwise, generated from the fetched Git tree.
- `revCount` (Integer, optional)
Number of revisions in the history of the Git repository before the fetched commit.
If set, pass through the value to the output attribute set.
Otherwise, generated from the fetched Git tree.
The following input types are still subject to change:
- `"path"`
Expand Down
16 changes: 3 additions & 13 deletions src/libfetchers/fetchers.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@

namespace nix::fetchers {

using InputSchemeMap = std::map<std::string_view, std::shared_ptr<InputScheme>>;

std::unique_ptr<InputSchemeMap> inputSchemes = nullptr;

void registerInputScheme(std::shared_ptr<InputScheme> && inputScheme)
Expand All @@ -22,17 +20,9 @@ void registerInputScheme(std::shared_ptr<InputScheme> && inputScheme)
inputSchemes->insert_or_assign(schemeName, std::move(inputScheme));
}

nlohmann::json dumpRegisterInputSchemeInfo() {
using nlohmann::json;

auto res = json::object();

for (auto & [name, scheme] : *inputSchemes) {
auto & r = res[name] = json::object();
r["allowedAttrs"] = scheme->allowedAttrs();
}

return res;
const InputSchemeMap & getAllInputSchemes()
{
return *inputSchemes;
}

Input Input::fromURL(const std::string & url, bool requireTree)
Expand Down
25 changes: 21 additions & 4 deletions src/libfetchers/fetchers.hh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -149,14 +149,26 @@ struct InputScheme
*/
virtual std::string_view schemeName() const = 0;

/**
* Longform description of this scheme, for documentation purposes.
*/
virtual std::string schemeDescription() const = 0;

// TODO remove these defaults
struct AttributeInfo {
const char * type = "String";
bool required = true;
const char * doc = "";
};

/**
* Allowed attributes in an attribute set that is converted to an
* input.
* input, and documentation for each attribute.
*
* `type` is not included from this set, because the `type` field is
* `type` is not included from this map, because the `type` field is
parsed first to choose which scheme; `type` is always required.
*/
virtual StringSet allowedAttrs() const = 0;
virtual std::map<std::string, AttributeInfo> allowedAttrs() const = 0;

virtual ParsedURL toURL(const Input & input) const;

Expand Down Expand Up @@ -193,7 +205,12 @@ struct InputScheme

void registerInputScheme(std::shared_ptr<InputScheme> && fetcher);

nlohmann::json dumpRegisterInputSchemeInfo();
using InputSchemeMap = std::map<std::string_view, std::shared_ptr<InputScheme>>;

/**
* Use this for docs, not for finding a specific scheme
*/
const InputSchemeMap & getAllInputSchemes();

struct PublicKey
{
Expand Down
168 changes: 151 additions & 17 deletions src/libfetchers/git.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -191,25 +191,159 @@ struct GitInputScheme : InputScheme
return "git";
}

StringSet allowedAttrs() const override
std::string schemeDescription() const override
{
return stripIndentation(R"(
Fetch a Git tree and copy it to the Nix store.
This is similar to [`builtins.fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit).
)");
}

std::map<std::string, AttributeInfo> allowedAttrs() const override
{
return {
"url",
"ref",
"rev",
"shallow",
"submodules",
"lastModified",
"revCount",
"narHash",
"allRefs",
"name",
"dirtyRev",
"dirtyShortRev",
"verifyCommit",
"keytype",
"publicKey",
"publicKeys",
{
"url",
{
.type = "String",
.required = true,
.doc = R"(
The URL formats supported are the same as for Git itself.
> **Example**
>
> ```nix
> fetchTree {
> type = "git";
> url = "git@github.com:NixOS/nixpkgs.git";
> }
> ```
> **Note**
>
> If the URL points to a local directory, and no `ref` or `rev` is given, Nix will only consider files added to the Git index, as listed by `git ls-files` but use the *current file contents* of the Git working directory.
)",
},
},
{
"ref",
{
.type = "String",
.required = false,
.doc = R"(
A [Git reference](https://git-scm.com/book/en/v2/Git-Internals-Git-References), such as a branch or tag name.
Default: `"HEAD"`
)",
},
},
{
"rev",
{
.type = "String",
.required = false,
.doc = R"(
A Git revision; a commit hash.
Default: the tip of `ref`
)",
},
},
{
"shallow",
{
.type = "Bool",
.required = false,
.doc = R"(
Make a shallow clone when fetching the Git tree.
Default: `false`
)",
},
},
{
"submodules",
{
.type = "Bool",
.required = false,
.doc = R"(
Also fetch submodules if available.
Default: `false`
)",
},
},
{
"lastModified",
{
.type = "integer",
.required = false,
.doc = R"(
Unix timestamp of the fetched commit.
If set, pass through the value to the output attribute set.
Otherwise, generated from the fetched Git tree.
)",
},
},
{
"revCount",
{
.type = "integer",
.required = false,
.doc = R"(
Number of revisions in the history of the Git repository before the fetched commit.
If set, pass through the value to the output attribute set.
Otherwise, generated from the fetched Git tree.
)",
},
},
{
"narHash",
{},
},
{
"allRefs",
{
.type = "Bool",
.required = false,
.doc = R"(
If set to `true`, always fetch the entire repository, even if the latest commit is still in the cache.
Otherwise, only the latest commit is fetched if it is not already cached.
Default: `false`
)",
},
},
{
"name",
{},
},
{
"dirtyRev",
{},
},
{
"dirtyShortRev",
{},
},
{
"verifyCommit",
{},
},
{
"keytype",
{},
},
{
"publicKey",
{},
},
{
"publicKeys",
{},
},
};
}

Expand Down
Loading

0 comments on commit 10d663b

Please sign in to comment.