Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare to similar tools #135

Merged
merged 12 commits into from
Mar 11, 2024
Merged

Compare to similar tools #135

merged 12 commits into from
Mar 11, 2024

Conversation

srid
Copy link
Collaborator

@srid srid commented Mar 7, 2024

Rendered

  • What other tools should we compare with?
  • What other aspects should we mention?

This table is just a start, and not complete.

@srid srid requested a review from shivaraj-bh March 7, 2024 20:41
@srid
Copy link
Collaborator Author

srid commented Mar 7, 2024

cc @roberth for ideas, especially in regards to the aspects. Currently, we have 'pure flakes', 'multiple services', 'sharing' and 'flake apps' (devenv doesn't produce runnable packages, as far as I can tell), etc.

@srid
Copy link
Collaborator Author

srid commented Mar 7, 2024

@shivaraj-bh To substantiate our "share services across flakes", we probably need to document it first.

README.md Outdated
| --- | --- | --- |
| macOS support | ✔️ | ✔️ |
| Pure Flakes | ✔️ | ❌[^1] |
| Share services across flakes | ✔️[^2] | ❌ |
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This maybe inaccurate.

cf. https://devenv.sh/guides/using-with-flake-parts/#import-a-devenv-module

But I haven't used devenv enough to say if its module system works without any idiosyncratic shortcomings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on what I see, if you are able to import a module, you should be able to share services even using devenv.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I tried it out:

Here’s the perSystem block from my flake:

Screenshot 2024-03-08 at 11 43 16 AM

And this is the devenv-module.nix:

Screenshot 2024-03-08 at 11 43 43 AM

I do use nix develop —impure followed by devenv up to run them but in terms of sharing services, it works, i.e if you are willing to stick to devenv cli.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although it allows sharing modules between flakes like above, you can’t really share it within the flake to an extent we do in Nammayatri, to provide nix run #load-test-dev, which adds a bunch of processes of its own after sharing the entire backend configuration of nammayatri that is started via nix run .#run-mobility-stack-dev.

In short, devenv allows you to create and merge processes declaratively but all these processes belong to the same group, you can’t define two sets of different groups with a few/all distinct processes.

Copy link
Collaborator Author

@srid srid Mar 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The screenshots you posted above show modular import of devenv config, not devenv services. ie., we can't do the following, right?

devenv.shells.default = {
  services = {
    imports = [ ./mymod.nix ];
    myservice.enable = true;
  };
};

Which explains why you can't compose and share services per se, and must create modules for whole devenv config.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't do the following, right?

Yes, hope I haven’t missed something. Let me give it one more shot to be certain, don’t want to make false claims because of missing some details.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

@srid srid Mar 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't gotten any clear and unambiguous response from the devenv community, so let's just merge this. If they correct us by providing an example, we can change the README obviously.

README.md Outdated
| macOS support | ✔️ | ✔️ |
| Pure Flakes | ✔️ | ❌[^1] |
| Share services across flakes | ✔️[^2] | ❌ |
| Services as flake apps | ✔️ | ❌[^3] |
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From https://devenv.sh/guides/using-with-flake-parts/

If you're familiar with the Nix language and ecosystem, devenv can be used without the devenv CLI by integrating into Nix Flakes using flake-parts.

But the page doesn't explain how exactly one starts devenv services without using its CLI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the flake I created here, I explored the flake in nix repl, and these are the packages it exports:

nix-repl> packages.x86_64-linux.
packages.x86_64-linux.container-processes  packages.x86_64-linux.default
packages.x86_64-linux.container-shell      packages.x86_64-linux.devenv-up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I can’t seem to be able to nix run it:

❯ nix run .#devenv-up --impure
warning: Git tree '/home/shivaraj/oss/devenc' is dirty
trace: warning: getExe: Package "cowsay-3.7.0" does not have the meta.mainProgram attribute. We'll assume that the main program has the same name for now, but this behavior is deprecated, because it leads to surprising errors when the assumption does not hold. If the package has a main program, please set `meta.mainProgram` in its definition to make this warning go away. Otherwise, if the package does not have a main program, or if you don't control its definition, use getExe' to specify the name to the program, such as lib.getExe' foo "bar".
error: unable to execute '/nix/store/67lf41ig3sf8z4i6lnigfn4c0icyb830-devenv-up/bin/devenv-up': Not a directory

Probably it doesn’t have a bin output

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works fine: ❯ nix build .#devenv-up —impure

And produces a result whose contents are:

❯ cat /nix/store/67lf41ig3sf8z4i6lnigfn4c0icyb830-devenv-up
#!/nix/store/087167dfxal194pm54cmcbbxsfy3cjgn-bash-5.2p26/bin/bash


/nix/store/lb2b16q8kc3xynj27d25sahhq95ph43h-process-compose-0.88.0/bin/process-compose --config /nix/store/vhxxz7j5bjlyi0ggfx4dlcf9an61p09q-process-compose.yaml \
  --port ${PC_HTTP_PORT:-9999} \
  --tui=${PC_TUI_ENABLED:-1} \
  up "$@" &


if [[ ! -d "$DEVENV_STATE" ]]; then
  mkdir -p "$DEVENV_STATE"
fi

stop_up() {
  echo "Stopping processes..."
  kill -TERM $(cat "$DEVENV_STATE/devenv.pid")
  rm "$DEVENV_STATE/devenv.pid"
  wait

  echo "Processes stopped."
}

trap stop_up SIGINT SIGTERM

echo $! > "$DEVENV_STATE/devenv.pid"

wait

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But when I try to run it, postgres service never starts, it just times out and fails with:

❯ ./result
mkdir: cannot create directory ‘’: No such file or directory
./result: line 25: /devenv.pid: Permission denied

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the same configuration works (including postgres starting successfully) when I do nix develop —impure and then devenv up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I run ./result after removing process.implementation = “process-compose”, it uses honcho by default , which seems to work fine.

I was able to execute queries using psql:

shivaraj=# create table id(id int);
CREATE TABLE
shivaraj=# \dt
        List of relations
 Schema | Name | Type  |  Owner
--------+------+-------+----------
 public | id   | table | shivaraj
(1 row)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But when I try to run it, postgres service never starts, it just times out and fails with:

❯ ./result
mkdir: cannot create directory ‘’: No such file or directory
./result: line 25: /devenv.pid: Permission denied

This is because of not exporting DEVENV_STATE, because I am no longer in devShell, I will have to do it manually:

 DEVENV_STATE=$(pwd) ./result

But even after that I couldn’t get it running using the process-compose implementation without being in the devShell

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In conclusion, we can confidently say that devenv doesn't support running services (nevermind individual process groups) as a flake app.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it, yes

@srid
Copy link
Collaborator Author

srid commented Mar 8, 2024

This is what it looks likes right now:

image

Anything that needs changing?

@shivaraj-bh
Copy link
Member

IMO adding an example each for “Share services” and “Reusable services” would be useful.

@shivaraj-bh
Copy link
Member

Right, you had mentioned this earlier: #135 (comment)

That’s something I can work on adding.

@shivaraj-bh
Copy link
Member

@shivaraj-bh To substantiate our "share services across flakes", we probably need to document it first.

Here’s an example: #137

Once we confirm on the example, I will also flesh out the a doc for this.

@srid srid marked this pull request as ready for review March 11, 2024 18:18
@srid
Copy link
Collaborator Author

srid commented Mar 11, 2024

Merging. Let's make follow-up edits in separate PRs.

@srid srid merged commit 705c2cc into main Mar 11, 2024
3 checks passed
@srid srid deleted the compare branch March 11, 2024 18:21
@srid srid mentioned this pull request Mar 11, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants