Constraint syntax is not expressive enough.

[dcoutts]

Here is a property that we would quite like the solver to have:

given any solver result/solution we should be able to construct a set of solver input constraints that uniquely determine the same result (in any environment in which that solution exists)

Or to put it another way, we should be able to freeze the result of the solver and feed it back into the solver later to get the same result. The solver almost has this property but not quite. In particular the introduction of setup deps and qualified goals means that simple "pkg == version" constraints are no longer expressive enough to describe all the solutions the solver can produce. If the solver picks two versions of package foo (e.g. one a "main" lib package and another as a setup dep) then we cannot just constrain "foo == 1.0", we would need to specify the scope of the constraint. Notice also that "primary" vs "setup" dep scope is not enough, since we could pick two versions of foo both as setup deps. Probably what is required to express all the solutions is something corresponding to the internal goal qualifications.

[dcoutts]

@kosmikus @grayjay @edsko opinions? ideas?

[grayjay]

Does freeze need to handle changes in the set of installed packages?

[dcoutts]

@grayjay I'm not sure I totally understand the question. Want to grab me on IRC in #hackage for a discussion? Or ask again with more details.

The new freeze command (in PR #3503) takes the current solution (having made sure that was up to date) and translates it into a set of constraints and then writes it out. The new build/repl/configure/freeze all will re-run the solver if the solver inputs would change, so this includes changes in the set of installed packages in the global package db (but not in the user or store dbs) and changes in the set of source pakages from hackage. In the new scheme we never tell the solver about installed packages in the store, just source packages and the global installed packages. I'm not sure if that answers the question.

[grayjay]

@dcoutts I was wondering what should happen when a user runs new-freeze, removes a globally installed package, and then runs new-configure again. Should new-freeze just freeze the version numbers, so that cabal can try to rebuild the previously installed package? Or does it need to ensure that the dependencies are exactly the same, including the flag assignments? I joined #hackage, if you want to discuss it there.

[dcoutts]

Ah ok. So if at the point when we ran new-freeze the solution involved an installed package from the global db, (e.g. the copy of containers that is bundled with ghc) then yes we only know the version and not flags, so we can only generate version constraints, not flag constraints. For all packages outside of the global db we do know all the flags and do produce flag constraints with new-freeze.

So you're right to point out that this is a slight hole, however it's not one I'm really worried about. While at the moment we use the whole global package db, our assumption is that these days it only really includes a small set. We can make that assumption properly true by using a "core" package db that really only contains the core packages. Also, of the existing packages typically in the global db, I'm not aware of any with automatic flags that significantly affect things.

[grayjay]

@dcoutts Thanks for explaining. I'm still not very familiar with the overall design for new-build, but I think that using goal qualifiers in freeze constraints makes sense. The only problem that I can think of is that we might add flags to modify goal qualification, such as --independent-goals. It's probably not a big deal, though, since we could just specify the goal qualification scheme in the freeze file.

[kosmikus]

@dcoutts In principle, I agree. But for the use case in question, wouldn't it be even better if we'd have a dedicated new solver (we could call it --explicit-solver or --trivial-solver) that won't solve at all, but simply deserialize an install plan that is passed via some mechanism. The sanity check would still run, of course.

Or are there particular reasons why you want to have this feature in connection with being able to make some modifications, and need the actual solver machinery in the background?

[23Skidoo]

We actually already have something like --trivial-solver in Cabal, it's called --explicit-configuration. Perhaps we only need to port this feature to cabal-install?

[dcoutts]

@grayjay I'm not too worried about things like --independent-goals since we're not going to turn those on by default.

@kosmikus yes, the purpose is the same as the current cabal freeze, to give you a starting point but that you can manually tweak later.

[kosmikus]

@dcoutts Ok, I get it. The only thing that worries me is that I still don't consider the whole setup dependency / qualified goals stuff to be extremely stable. If we allow constraint syntax for these kinds of constraints, we may expose rather fragile solver internals to the end user.

So, syntax proposals welcome.

[ezyang]

I need this feature for other reasons; specifically, when I build cabal-install's test suite from source with cabal-install HEAD, I accidentally pick the in-tree Cabal library to build the custom setup for happy with (which causes it to be built inplace). It would be far better if I could specify a constraint that the custom setup should NOT use the inplace Cabal. Syntax here (along with a way to distinguish between local and remote) would go a long way.

[ezyang]

In #3932 we wanted to add a constraint that all versions of Cabal which are brought in from setup dependencies must be >= 1.20. Since we don't have qualified constraints, the best we can do is just say that Cabal, ANYWHERE, must have >= 1.20. That's suboptimal; a more precise constraint would be better. Once we fix this, we can make the constraint in @dagit's patch more precise.

[grayjay]

A constraint that applies to all packages' setup scripts doesn't correspond to a qualifier in the solver, but it probably wouldn't be much harder to implement.

[grayjay]

@robjhen Are you still interested in working on this issue?

[ghost]

@grayjay Yes I'd be interested but I've been trying with no success to get the latest Haskell Platform and hence the Cabal project to build on my current operating system, Debian 7.

I've uploaded the code from the hackathon: 2d9c460

To install Debian 8 and then complete this feature would likely take me about two months as I have to fit it into limited spare time. I guess that may be too long to wait? Let me know if either a) you would prefer that someone else who has a working build environment take over development or b) you don't mind it taking that amount of time. Sorry about this!

[grayjay]

No problem! I just wanted to make sure that we don't duplicate work. Thanks for working on it. I don't know how soon this feature will be needed, though. @dcoutts, @ezyang Do you know if this is needed for 2.0?

[ezyang]

I don't think it's a blocker for 2.0. Feels important though; I'd put it on the default list (indeed I just have.)

[grayjay]

Thanks. So two months would be soon enough?

[quasicomputational]

What's actually left to do for this issue? Qualified constraints seem to just work now.

v2-freeze is still unaware of that, but there are other issues filed for that.

[quasicomputational]

Ah - executable constraints. foo:bar:exe.baz == 1 will be generated, but not parsed.

[phadej]

If nobody takes ownership (i.e. assigns to themselves) before Thursday 2019-12-05, I'll move this issue to 3.4 milestone.

[phadej]

remilestoning for 3.4

[Ericson2314]

I had to deprecate the latest happy release because of this :(. there is no way to put in a cabal.project file a flags: .... that should apply to qualified goals, like happy used as a build-tool-depends.