Extract hs-ucan from fission-core #571

matheus23 · 2022-01-18T14:07:57Z

This is work on #561

To do this, some dependencies on fission-specific UCAN stuff had to be broken up. The main point here being the definition of Fact and the resource type Scope Resource. (In hindsight I think Scope actually still belongs to hs-ucan, but now I've already done it this way, and we'll have to change these parts of the code when supporting the new UCAN version anyway.)

That means that we can't directly run JWT.check and know how to check delegation validity in UCANs. So I've abstracted that logic out into a ResourceSemantics class:

class ResourceSemantics rsc where
  canDelegate :: rsc -> rsc -> Bool

(Which is structurally equivalent to something like a PartialOrder typeclass, but I like giving it a dedicated name.)

There are still some parts that need work.

I need to still get the tests compiling (and then working?)
I don't 100% like how the fission-core and hs-ucan packages separated. E.g. I don't think Error.AlreadyExists should be a hs-ucan module.
Generally I'll need to look through hs-ucan's .Internal modules again, making sure that we really only have modules in there that make sense
~~I think I want to introduce a Web.JWT module, which reexports Web.JWT.Types.~~ My current thinking is: Let's introduce that file once we have something to put into it.
I don't like how fission-core's newtype around the hs-ucan JWT works. Either both newtype JWT and newtype Proof or both are type aliases. But if I didn't type-alias Proof, I'd be taking non-newtyped proofs out of JWTs and just re-newtype them. I'm thinking that this might get obsolete when I start working on hs-ucan anyway. I have some recursion-schemes thoughts.
I really don't like the module name Web.JWT.* for the hs-ucan stuff. I'd love some input. Maybe Web.Ucan.*?
Oh yeah and I need to figure out a better way to do the Proof.Resolver class, which now has 3 type parameters instead of one.

And a final TODO item unrelated to hs-ucan or fission-core:

Get all other packages & tests compiling again

This reverts commit b6504d9.

matheus23 · 2022-01-19T17:28:24Z

Don't you like it when your TODO list grows while you're chasing TODO items? :P

Re-rename everything to use uppercase UCAN instead of Ucan
Abstract over Web.Ucan.Potency
Rename ResourceSemantics to DelegationSemantics
Split out tests in fission-core (abstract over fission-specific Facts, Resources, Potencys) & move them to hs-ucan

hs-ucan/LICENSE

fission-web-server/library/Fission/Web/Server/User/Destroyer/Class.hs

fission-web-server/library/Fission/Web/Server/Types.hs

fission-web-server/library/Fission/Web/Server/Handler/User/Verify.hs

expede · 2022-01-21T06:39:37Z

fission-web-server/library/Fission/Web/Server/Authorization/Types.hs

@@ -27,7 +27,7 @@ data Heroku = Heroku
 data Authorization = Authorization
  { sender   :: Either Heroku DID
  , about    :: Entity User
-  , potency  :: Potency
+  , potency  :: Maybe Potency -- Nothing indicates "just used for signature checking"


Rather than dealing with the Maybe for all possible call sites in the future, could we consider either abstracting over Potency, or parsing that case to a different data type? It's not usable if only checking the signature, no? This type is specialized to look at the database with User, so it's not a general use thing. Let's make error cases as unrepresentable as possible, if possible!

Let's make error cases as unrepresentable as possible, if possible!
The new Potency type lacks a variant the old one had before: AuthNOnly. So basically Old.Potency is isomorphic to Maybe New.Potency. So I don't think there should be any changes in terms of what's representable. I basically moved the comment for Nothing over from the definition of the old Potency type.

In any way, as soon as hs-ucan is updated to UCAN 0.8 and I'm incorporating that into the fission stuff, this will change.

fission-core/test/Fission/Test/Web/Auth/Token/UCAN/Resource.hs

fission-core/test/Fission/Test/Web/Auth/Token/Bearer.hs

hs-ucan/test/Test/Web/UCAN/Example.hs

matheus23 · 2022-01-21T10:14:42Z

fission-cli/library/Fission/CLI/Types.hs

-        case eitherDecodeStrict resolvedBS of
-          Left  _   -> Left $ InvalidJWT resolvedBS
-          Right jwt -> Right (JWT.contentOf (decodeUtf8Lenient resolvedBS), jwt)
+          Right (UCAN.contentOf (decodeUtf8Lenient resolvedBS))


Here you can see an effect of one of the changes.
I've changed the UCAN.Resolver class to only resolve to UCAN.RawContent, instead of also decoding the JWT.

Hmm I have some vague memories of also going down this path (several times?) and then putting the JWT back in 🤔 I can't remember what the scenario was — something about needing the decoded case in various code paths? — so not going to say to not do this, but just mentioning it

Okay. I just now realized that RawContent isn't the whole JWT encoded. It's only the part without the signature. That's what I get for not reading its docs!
I'll fix this in another PR. I'm thinking that the easiest way is to return either ByteString from resolve, or some kind of newtype?

matheus23 · 2022-01-21T10:16:47Z

fission-core/library/Fission/Internal/UTF8.hs

-  , toBase58Text
  , fromRawBytes
-  , stripOptionalPrefix
-  , stripOptionalPrefixBS
-  , stripOptionalPrefixLazyBS
-  , stripOptionalSuffix
-  , stripOptionalSuffixBS
-  , stripOptionalSuffixLazyBS
-  , stripPadding
-  , stripQuotes
-  , stripQuotesBS
-  , stripQuotesLazyBS
-  , stripN
-  , stripNBS
-  , stripNewline
+  , toBase58Text
  , textShow
  , wrapIn
+  , module Web.UCAN.Internal.UTF8


I wasn't quite sure what to do with the .Internal stuff that we've been depending on in fission-core.

So what I did was split it up, you can see me removing stuff from the fission-core .Internal.UTF8 module here, and instead reexporting the Web.UCAN.Internal.UTF8 module, where that stuff was moved to.

Hmm good point! A bunch of this stuff is actually now available in other libraries. We could switch to those, or break out a fission-utils or something 🤔

matheus23 · 2022-01-21T10:19:15Z

fission-core/library/Fission/Web/Auth/Token/UCAN/Potency/Types.hs

@@ -12,30 +16,30 @@ append-only rights, so

 -- | How much power allowed in an authorization
 data Potency
-  = AuthNOnly   -- ^ Read signature only -- just a proof. Cannot delegate further.


I removed the AuthNOnly constructor here, in favor of using Maybe Potency instead of Potency everywhere.
The reason is that in hs-ucan we don't want to depend on Potency (because it's also fission-specific), but we need to decode to some potency in case the ptc key is missing completely.

Yeah, for this version of UCAN totally.

Thinking out loud: as we bring things to up date, I imagine that we'll have something like...

data Capabilities custom = AuthNOnly -- None | UserDefined [custom] -- Some | AllByParenthood DID -- All (e.g. Linking)

...but obviously we need to actually solidify that in the v0.8 spec before we do that.

matheus23 · 2022-01-21T10:20:50Z

fission-core/library/Fission/Web/Auth/Token/UCAN/Resource/Types.hs

+  canDelegate rsc rscProof =
+    case (rsc, rscProof) of
+      (FissionFileSystem path, FissionFileSystem proofPath) ->
+        case (isPrivate path, isPrivate proofPath, pathSubset path proofPath) of
+          (True, True, _) ->
+            bitwiseContains proofPath path
+
+          (_, _, True) ->
+            True
+
+          _ ->
+            False
+
+      (RegisteredDomain _, RegisteredDomain Complete) ->
+        True
+
+      (FissionApp _, FissionApp Complete) ->
+        True
+
+      _ ->
+        rsc == rscProof


This is code adapted from a previous function Fission.Web.Auth.Token.JWT.Proof.WNFS.compareSubsets

matheus23 · 2022-01-21T10:29:11Z

hs-ucan/library/Web/UCAN/Resolver/Class.hs

+import           Web.UCAN.Resolver.Error as Resolver
+
+class Monad m => Resolver m where
+  resolve :: CID -> m (Either Resolver.Error UCAN.RawContent)


Here's what I've been talking about (above?): The Resolver class now returns
Either Resolver.Error UCAN.RawContent instead of
Either Resolver.Error (UCAN.RawContent, UCAN.UCAN fct ptc rsc),
which just makes it a little simpler by getting rid of some dependencies on what fct, ptc and rsc are.

matheus23 · 2022-01-25T13:25:33Z

I've changed the base branch to ucan-upgrade as per discussion with @expede: We'll keep that as another branch running alongside main while we're upgrading hs-ucan, so we can merge partially-typechecking PRs while stuff is in flux.
We keep main clean so we can hotfix stuff.
Eventually ucan-upgrade will get merged into main.

expede · 2022-01-26T03:28:50Z

fission-core/library/Fission/Web/Auth/Token/UCAN/Resource/Scope/Types.hs

+instance DelegationSemantics rsc => DelegationSemantics (Scope rsc) where
+  Complete          `canDelegate` _            = True
+  (Subset _)        `canDelegate` Complete     = False
+  (Subset rscProof) `canDelegate` (Subset rsc) = rscProof `canDelegate` rsc


expede

@matheus23 this looks great 🎉

It's a lot of files, but the changes are extremely straightforward 👍

* Add ipfs in hie.yaml * Move fission-core tests into fission-core * Move Fission.Test.Web.Server.Auth to Fission.Test.Web.Auth * Move some modules around * Move SemVer and Potency over * Move key stuff * Move Internal & Orphanage stuff * Format files * Remove remaining dependencies * Remove hie.yaml * Move over hs-ucan stuff * Remove most Fission.Prelude imports from hs-ucan * Abstract out facts & resource types in JWT * Format files * Make hs-ucan and fission-core compile * Make fission-core compile & run * Rename a test function * Move AlreadyExists back to fission-core * Format files & don't duplicate UTF8 helpers * Also re-export symmetric in Fission/Key * Use type alias instead of newtype * Move things from Web.JWT or Ucan.Internal to Web.Ucan * Rename JWT stuff into Ucan * Make Resolver use only a single type param * Format files * Revert "Format files" This reverts commit b6504d9. * Format files * Get everything compiling except for fission-web-server * Make everything COMPILE AGAIN 🙌 * Use the same uppercase UCAN everywhere * Upgrade cachix/install-nix-action@v16 * Move Potency type to fission-core (& make it Maybe) * Add a test for Potency * Make everything compile (& format) * Rename `ResourceSemantics` to `DelegationSemantics` * Once and forall fix this broken test * Initialise test suite in hs-ucan * Write partial order property tests for canDelegate * Copy over some tests and write some new * Fix signature * Don't use Flow * Fix imports * Switch LICENSE to Apache 2 * Use >>= instead of <&> * Remove unneccesary AllowAmbigousTypes * We need more $$$ * Even more $$$

matheus23 added 15 commits January 14, 2022 12:48

Add ipfs in hie.yaml

0bef3a1

Move fission-core tests into fission-core

e29ca04

Move Fission.Test.Web.Server.Auth to Fission.Test.Web.Auth

b2ee3c4

Move some modules around

f0b735e

Move SemVer and Potency over

b4fe41b

Move key stuff

2c92427

Move Internal & Orphanage stuff

48a5ebb

Format files

c7d828f

Remove remaining dependencies

dcabdd0

Remove hie.yaml

3fe0aec

Move over hs-ucan stuff

0de975b

Remove most Fission.Prelude imports from hs-ucan

2b32db2

Abstract out facts & resource types in JWT

515432a

Format files

edbaccc

Make hs-ucan and fission-core compile

b464fa9

matheus23 self-assigned this Jan 18, 2022

matheus23 added 13 commits January 18, 2022 16:11

Make fission-core compile & run

d581269

Rename a test function

9f0ee5c

Move AlreadyExists back to fission-core

2ee53e0

Format files & don't duplicate UTF8 helpers

d741011

Also re-export symmetric in Fission/Key

c2b628a

Use type alias instead of newtype

0637629

Move things from Web.JWT or Ucan.Internal to Web.Ucan

006007a

Rename JWT stuff into Ucan

9fec132

Make Resolver use only a single type param

794acf2

Format files

b6504d9

Revert "Format files"

5950887

This reverts commit b6504d9.

Format files

9d06452

Get everything compiling except for fission-web-server

e88de95

matheus23 added 2 commits January 20, 2022 21:14

Don't use Flow

54cda1c

Fix imports

59a0e36