Guidance on versioning implementation 1 #362

neilsjefferies · 2019-06-04T14:30:35Z

This should work regardless of where the temp files are located. There needs to be some temp files when we write to the object inventory anyway. Fixes #348

rosy1280 · 2019-06-05T13:06:15Z

draft/implementation-notes/index.html

+          <li>  
+          Temporary Version Directories: New versions should be created in a temporary version directory.
+          This could reside in the OCFL Storage Root workspace directory ("deposit"), the same location as
+          other version directories in an OCFL Object or elsewhere. In any case, only one such directory 


what the does "the same location as other version directories in an OCFL Object or elsewhere" mean.

I'm not saying where the workspace is....just that there has to be one and the contents are predictable and therefore identifiable.

that i would change the sentence to say this, because right now i don't really know what that sentence says.

I seems really hard to discuss a semi-blessed deposit directory inside objects in a PR. May I suggest creating a new issue (that points back to #320) that proposes guidance on this issue in the implementation notes while not changing the normative specification?

Note also should use code font <code>deposit</code> instead of quoting literals

...this is guidance in the implementation notes

Sorry for not being clear: my point is that it would be good to merge other parts of this PR but I don't think we can introduce a new in-object deposit directory without more discussion on a separate issue

Limited discussion to "deposit" directory in Storage Root.

rosy1280 · 2019-06-05T13:07:26Z

draft/implementation-notes/index.html

+          Temporary Version Directories: New versions should be created in a temporary version directory.
+          This could reside in the OCFL Storage Root workspace directory ("deposit"), the same location as
+          other version directories in an OCFL Object or elsewhere. In any case, only one such directory 
+          can exist per OCFL Object at any time and updates to it should be managed by a single 


only one deposit directory can exist per object? only one temporary version directory can exist per object? this isn't clear.

You're right. The temporary version directory is called "deposit..."

Hoopefully clarified.

rosy1280 · 2019-06-05T13:09:21Z

draft/implementation-notes/index.html

+          other version directories in an OCFL Object or elsewhere. In any case, only one such directory 
+          can exist per OCFL Object at any time and updates to it should be managed by a single 
+          controlling process to avoid conflicts. It is advised that object updates should be given
+          a unique transaction identifier to simplify any overlying process control logic. For 


This is way to into the weeds on how to architect a repository.

Bingo! I refer you to Fedora 4 and DPN - apparently this is something some people find rather hard. This is just a guideline on how not to screw it up too badly.

i'm suggesting that you remove this from the language because it shouldn't be on us to tell folks how to architect their repository.

rosy1280 · 2019-06-05T13:14:11Z

draft/implementation-notes/index.html

+          example, the identifier could be used as an an eTag for a REST API implementation. A 
+          good temporary directory name should be "deposit_transaction ID_OCFL Object ID". 
+          If there is guaranteed to only be a single process controlling updates to the contents 
+          of an OCFL storage root, then it is permissible to use "deposit_OCFL Object ID" as a 


If I have a top level deposit directory, then I don't need to name a directory this because I know everything in that directory is for deposit into the storage root.

There might be multiple transactions going on in that directory so they need separate distinguishable subdirectories. Embedding a transaction ID also gives protection against multiple writes to the same object by different processes.

that's one way to do it, but another is to just not let people write to the same object while another version is being deposited. see comment above about this being way too into the weeds and we shouldn't be telling folks what to do.

That's the point - the only way to do any form of locking is to stick a flag somewhere. Since OCFL aims to do away with non-file-based data, this is an OCFL-ish way to do it. This is guidance, as stated above, not the spec so it's just a suggestion.

rosy1280 · 2019-06-05T16:34:15Z

draft/implementation-notes/index.html

+          If there is guaranteed to only be a single process controlling updates to the contents 
+          of an OCFL storage root, then it is permissible to use "deposit_OCFL Object ID" as a 
+          temporary version directory name. If temporary directories are stored within OCFL 
+          Objects then the OCFL Object ID part may be omitted since it is implicit.


so you're saying that there would be just a V2 directory? how would you know that a version hasn't been completed in the directory?

No, but in any case, version completeness can be determined from the inventory.

rosy1280 · 2019-06-05T16:39:12Z

@neilsjefferies I haven't finished reviewing this, but I would suggest that we re-visit the previous conversations we had around deposit directories and identify what this section's purpose is because right now this doesn't necessarily seem like implementation guidance.

zimeon · 2019-06-05T16:51:15Z

draft/implementation-notes/index.html

+      </p>
+
+      <section>
+        <h2>Temporary Names</h2>


"Temporary Names" sounds odd to me, should this be "Temporary Files and Directories"?

Agreed...done

zimeon · 2019-06-05T16:54:45Z

draft/implementation-notes/index.html

+          example, the identifier could be used as an an eTag for a REST API implementation. A 
+          good temporary directory name should be "deposit_transaction ID_OCFL Object ID". 
+          If there is guaranteed to only be a single process controlling updates to the contents 
+          of an OCFL storage root, then it is permissible to use "deposit_OCFL Object ID" as a 


I wouldn't recommend directory names with spaces but maybe that is just old-school of me.

I think we also have to be careful around language like permissible as this is non-normative, the issue here is that with just a single process it is adequate without the transaction id.

Yes, shouldn't be spaces - will fix that. How often will we ever have a single process controlling access since it is likely that any overlying API will be RESTful.

This is in the implementation notes so not part of the spec and therefore shouldn't it all be non-normative. Adequate is adequate however!

Mostly scrapped.

zimeon · 2019-06-05T16:56:31Z

draft/implementation-notes/index.html

+      does not specify in detail update and file locking mechanisms since these are implementation
+      dependent features. Nevertheless, this section includes some basic guidance on how to 
+      update OCFL objects in a manner that tries to ensure that updates are limited to a single  
+      transaction and that failures are detectable and recoverable.


Do we need to point out that advice here will result in objects that cannot be validated during update, validation should be performed only when updates are complete?

zimeon · 2019-06-05T16:58:08Z

draft/implementation-notes/index.html

+            Update object as described earlier.
+            </li>
+            <li>
+            Ideally, update the local inventory after each file change operation, not at the end


Not sure whether this is ideal. If one thinks of an atomic update of many files then it is just a waste of time

Agreed. Advantages both ways depending on use case. Scrapped as we don;t really need to say anything about it.

zimeon · 2019-06-17T15:45:17Z

draft/implementation-notes/index.html

+          <li>  
+          Temporary Version Directories: New versions should be created in a temporary version directory.
+          This could reside in the OCFL Storage Root workspace directory ("deposit"), the same location as
+          other version directories in an OCFL Object or elsewhere. In any case, only one such directory 


Sorry for not being clear: my point is that it would be good to merge other parts of this PR but I don't think we can introduce a new in-object deposit directory without more discussion on a separate issue

neilsjefferies added 3 commits June 4, 2019 15:12

Update index.html

9502667

Update index.html

c0d8fdf

Update index.html

9c5cefb

zimeon requested review from ahankinson, zimeon, awoods, julianmorley and rosy1280 June 4, 2019 14:32

rosy1280 reviewed Jun 5, 2019

View reviewed changes

zimeon reviewed Jun 5, 2019

View reviewed changes

Update index.html

0bb6b6a

julianmorley previously approved these changes Aug 20, 2019

View reviewed changes

Update .travis.yml

3aacc61

neilsjefferies dismissed julianmorley’s stale review via 3aacc61 August 20, 2019 13:33

zimeon approved these changes Aug 20, 2019

View reviewed changes

rosy1280 approved these changes Aug 20, 2019

View reviewed changes

julianmorley approved these changes Aug 20, 2019

View reviewed changes

ahankinson approved these changes Aug 27, 2019

View reviewed changes

rosy1280 merged commit 20e792a into master Aug 27, 2019

rosy1280 deleted the Guidance-on-versioning-implementation-1 branch October 3, 2019 21:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Guidance on versioning implementation 1 #362

Guidance on versioning implementation 1 #362

neilsjefferies commented Jun 4, 2019 •

edited

Loading

rosy1280 Jun 5, 2019

neilsjefferies Jun 5, 2019

rosy1280 Jun 5, 2019

zimeon Jun 5, 2019

neilsjefferies Jun 17, 2019

zimeon Jun 17, 2019

zimeon Jun 17, 2019

neilsjefferies Aug 20, 2019

rosy1280 Jun 5, 2019

neilsjefferies Jun 5, 2019

neilsjefferies Aug 20, 2019

rosy1280 Jun 5, 2019

neilsjefferies Jun 5, 2019

rosy1280 Jun 5, 2019

neilsjefferies Aug 20, 2019

rosy1280 Jun 5, 2019

neilsjefferies Jun 5, 2019 •

edited

Loading

rosy1280 Jun 5, 2019

neilsjefferies Jun 17, 2019

neilsjefferies Aug 20, 2019

rosy1280 Jun 5, 2019

neilsjefferies Jun 17, 2019

neilsjefferies Aug 20, 2019

rosy1280 commented Jun 5, 2019

zimeon Jun 5, 2019

neilsjefferies Aug 20, 2019

zimeon Jun 5, 2019

neilsjefferies Jun 17, 2019

neilsjefferies Aug 20, 2019

zimeon Jun 5, 2019

neilsjefferies Aug 20, 2019

zimeon Jun 5, 2019

neilsjefferies Aug 20, 2019

zimeon Jun 17, 2019

Guidance on versioning implementation 1 #362

Guidance on versioning implementation 1 #362

Conversation

neilsjefferies commented Jun 4, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neilsjefferies Jun 5, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rosy1280 commented Jun 5, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neilsjefferies commented Jun 4, 2019 •

edited

Loading

neilsjefferies Jun 5, 2019 •

edited

Loading