Community requirements discussion: how do we make this repo more sustainable?

[ross-spencer]

The first three public commits to this repo have all being incompatible with the anticipated workflow which was to edit a CSV (as a low barrier of entry) and then have a script do the following:

• extract the csv into two components, front-matter and csv,
• use the csv contents to create an alphabetized table of contents,
• output a markdown file in three formatted sections, front-matter, contents, and data.

But the scripting is brittle as I describe here: #9 (comment) and we have seen in the other commits.

What are the actual community needs here?

• Simple to update?
• Is CSV easier than Markdown?
• Something else?

Please leave comments below.

[ross-spencer]

NB. if we adopted a markdown first approach, we could still use a github workflow to make some aspects of editing this work easier, e.g. a workflow to create the table of contents and insert it automatically. It really depends on what people require.

[kieranjol]

I think working directly on the markdown would be preferable. If memory serves, the three contributions (mine included) all made commits on the markdown. Could it be markdown first, which then generates the CSV?

…

On Fri 29 Mar 2024 at 12:14, Ross Spencer ***@***.***> wrote: NB. if we adopted a markdown first approach, we could still use a github workflow to make some aspects of editing this work easier, e.g. a workflow to create the table of contents and insert it automatically. It really depends on what people require. — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAITFPUXV2RFD5IQVKR5M7DY2VEIPAVCNFSM6AAAAABFOK7466VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRXGE3DQNJQG4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ross-spencer]

Could it be markdown first, which then generates the CSV?

That's a really nice idea!

[anjackson]

I'm interested in helping with this, ideally using a consistent approach across the various resources under digipres.org.

My current experimentation is with DecapCMS which provides a UI but simply stores all the information as e.g. Markdown with YAML frontmatter (but not CSV, see here. This could still generate the website (https://www.digipres.org/policies/) and the repo README could perhaps direct people there rather than also containing a copy of the same information (which I think is causing some of the confusion).

But there are various fairly low-tech options that could still be used to generate a website/index:

• Use plain Markdown, and rely on manual formatting.
• Markdown in 'awesome list' format and use the same validation as the Digital Preservation Awesome List.
• Use a shared Zotero collection.
• Use a shared Zenodo collection and actually collect the policy documents.
• Google Sheet
• Airtable
• etc.

I personally think this data is very difficult to edit in CSV, so I'm in favour of dropping that (perhaps just generating CSV rather than editing it)...

BUT, I'm very aware we've not heard from @caylinssmith and I'm worried about accidentally making things worse!

[ross-spencer]

My current experimentation is with DecapCMS which provides a UI but simply stores all the information as e.g. Markdown with YAML frontmatter (but not CSV, see here.

Have you more information on this @anjackson?

It sounds a bit over-engineered for a simple markdown document. Importantly does it keep the standard GitHub pull-request/review/accept/merge process open?

WRT to the CSV. Perhaps that could be considered a secondary output, i.e. for sometime down the line if there is a desire (seeing what Caylin says?) There are ways to improve the original structure in plain markdown as follows:

• header: markdown front-matter, e.g. recognize Caylin as primary author
• submission metadata: markdown table (easily added to with provenance info)
• body: policy/strategy listing in alphabetical order

(Or indeed awesome-list format sounds nice and is well understood and (i believe?) has additional tooling).

Combined with basic pre-commit options to strip spaces and warn on markdown errors, KISS shouldn't also mean high-barrier.

[anjackson]

Yeah the DecapCMS thing is more useful when there's a more complicated data model. Agree it's probably overkill here. I was only thinking of it because I'm trying to avoid using too many different approaches across the different digipres.org sections. (FWIW it does just provide an alternate UI for standard GitHub workflows/interations. I'll be writing that up elsewhere.)

I think I prefer the idea of the Awesome List. We're already using that here, including automated linting. And Awesome List is probably structured enough that I could find or write something to generate a CSV from it.

But again, it depends on what people want. Certainly, those who've made submissions so far seem to be happy editing Markdown.

[caylinssmith]

Hi both, if you're available, I'd be happy to have a call later this week or next to discuss these ideas further.

[ross-spencer]

I'd be happy to have a call later this week or next to discuss these ideas further.

I feel it's important to keep convos open too, but if someone else can attend and provide a little more feedback here that'd be great.

[anjackson]

I just had a chat with @caylinssmith which I think I can summarise as:

• The CSV form reflects that this started as a Google Sheet.
• The primary requirement is that people can add changes and updates.
• Everyone who has edited so far is happy editing Markdown.
• Link checking would be useful
• CSV might be useful as an output for analysis, but this is not a requirement.

So, the absolutely simplest approach is to just switch to Markdown, drop CSV support, and maintain the formatting of the document by hand. I could then add a Markdown link checker to check for broken links - I already do this for the DigiPres Awesome List.

Do we need more than that?

[kieranjol]

This sounds great to me! Link checker is a great addition. K

…

On Fri 12 Apr 2024 at 09:33, Andy Jackson ***@***.***> wrote: I just had a chat with @caylinssmith <https://github.com/caylinssmith> which I think I can summarise as: - The CSV form reflects that this started as a Google Sheet. - The primary requirement is that people can add changes and updates. - Everyone who has edited so far is happy editing Markdown. - Link checking would be useful - CSV might be useful as an output for analysis, but this is not a requirement. So, the absolutely simplest approach is to just switch to Markdown, drop CSV support, and maintain the formatting of the document by hand. I could then add a Markdown link checker to check for broken links - I already do this for the DigiPres Awesome List. Do we need more than that? — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAITFPXRBCXTGJY6UPBIJATY46L6FAVCNFSM6AAAAABFOK7466VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJRGI4DOOJWGI> . You are receiving this because you commented.Message ID: ***@***.***>

[ross-spencer]

+1 here. Also keen to use this same markdown link checker!

(Markdown lint might be helpful too?)

I've been using this one in pre-commit:

- repo: https://github.com/igorshubovych/markdownlint-cli
  rev: v0.35.0
  hooks:
  - id: markdownlint