Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osbuild: first step towards stage separation - allow meta.json for stages #1460

Closed
wants to merge 4 commits into from

Conversation

mvo5
Copy link
Contributor

@mvo5 mvo5 commented Nov 21, 2023

Instead of always parsing the python stage to load meta information
allow the user of a new {stage}-meta.json file. This is a first
step towards allowing modules to be written in a different language
than python. It also has some practical advantages:

  • slightly faster as it avoids calling python to output the schemas
  • easier to write schemas as this can be done in a real json editor
    now
  • more extensible in a future where stages maybe binaries with
    shlib dependencies that are only satisfied in the buildroot
    but not on the host

An example/toy stage that uses this can be seen in https://github.com/osbuild/osbuild/compare/main...mvo5:stage-separation-json-2?expand=1 (the last two commits)

@mvo5 mvo5 requested a review from supakeen November 21, 2023 21:19
Copy link
Member

@supakeen supakeen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like moving this information out of the Python files and into a separate metadata format. It allows better validation and opens the door to other types of stages.

osbuild/meta.py Outdated Show resolved Hide resolved
Comment on lines +432 to +487
long_description = meta.get("description", "no description provided")
if isinstance(long_description, list):
long_description = "\n".join(long_description)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand why, and I can't come up with a better plan, but I don't like having to join a list to have multi-line strings in JSON :(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's really not great but I couldn't think of a better way (with plain json)

Comment on lines +436 to +497
info = {
"schema": {
"1": meta.get("schema", {}),
"2": meta.get("schema_2", {}),
},
"desc": meta.get("summary", "no summary provided"),
"info": long_description,
"caps": meta.get("capabilities", set()),
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's describe what's in -meta.json with a jsonschema and validate it when loading.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a schema now, please let me know if that is what you have/had in mind or if it too broad still?

@mvo5 mvo5 force-pushed the stage-separation-json branch 2 times, most recently from f079fb0 to 9f0ad27 Compare December 5, 2023 11:02
Instead of always parsing the python stage to load meta information
allow the user of a new `{stage}-meta.json` file. This is a first
step towards allowing modules to be written in a different language
than python. It also has some practical advantages:
- slightly faster as it avoids calling python to output the schemas
- easier to write schemas as this can be done in a real json editor
  now
- more extensible in a future where stages maybe binaries with
  shlib dependencies that are only satisfied in the buildroot
  but not on the host
Some existing code/test assumes that anything in `stages/` is
executable. This commit fixes this and excludes e.g. json from
lintables.
With `.meta-json` it's a "real" file extension. The `-meta.json`
feels a like bit too ad-hoc.
Copy link

This PR is stale because it has been open 30 days with no activity. Remove "Stale" label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale label Jan 22, 2024
@mvo5
Copy link
Contributor Author

mvo5 commented Jan 22, 2024

Seems a slightly bad time to get reviews currently so I close this for now

@mvo5 mvo5 closed this Jan 22, 2024
@supakeen
Copy link
Member

Can you re-open this? I'd like to review and get this in in one shape or another.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants