Releases: devcurmudgeon/ybd
GLB
e74d938 Note for move of ybd upstream
96a1ddd Fix declaration of 'readme.md' in setup.cfg
ef460b2 Merge branch 'tacgomes/ybd-tacgomes/fix-git-mirroring'
75511a1 Avoid running the git gc in the background
c24b2c9 Fix error message
57ae275 Merge branch 'james/chmod_pip' into 'master'
a7a8f41 Make get-pip executable
3cdf9bd Merge branch 'jjardon/get_pip_fix' into 'master'
465e9ce install_dependencies.sh: Fix get-pip.py execution
437749a Merge branch 'jjardon/sudo_fix' into 'master'
7631d2c install_dependencies.sh: Make sudo optional
65bd91f Merge branch 'tiagogomes/improve-docs' into 'master'
4b02337 Improve documentation
d8aba23 Merge branch 'jjardon/gitlab_move' into 'master'
5f49085 ybd is hosted in GitLab now
e7be39b Include max-jobs in cache-key
db7d285 Show download count
e83f8b0 Rejects were causing status page 500 error
86f8e0b Force ref to be a string
766b244 Fixes #241 - reorder tar adds so dirs are last
3e0ec8c Fixes #241
c35ede2 Merge branch 'jjardon/ci' into 'master'
ef4d449 .gitlab-ci.yml: Do not print all the environment variables
43aab08 Merge branch 'ps/simpler-walk' into 'master'
f072299 Simplify logic for walk and parse definitions
969df36 Regularise artifact-version range checks
33aba94 Merge branch 'jjardon/ci' into 'master'
259d1a7 .gitlab-ci.yml: Build minimal system
3f8bd65 .gitlab-ci.yml: keys-only mode and artifact cache:1 only in cache-keys stage
f77415f .gitlab-ci.yml: Remove installation stage
e918103 .gitlab-ci.yml: Fix cache key: path -> paths
b972ca1 Update readme.md: Add gitlab build status
ec37d28 We only need root permissions for actual builds
91daaf0 Pep8
26a3170 Move permissions check after usage check
d2e954a Merge branch 'master' of https://github.com/devcurmudgeon/ybd
76a5758 Spot the deliberate mistake in previous commit :/
960914e Merge pull request #240 from leeming/leeming/missing-cwd-conf
f29fb1c Fix for #238
2c7034d Pep8
0ae83f8 Fixes bug that ignores ybd.conf on cwd
c069aba Check every field in every definition
340be12 Add tristan's 'awful hack' until it lands in released sandboxlib
9ba66ef Merge pull request #239 from jjardon/jjardon/fix_python_dep
b332ab7 install_dependencies.sh: Python2 is python2 in Arch
a39961b Merge pull request #236 from leeming/leeming/redundant-exit-setting
3a7ede8 Simplified setting 'exit' var from config setting
831c19f Revert "Always exit if specified morph file is missing"
fec1a23 Need to include 'devices' in cach-key
afb2477 Add python to dependencies
b4a1be9 Always exit if specified morph file is missing
COV
New in this release
- YBD can now generate a Concourse-parseable pipeline for visualisation (but not build, yet). Set mode: parse-only to try it.
- fixed a bug where ybd would ignore errors in the middle of set combined commands in any build-step
- fixed another example of mishandled unicode file paths
- artifact-version bumped to 6
47424eb Update readme with basic kbas config info
7bcde1c Bump artifact-version for the sh -e bugfix"
887d941 Fix local variable 'key' referenced before assignment
588e7c7 Fix for #233
76b6fc7 Force mimetype for artifacts
8bb6e76 Concourse visualisation working
50eb61f Don't lose stratum depends
625060b Fix write_chunk_metafile so it can handle unicode paths
72ac462 Force env['LANG'] => 'en_US.UTF-8'
bbb9a66 Revert "Force UTF-8 encoding in OSFS"
eebce03 Only report arch mismatch once
bfda426 Force UTF-8 encoding in OSFS
c8522b8 Lose stderr too
f636779 Lose stdout from wget on tarballs
a6c5c02 Rename basenames => filenames
TDY
All of these changes are fixes and tidy ups - no new functionality.
b6f6a55 Exit if unpack fails - it's no good continuing
3c82bc2 Drop deprecated GitLab badge for now
07bbc42 Pep8
35c3e23 Log arch mismatch
45ef539 Exit if attempt to deploy unbuilt system
f0a7992 Check fields in morph files, and exit if something odd is found
2f25e36 Compose subsystems before systems
03c09d0 Default to warn on missing install-commands
88e2d88 Merge pull request #229 from leeming/leeming/issue-218
d6cd8f0 Adding in exception handler to catch Unicode Encoding error and handle unicode filepaths
8b6012d Dump contents of dn
KLN
Main changes are
- various tidyups to the code
- fixing some corner-cases to deal with malformed definitions
- coping with nasty overlaps
- kbas now has download links
- artifact sizes now contain commas, so they're easier to read
6c2dc60 Pep8
29c8430 Tidyup some messages
0b19284 Try unlink, then remove
a7696d1 Revert back to tristan's fix
e560f0f Probable fix for #228
8db496d Revert "Allow + artifact names"
6c7b182 Allow + artifact names
17d1f4d Reuse _process_tree code for _process_list
7f7032c Fix link colour, and try relative links
0ee8a50 Merge pull request #227 from gtristan/staging-symlinks
39ecc9d Ensure that we only stage relative paths for symbolic links.
eb23f0d Provide download links for artifacts
cbaeb02 add debug
8299778 Trap if chunks or strata are not lists
c18c649 Skip 'tree' and 'cache' for release notes comparison
08264d9 Renames for readability, i => key
a609a3c Fix release-note logic
f9e7671 Release note only includes changes to target
c3e1c98 Only save entries in .trees where ref: is a SHA
b81c8b3 Log number of trees re-used from trees file
6e61371 Save definitions.yml again
cb744a6 Tidyup checks in Morphs, so check-definitions: flag is honoured
68bba6f Exit if a definition lookup fails
7bf0892 Move app.exit function in to app.log
2e076dd Add more logging for split processing
36e950c Add traceback for failed split install
e36813e Fix stratum['name'] + split
0aba49a Better log of installing splits
865c202 Use .trees file properly, in artifacts dir
ac41d64 Fix #224 properly
7a9cda5 Revert "Fix #226"
6faf067 Revert "Rename unpack => save, and don't unpack!"
4809a04 Fix #226
4856263 Rename unpack => save, and don't unpack!
11d6dba WIP fix for #224 system-level splitting with system-integration-commands
405b3ee Separate thousands in artifact sizes
b16dd6f Rename component => dn
82697b7 Rename item => dn
REL
The main improvements this time are
- cache-key calculation now makes better use of the .trees file (no need for check_trees)
- added a release-note config option, which creates a release-note with
- all definitions changes since last release
- all new commits (for all components) since last release
54ee540 Disable release-note by default
0a67719 Add release_note.py
4e7eaf8 Remove unused lines
02b79c4 Rename dn['build'] => dn['checkout']
7c4823f Rename definitions.py => morphs.py
cd2e3cd Separate out Morphs class and new Pots class
098063f No need for check_trees function, so remove it
c43828d Add 'get_last_tag' and 'run' functions
27d1879 Some renaming, lose a line
773a853 Revert concourse-specific internal data-model change
6e2cc03 Lose a line
c32a92f Swtich from [0, 1, 2] etc to range()
5cdc018 Rename component => dn in assembly.install_contents
2345535 Rename component => dn in assembly.install_dependencies
e0ffc6a Rename component => dn in assembly.compose
978ce1a Rename component => dn in assembly.build
c19764f Deal with not-yet declared foo mentioned as build-depend
8e354b1 Fix schema-validation (check or not) logic
cfa515e Write out internal yaml for no-build mode too
06ddf1b Various fleck8 fixes
f869d8c Fix missed import of app.exit
27b92f3 Pep8
16.24
Quite a lot of cleanups, not much new functionality:
- settled on 'dn' as a generic variable name for a definition
- WIP parse-only mode now generates a Concourse pipeline which displays (but doesn't run)
- log counts for systems, strata and chunks involved to make target
3cd67d7 Always log counts
815ee7d Shorten some lines
3b24123 Revert "Shorten some lines"
ec035e2 Shorten some lines
27b15fe WIP Concourse pipeline for parse-only
3c2adc8 Show counts for chunks, strata, systems
fbceb4d Tidy up for legibility
b2c56f5 Stop false warnings on old artifact-versions
3b3eae1 Fix #225 too many pies
26f2dfa Simplify some definitions code
a883cd4 Remove all the 'defs' parameters
57bb1c9 Rename definition => dn
d69e06b Rename 'this' => dn
3b55265 Only upload chunks by default
f6bfb0e Artifact version 5 - standardising 'path' key
6c464db De-morph paths from artifact-version 5
bfb93d9 As suggested by rjek, f.write(response.content)
ab3909a Lock down artifact-version check in integer range
5a29b4d Re-enable default-splits logic
95b75bc Fix for #223 dd checking of each definition name vs path
1754181 Add 'cleanup' option so we can leave debris around if required
AV4
Apart from various cleanups, this version is mainly about
- artifact-version: 4 (include definitions repo: and ref: in metadata for systems and strata)
- @rjek provided a fix for ybd failing to handle nginx compressed downloads
- @jjardon added support for running ybd on Arch Linux
e334c50 Add definitions repo+ref to .meta, from artifact-version: 4 onwards
77ad4e4 Only report skipping once
2b50266 Delay sandbox creation - may help for #139
fde2f67 Fix #175 using Eli Bendersky's Counter class
52e802d Gah... fix 543b654
543b654 Add 'skipping compose' message
8aed0f8 Lose a couple of lines
e403cfb Reverse logic, lose a line
62fedc2 Fix comments in default ybd.conf
7a3cb92 Fallback to copy all files if metadata copy fails
63e3567 Shorten some lines by 'from app import...'
739631b Shorten lines by 'from app import...'
016398f Make 'contents' field dict, containing splits if any
d33e0c8 Artifact-version: 4, include repo: & ref: in meta
13ab7a0 Standardise .meta artifact|files => artifact|components
dc24b2f Cache-key only in .meta from artifact-version 2
2b3786f System cache-key varies with default-splits
25c58d2 Report actual 'kind' in splitting log
b43bb44 Clarify what the try/except is for
41e064a Lose a couple of lines
4304456 Rename element => product
a1c8bc3 Don't include anchors in parse output
f00e6cd Output internal format as yaml
b09eeb4 Revert "Attempt fix for #218"
94461f4 KBAS was using response.raw, instead of response.content
b2f0866 Missed another absolute /static
f34bd8a Use relative path for css
5ccb421 Exit 1 on calling with no params
38c2220 Merge pull request #219 from jjardon/jjardon/arch
155c828 install_dependencies.sh: Add support for Arch
b4e6cdd Attempt fix for #218
5c2c9a3 Exit if YAML parsing fails on any definition
4972e30 Check for installation success
8959629 Check that apt|yum|dnf is found
dace4d1 Disable splitting by default, because it's slower
6aa0ffb No need to exit if bootstrap artifacts missing
42a2cd7 Drop 'Cache key is' text
5300c64 State where surprise exceptions happen
5ed3cba Exit on missing morph file
3155ba9 Only support split systems from artifact-version 3
0d2d04f Pep8
779d3e7 Force new cache-key for split systems
461d282 Create runtime systems by default
7c73938 Move checkboxes so they are easier to scan
f33c244 Fixes #163 Warn if any specified .morph file is missing
950d6aa Remove wrangler.py
SPL
Mainly this week has been about re-writing how ybd installs contents and dependencies
into sandboxes.
- splitting.py now uses ybd's sanitised definitions data model, rather than leftover information from morphs. The key commit is af40460.
- ybd now shows a checkbox for whether the artifact already exists or not.
32325cf Show more details on sandbox error
bb09db1 Re-write install_dependencies too
af40460 Fix ybd representation of splits
0ecd262 Show checkbox for cache-keys already found
c6d3922 Rename get_metafile, and lose some lines
b03b41d Lose some lines importing app functions by name
6460b10 Remove install_contents
e5381b7 Re-write install_contents as add_contents
84630ae Move assembly function into compose
32b972a Pep8
4f67f22 Tidyup manifest handling
26fb0e9 Lose a few lines by adding 'verbose' flag
927643f Remove leading space for 'omit' mode
c7297b8 Add results of splitting test
4892798 Allow current cache.py to run with original splitting
e278b36 Lose a couple of lines
7a0d2f0 Shorten a couple of lines
AV3
Key changes
- artifact-version is now 3 (we now only hash applicable build system in each key, rather than all of them)
- if max-jobs and instances not specified, ybd will set them to parallelise with no instance getting more than 9 cores
- @nuxeh fixed a long-standing bug in subsystem deployment #215
- fixed #206, where absence/presence of .git could break git clones on switching protocols in aliases
9fb0a8e Cleanups in definitions.py
5e04a62 Rename definition => item
0a076c7 Rename definitions => data
f7b7b9e Tidyups in splitting.py, this => component
1c9e2c3 Move get_meta functions into splitting.py
ec36a33 http repo urls must end in .git
e6eff37 Add splitting result from rdale's original branch
54fc057 And now fix max-jobs/instances properly
580b7da Set instances, max-jobs if user doesn't
80fadf2 Remove some unused imports/variables
c8cc2d0 Code improvement from discussion on #baserock
40162ba Fix ugly indent and lose a line
92d0b42 Merge pull request #215 from nuxeh/edcragg/fix-subsystem-deployment
8a76f05 Fix subsystem deployment
0ff0e98 Revert "Parse put into yaml instead of json"
621d946 Parse put into yaml instead of json
3f722ea Draft of new test appraoches for parsing and splitting
bad3653 Allow result-file to be set in config
f202922 Add new 'parse-only' and 'log-timings' modes
e4671d3 Improve config comment
ae5bbfd First-pass at manifest creation
95fd50e Catch exceptions on run_sandbox_with_redirection
1220e80 Now get_repo_url won't break for repo=None
bd04031 Maybe fix for #209 groff copy fail
e09ab43 Gah... 'this' was not specified
16f0dea Merge branch 'devel'
14a2d92 artifact-version: 3
78a8a20 Log sandbox dir
4d8b5a2 Show ybd version at top of log
689dffa Exit if cache-key fails for stratum content chunk
0cb5278 Prettify setup config for easier reading
258fe7c Capitalise 'splitting' messages
bdebac9 Keep going if stratum reproduction fails
6bacc95 Use os.stat(x).st_atime, no need to touch files for access time
eafb0fe Fixes #207 - exit if we can't process metafile
82e7159 Improvements to concourse pipeline generator
ea12b8c If mode is no-build, no need to do the hashing
de329fc Some readme fixes, including adding link to c.b.o
4846017 Some readme fixes, including adding link to c.b.o
2058035 Fix spelling mistake and pep8
566c62e Fixes #206 - tested on ci.morph
18691e8 Stripping .git on (say) http://foo.bar/bip.git breaks clone
6e8f787 28792b fix for #205 was incompatible with new bottle versions
4cf58e6 Restrict upload filenames
7359cf3 [ci skip] Various README fixes
0d13769 Add comments to kbas config, set default port to 8000
28792b9 Fixes #205, but how come it was working in general?
22b7b3b Add check-overlaps: ignore, warn, exit
934fd7e Amend cache-key algorithm - per build-system