fix cache of build pipeline

[caco3]

For the Github Actions, we can use caches. One of them caches the Espressif build objects, making a build speed up from 10 to 2..3 minutes.
This no longer seems to work.

the platformio.ini changed a lot in the last weeks. I looked a bit into it but already the slightest changes break the caching :(

[caco3]

@nliaudat do you understand why caching no longer works?

As you can see here, it stopped working after this changes: e661e1d

@Slider0007 I looked into it but could not trace it down to the reason :(

[nliaudat]

I mean sdkconfig.esp32cam-testing and in general all sdkconfig. XXX another than default should no be uploaded to repo.
The github cache is limited in size and many pull request in a short time from different branches can lead to a bad cache.
I propose deleting all cache entries and sdkconfig xxx

• sdkconfig - Kopie.defaults
• sdkconfig - Kopie.esp32cam
• sdkconfig.esp32cam-testing

https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#managing-caches

[caco3]

could you give it a try?
You can also do it in your own fork if you want.

[nliaudat]

Last run fast ...

[nliaudat]

next build : Cache not found for input keys: Linux-pio-1829/merge

Cache is full and latest build are not stored.

[caco3]

next build : Cache not found for input keys: Linux-pio-1829/merge

We have separate caching for each branch, and because its the first build for that branch, obviously there is no chache available yet, see https://github.com/jomjol/AI-on-the-edge-device/actions?query=branch%3Aesp_sys_info

Cache is full and latest build are not stored.

The cache can not get full in this way. See https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries, github automatically starts deleting other caches if space is exceeded!

Last run fast ...

That is interesting!

Sadly I can not reproduce this :(
I merged your sdkconfig cleanup, let it build in rolling, then did a simple change of the readme, but still it runs a ful build, see https://github.com/jomjol/AI-on-the-edge-device/actions/runs/3915776245

[nliaudat]

Have to investigate futher ....

What about "Create clean-branch-cache-after-pull-request-got-closed.yml"

[nliaudat]

Latest small commit (add file to git ignore) :
#1830
Cache not found for input keys: Linux-pio-1830/merge

[nliaudat]

use build cache v3.2.3 ??

 - name: Use Build Cache
      uses: actions/cache@v3
      with:
        path: ./code/.pio/
        key: ${{ runner.os }}-pio-${{ github.ref_name }}

[nliaudat]

I've deleted all cache in my repo => compile time ~2 min

Here with no deletion => compile time ~7 min

[nliaudat]

I've add manually triggered workflow clear_cache.yml in #1831

[caco3]

What about "Create clean-branch-cache-after-pull-request-got-closed.yml"

I tried to use the example from https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries, but it does not work due to missing credentials.

I've add manually triggered workflow clear_cache.yml in #1831

❤️

Does that fix the caching issue?

[caco3]

Looking at following screenshot shows that it works for your forked rolling but not ours:

maybe https://stackoverflow.com/a/63590002 has an explanation:

Once you create a cache, you cannot change the contents of an existing
cache but you can create a new cache with a new key.

Since we modified the platformio.ini, the cache gets invalid but can not be updated.

We need to delete that cache somehow or update the key used for it.

GitHub will remove any cache entries that have not been accessed in over 7 days.

Or maybe it solves itself after a week?

[caco3]

I reworked the build pipeline according the example in https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache

Hopefully this fixes it now.
It now picks the latest cache, uses it and creates a new (updated) cache. This way a cache newer is outdated. The older caches automatically get deleted after 7 days.
Even more, according to the docs, the branches use the caches from rolling. This means that the first build of a branch uses the cache from its parent and does not need to run a full build.

=> #1836

[caco3]

@nliaudat sorry to have accused your changes to platformio.ini.
I now learned a lot about the caching and could improve it even further with that pull request.

[nliaudat]

@caco3 np, but please learn platformio.ini env asap :)