Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve dependencies cache usage #367

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions __tests__/cache.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,22 @@ describe('dependency cache', () => {
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith('maven cache is not found');
});
it('downloads cache with a custom key prefix', async () => {
createFile(join(workspace, 'pom.xml'));

await restore('maven', 'YYYY-MM');
expect(spyCacheRestore).toBeCalledWith(
[expect.stringContaining('/.m2/repository')],
expect.stringContaining(`setup-java-YYYY-MM-${process.env['RUNNER_OS']}-maven-`),
[
`setup-java-YYYY-MM-${process.env['RUNNER_OS']}-maven-`,
`setup-java-YYYY-MM-${process.env['RUNNER_OS']}-`,
'setup-java-YYYY-MM-'
]
);
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith('maven cache is not found');
});
});
describe('for gradle', () => {
it('throws error if no build.gradle found', async () => {
Expand Down
3 changes: 3 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ inputs:
cache:
description: 'Name of the build platform to cache dependencies. It can be "maven", "gradle" or "sbt".'
required: false
cache-key-prefix:
description: 'Custom key prefix to give extra flexibility on managing the cache expiration'
required: false
job-status:
description: 'Workaround to pass job status to post job step. This variable is not intended for manual setting'
default: ${{ job.status }}
Expand Down
31 changes: 31 additions & 0 deletions docs/advanced-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
- [Publishing using Apache Maven](#Publishing-using-Apache-Maven)
- [Publishing using Gradle](#Publishing-using-Gradle)
- [Hosted Tool Cache](#Hosted-Tool-Cache)
- [Expiring Dependencies Cache](#Expiring-Dependencies-Cache)

See [action.yml](../action.yml) for more details on task inputs.

Expand Down Expand Up @@ -350,3 +351,33 @@ GitHub Hosted Runners have a tool cache that comes with some Java versions pre-i
Currently, LTS versions of Adopt OpenJDK (`adopt`) are cached on the GitHub Hosted Runners.

The tools cache gets updated on a weekly basis. For information regarding locally cached versions of Java on GitHub hosted runners, check out [GitHub Actions Virtual Environments](https://github.com/actions/virtual-environments).

## Expiring Dependencies Cache

You can define the `cache-key-prefix` input and either bump it manually, or use a certain time period. That way you can prevent the cache from growing abnormally.

### Manually
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
with:
cache-key-prefix: V1
- run: java -cp java HelloWorldApp
```
And then just bump `V1` manually.

### Monthly
```yaml
steps:
- uses: actions/checkout@v3
- name: Get current month
id: month
run: echo "::set-output name=month::$(date +'%Y-%m')"
- uses: actions/setup-java@v3
with:
cache-key-prefix: ${{ steps.month.outputs.month }}
- run: java -cp java HelloWorldApp
```
The cache is busted automatically every month.
You can define any time period that better suits your pipeline.
37 changes: 30 additions & 7 deletions src/cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import * as glob from '@actions/glob';

const STATE_CACHE_PRIMARY_KEY = 'cache-primary-key';
const CACHE_MATCHED_KEY = 'cache-matched-key';
const CACHE_KEY_PREFIX = 'setup-java';
const SETUP_JAVA_CACHE_PREFIX = 'setup-java';

interface PackageManager {
id: 'maven' | 'gradle' | 'sbt';
Expand Down Expand Up @@ -67,24 +67,48 @@ function findPackageManager(id: string): PackageManager {
return packageManager;
}

function computeKeyPrefix(keyPrefix: string | undefined) {
if (keyPrefix === undefined) {
return SETUP_JAVA_CACHE_PREFIX;
}

return `${SETUP_JAVA_CACHE_PREFIX}-${keyPrefix}`;
}

/**
* A function that generates a cache key to use.
* Format of the generated key will be "${{ platform }}-${{ id }}-${{ fileHash }}"".
* If there is no file matched to {@link PackageManager.path}, the generated key ends with a dash (-).
* @see {@link https://docs.github.com/en/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key|spec of cache key}
*/
async function computeCacheKey(packageManager: PackageManager) {
async function computeCacheKey(packageManager: PackageManager, cacheKeyPrefix: string) {
const hash = await glob.hashFiles(packageManager.pattern.join('\n'));
return `${CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${packageManager.id}-${hash}`;
return `${cacheKeyPrefix}-${process.env['RUNNER_OS']}-${packageManager.id}-${hash}`;
}

/**
* A function that generates a list of restore keys to use.
* The restore keys will follow the same format as the computed cache key.
* @see {@link https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#example-of-search-priority|spec of cache key}
*/
async function computeRestoreKeys(packageManager: PackageManager, cacheKeyPrefix: string) {
return [
`${cacheKeyPrefix}-${process.env['RUNNER_OS']}-${packageManager.id}-`,
`${cacheKeyPrefix}-${process.env['RUNNER_OS']}-`,
`${cacheKeyPrefix}-`
];
}

/**
* Restore the dependency cache
* @param id ID of the package manager, should be "maven" or "gradle"
* @param customKeyPrefix Optional cache key prefix. If not present will default to the action name.
*/
export async function restore(id: string) {
export async function restore(id: string, customKeyPrefix: string | undefined = undefined) {
const packageManager = findPackageManager(id);
const primaryKey = await computeCacheKey(packageManager);
const cacheKeyPrefix = computeKeyPrefix(customKeyPrefix);
const primaryKey = await computeCacheKey(packageManager, cacheKeyPrefix);
const restoreKeys = await computeRestoreKeys(packageManager, cacheKeyPrefix);

core.debug(`primary key is ${primaryKey}`);
core.saveState(STATE_CACHE_PRIMARY_KEY, primaryKey);
Expand All @@ -96,8 +120,7 @@ export async function restore(id: string) {
);
}

// No "restoreKeys" is set, to start with a clear cache after dependency update (see https://github.com/actions/setup-java/issues/269)
const matchedKey = await cache.restoreCache(packageManager.path, primaryKey);
const matchedKey = await cache.restoreCache(packageManager.path, primaryKey, restoreKeys);
if (matchedKey) {
core.saveState(CACHE_MATCHED_KEY, matchedKey);
core.setOutput('cache-hit', matchedKey === primaryKey);
Expand Down
1 change: 1 addition & 0 deletions src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export const INPUT_DEFAULT_GPG_PRIVATE_KEY = undefined;
export const INPUT_DEFAULT_GPG_PASSPHRASE = 'GPG_PASSPHRASE';

export const INPUT_CACHE = 'cache';
export const INPUT_CACHE_KEY_PREFIX = 'cache-key-prefix';
export const INPUT_JOB_STATUS = 'job-status';

export const STATE_GPG_PRIVATE_KEY_FINGERPRINT = 'gpg-private-key-fingerprint';
3 changes: 2 additions & 1 deletion src/setup-java.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ async function run() {
const packageType = core.getInput(constants.INPUT_JAVA_PACKAGE);
const jdkFile = core.getInput(constants.INPUT_JDK_FILE);
const cache = core.getInput(constants.INPUT_CACHE);
const cacheKeyPrefix = core.getInput(constants.INPUT_CACHE_KEY_PREFIX);
const checkLatest = getBooleanInput(constants.INPUT_CHECK_LATEST, false);

const installerOptions: JavaInstallerOptions = {
Expand Down Expand Up @@ -43,7 +44,7 @@ async function run() {

await auth.configureAuthentication();
if (cache && isCacheFeatureAvailable()) {
await restore(cache);
await restore(cache, cacheKeyPrefix);
}
} catch (error) {
core.setFailed(error.message);
Expand Down