OCIRepositoryReconciler no-op improvements

[darkowlzz]

#913 introduced optimizations to OCIRepository reconciliation by skipping image pull when the artifact revision and the remote image revisions are the same. And it also introduced .spec.layerSelector which contributes to how the artifact is built.
Layer selector and the source ignore rules defined in the spec are ignored by the optimization as described in #913 (comment).

This change introduces content config checksum in the status fields to be able to detect any change in the source configurations and trigger an artifact rebuild if needed. It introduces .status.contentConfigChecksum to store the checksum of the observed values that contribute to the source artifact. Source ignore and layer selector values are considered in the checksum at present. Content config checksum along with the revision of the image to determine if the image pull can be skipped.

Also, when building the artifact, an ArchiveFileFilter is passed to the Storage.Archive() to effectively apply the source ignore rules.

Tests have been updated and new tests are added to cover all the changes described above.

NOTE: This change updates the OCIRepository CRD with new fields which should be updated before any testing.

Refer to #724 for similar considerations in no-op clone reconciliations in GitRepository Reconciler.

[darkowlzz]

This appears like this:

  observedGeneration: 10
  observedIgnore: |
    hpa.yaml
    foo.txt
  observedLayerSelector:
    mediaType: application/vnd.docker.image.rootfs.diff.tar.gzip
    operation: extract
  url: http://source-controller.flux-system.svc.cluster.local./ocirepository/default/podinfo-layer/latest.tar.gz

Not sure if it's okay or if we should convert it to a string containing the checksum of the ignore value.

[stefanprodan]

Let’s go with https://github.com/fluxcd/source-controller/blob/main/api/v1beta2/gitrepository_types.go#L228

[hiddeco]

There are disadvantages to this:

1. The user now needs to be aware of the mechanics of .generation and .status.observedGeneration to be able to assume that the checksum will match the current .spec elements, over just looking at a reflection of plain values.
2. By combining the values into a single checksum, you lose the ability to selectively notice changes to a sub-set of the spec.

[stefanprodan]

We'll end up with 99% of the spec reflected into the status sub-resource which feels very wrong to me. IMO we should be reconciling on generation drift, but in SC we decided to skip reconciliation when interval changes, which got us to this point where we have to track every other filed. I'm for triggering a reconciliation on spec changes, is not like interval will change so often.

[darkowlzz]

I"ve updated the implementation to add content config checksum in the status. It looks like the following:

  contentConfigChecksum: sha256:e5be314a56171026ffe3b828fec184d6144fb602511ee7243d02193e26114c45
  observedGeneration: 6
  url: http://source-controller.flux-system.svc.cluster.local./ocirepository/default/podinfo/latest.tar.gz

Based on some private discussions, we are adding content config checksum here for now to align with the GitRepository API.
In a future release, we'll replace this checksum with more explicit observed values in the status to improve the usability by looking at the status and easily seeing the differences in the declared state and the observations made by the controller.

[stefanprodan]

LGTM

Thanks @darkowlzz 🏅