Implement fix for Connector-Restricted-Usage Policy not working since EDC CE v5.0.0

[tmberthold]

⚠️ Bug Fix Delivery Required for MDS by Wednesday 14.02 ⚠️

Bug Report

Description

The Connector-Restricted-Usage Policy is no longer working since EDC0 migration (EDC CE v5.0.0).

Problem log:

edc-extensions-edc2-1          | 2024-01-17 12:55:36 StateMachineManager [consumer-contract-negotiation] error caught 
edc-extensions-edc2-1          | java.lang.NullPointerException: Cannot invoke "org.eclipse.edc.spi.agent.ParticipantAgent.getClaims()" because the return value of "org.eclipse.edc.policy.engine.spi.PolicyContext.getParticipantAgent()" is null
edc-extensions-edc2-1          |        at de.sovity.edc.extension.policy.functions.AbstractReferringConnectorValidation.evaluate(AbstractReferringConnectorValidation.java:72)
[...]
edc-extensions-edc2-1          |        at org.eclipse.edc.protocol.dsp.dispatcher.DspHttpRemoteMessageDispatcherImpl.dispatch(DspHttpRemoteMessageDispatcherImpl.java:109)
[...]

Short version:
This has been a problem since migrating from core-edc MS8 to core-edc v0.2.1. Since then, the policy has not worked in any of our EDC CE versions (from EDC CE v5.0.0, regardless of whether dev or mds). With core-edc v0.1.2 breaking changes were introduced on how a core-edc's PolicyContext can be built by the class-constructor, which no longer requires a ParticipantAgent. The core-edc class DspHttpRemoteMessageDispatcher makes use of this possibility and builds a PolicyContext without a ParticipantAgent, whereupon our Connector-Restricted-Usage Policy AbstractReferringConnectorValidation subsequently throws the null pointer when invoking policyContext.getParticipantAgent.getClaims().

Technical derivation of the problem:
Our class: nullpointer because policyContext.getParticipantAgent() returns null when invoking getClaims() on it at:
https://github.com/sovity/edc-extensions/blob/45d15241c59c1313de31fb179eabe3bc7991929a/extensions/policy-referring-connector/src/main/java/de/sovity/edc/extension/policy/functions/AbstractReferringConnectorValidation.java#L72

Changes to core-edc v0.1.2:

• The new default constructor PolicyContextImpl() does not require a ParticipantAgent and set's it to null when invoking the second class-constructor (L36). LINK
• If the new builder of the class is used, no ParticipantAgent is set straight away through the new default constructor; the ParticipantAgent must be set explicitly using the additional method (L97).
• In core-edc's DspHttpRemoteMessageDispatcherImpl there is such a builder call that does not provide a ParticipantAgent and then triggers the policyEngine.evaluate after that (L91-95). LINK
• The logs above start from here. Due to the triggered policyEngine.evaluate, the non-existent ParticipantAgent in the PolicyContext is missing when evaluating the Connector-Restricted-Usage Policy, which then leads to the null pointer above by invoking policyContext.getParticipantAgent().getClaims().

Expected Behavior

The policy should work if it is selectable as a policy.

Observed Behavior

See above, a NullPointer exception is thrown at evaluation time for the reasons mentioned above.

Steps to Reproduce

Start any EDC CE variant, e.g. EDC CE DEV locally with the 2 standard EDC included in the docker-compose, add an asset, and select this policy within a contract definition as contract policy and negotiate the contract with a second connector.

Context Information

Outlook:
As of core-edc v0.4.0 (not yet migrated to by us), the PolicyContext's getParticipantAgent method itself has additionally been removed.
eclipse-edc/Connector@c9d6da4#diff-0c2ac585810576289fdc58fe672ffa775671645c49c2786f8db098c7a88ea4f5

Testing:
Brainstorming between @tmberthold @jkbquabeck @richardtreier

• We are considering to test the fix manually with the EDC CE dev-variant locally if there are no automated tests for this yet
  • deploying an EDC on sirius and testing it against the DAPS there could be a solution
  • Or testing it locally with a modified docker compose that connects to a DAPS

Possible Implementation and Work Breakdown

Tasks

Beta Give feedback

1. Understand the analysis and see if it is correct
2. Check whether it can be fixed, as it has been a problem since EDC 0 migration or decide on removing the policy-extension at all.
3. Provide a fix or remove the policy.
4. Run manual test before delivery

[AbdullahMuk]

@efiege as aligned during today's daily, as your highest prio please implement the fix for this gap. Some important points:

• after you analyse the problem and plan a solution for it, please add your effort estimation (number of work days needed to implement fix) to the "Story Points" field
• at the latest we want the fix to be implemented by Wednesday 14.02 so that we can ship out the fix in Raze release on Thursday 15.02 because we need to respect the M600 deadline for MDS 2.0. If your effort estimation exceeds this date, please contact me on Teams ASAP.
• we agreed at daily today that we will test the topic manually because the E2E tests we want to implement during the scalability bootcamp will not be ready in time. Please contact @richardtreier to get the sample tests he has related to this topic.

As needed, please contact me on Teams wherever I can support you more.

[tmberthold]

is working, I fell into a trap

[efiege]

This issue has been introduced when new PolicyScopes have been added to the core-EDC.

These PolicyScopes are evaluated on incoming and outgoing dsp calls. For the incoming dsp calls this has not been an issue, because the counterParties identity can be fetched from the incoming token into a ParticipantAgent. On the other side there is no counterParty token available for outgoing requests.

To circumvent this issue, the registration of the policy has been changed. It is no longer registered for all scopes, but just for contract.negotiation and catalog scopes, such that this policy is evaluated for contract negotiation and retrieving the catalog, but not on a dsp-request level.