[BUG] Run compatibility task is not stable and consistently failing

[ankitkala]

Describe the bug
Run compatibility task assumes that all plugins are building with the base changes used for the PR(which is not enforced). Currently the task is failing with:

Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/security-analytics.git]

https://github.com/opensearch-project/OpenSearch/actions/runs/5886729563/job/15965040392?pr=9019

sample failures:

e: file:///tmp/groovy-generated-tmpdir-610570451132579633/src/main/kotlin/org/opensearch/indexmanagement/util/SecurityUtils.kt:96:29 Unresolved reference: ActionListener

FAILURE: Build failed with an exception.

  symbol:   class StreamOutput
  location: class AsynchronousSearchContextId
/tmp/groovy-generated-tmpdir-6065366233871064290/src/main/java/org/opensearch/search/asynchronous/request/SubmitAsynchronousSearchRequest.java:103: error: cannot find symbol
    public SubmitAsynchronousSearchRequest(StreamInput in) throws IOException {
                                           ^
  symbol:   class StreamInput
  location: class SubmitAsynchronousSearchRequest
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
100 errors

FAILURE: Build failed with an exception.

[peternied]

There was a fixes the compataiblity tests [1] - if this reproduces can you please reopen with new failure logs?

• Fix GH runners memory issue by increasing swapfile #9596