BWC Failure on version increment

[amitgalitz]

What is the bug?

Currently in order to run our BWC tests we need to pull the latest AD artifact from ci.opensearch.org. However, when increment version for AD we run into circular problem. For example when upgrading from 2.0 to 2.1, our CI tries to pull the latest AD version which is 2.1 from ci.openearch.org but since we haven't yet pushed our version increment commit, the artifact doesn't exist yet.

How can one reproduce the bug?
Steps to reproduce the behavior:
Version Increment PR: #567

Do you have any additional context?

At the moment we are bypassing this check for the first intial PR by just merging the version increment which then leads to its upload so it can be fetched later on. We then either manually run the CI again on the main branch or check for BWC on the next upcoming PR.

This however isn't the right way to do things because we could miss a bwc failure on the initial PR merge. I propose that we add some sort of check that if there is no artifact available from AD we will build it during our CI run and utilize that zip itself.