Cross Account IRSA for multi-account cluster

[prasoon-pxc]

I have three EKS clusters which is running on three different AWS account , for now I have installed velero on every clusters separately and it is working fine and I am using IRSA based access for S3-bucket and volume-snapshot.
Now, I have requirement that I want to take backup from my dev clusters(test-ns1) and store it in a staging clusters(test-ns-1), i know that I have to use same S3-bucket for both accounts for it to work, but how can I pass credentials for dev-bucket to staging bucket using IRSA based access

[mdnahian]

I have not tried this out myself but this might be possible by configuring the dev EKS cluster to use the OIDC provider used for the staging clusters. Assuming that the S3 bucket is in the same AWS account as the staging clusters, you can create a role that gives federated users access to the bucket and then update the annotation in the Velero service account in the dev cluster. The procedure should be similar to the one described in this blog post:
https://aws.amazon.com/blogs/containers/cross-account-iam-roles-for-kubernetes-service-accounts/

Thanks, Md (CloudCasa).

[prasoon-pxc]

Thanks @mdnahian, I will try and update you.

[prasoon-pxc]

I have tried above things it is working if I am using my role with one service account and then using that service account in one of deployment but the same thing is not working in case of velero setup, I have tried to annotate velero service account with my federated roles but it does not work

Followed below doc for testing:
https://www.teracloud.io/single-post/cross-account-access-to-s3-using-irsa-in-eks-with-terraform-as-iaac

[prasoon-pxc]

So, I have successfully able to create cross-account backup using velero also, I need to mention the velero sa in irsa related policy like below:

condition {
      test     = "StringEquals"
      variable = "${local.staging_oidc_url}:sub" 
      values = [
        "system:serviceaccount:default:s3-sa",
        "system:serviceaccount:velero:velero"  ## need to add this line
      ]
    }
  }
} 

But my main question is that , now I want to use two different buckets one for same cluster backup and second one is for cross-account backup but velero service account does not allow me to annotate it with multiple roles. I have to use two roles for it to work.