Approvers are receiving GitHub notifications for every PR

[jpkrohling]

Even with the inclusion of the bot that auto assigns people as reviewers, everyone in the approvers group is receiving a notification to review all the issues. Example:

Notifications page:

PR page:

In order to not cause "alert fatigue" to approvers, the group should not be added as reviewer, but instead, a specific person chosen by the bot.

[tigrannajaryan]

+1 on the idea. @bogdandrutu what was the problem that required all approvers to be listed for all directories?

[jpkrohling]

I didn't know about this until right now, but GitHub does have a feature that automatically assigns people as reviewers and has the ability to not notify the whole group:

Doc about this feature: https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-organizations-and-teams/managing-code-review-assignment-for-your-team

[MikeGoldsmith]

I like the idea of auto-assigning a single approver per PR to help reduce the noise generated to the whole group. However, we have seen two cases where a vendor span exporter has had PRs merged without approval from a vendor representative.

#291
#1162

There is a specific rule setup in the CODEOWNERS file for the honeycomb exporter, but as the process selects only one approver, we just didn't get notified.

@paulosman @lizthegrey @open-telemetry/collector-contrib-maintainer

[jpkrohling]

In the meantime, the following query works for me to find out which reviews are waiting for me individually:

is:open is:pr review-requested:jpkrohling archived:false -team-review-requested:open-telemetry/collector-contrib-approvers 

[jpkrohling]

Ping @tigrannajaryan, @bogdandrutu

[mx-psi]

@bogdandrutu Could we give a try to the Github feature that @jpkrohling commented about in #1339 (comment) ?

The auto assign Github Action we use seems to have some issues (e.g. PR assignment is not balanced across approvers and it is sometimes a bit slow) and it could help fix this issue

[jpkrohling]

Folks, it's becoming really hard to keep the sanity while working with OpenTelemetry. Right now, I have 148 notifications in my queue for opentelemetry-collector and opentelemetry-collector-contrib, and it's been only one business day since I had my queue cleared. I doubt they are all relevant. In fact, I doubt even 20 of them are relevant to me.

I'm really about to resign as approver, as triaging this is consuming too much of my time.

[mx-psi]

I echo the concerns above, it is very time-consuming to go through the notifications even for some one who is an approver only for the contrib repo. It should be easy to at least try the "Code review assignment" feature so that we are able to manage this better.

[anuraaga]

I'd appreciate if we could try out the code review assignment technique too, if it works can close #3337 without doing anything as it would probably solve my current notification overload issue.

[jpkrohling]

Yesterday before leaving for the day, I had no notifications. This morning:

[tigrannajaryan]

@jpkrohling sorry, just saw your proposal above about auto-assigment. So, instead of assigning all approvers to each PR which will round-robin PRs?

[jpkrohling]

It can use round-robin or load balancing so that folks with an already big review queue won't keep getting new items to review.

https://docs.github.com/en/organizations/organizing-members-into-teams/managing-code-review-assignment-for-your-team

[tigrannajaryan]

@jpkrohling does that also stop the notifications?

[bogdandrutu]

I did this in the collector (not contrib) for the moment to test it.

[jpkrohling]

does that also stop the notifications?

My understanding is that the notifications are sent because the group is assigned to review the PR. If the assignee is now an individual, the notification should stop as well.

[bogdandrutu]

Changed contrib as well, we need to make a decision about the auto-assignment script, which runs in parallel and may assign a different person as Assignees.

[jpkrohling]

I just looked at a recently opened PR (#3701) and noticed that it got assigned to the team, instead of to an individual. The interesting part is this:

Clicking on "code owners" led me to this:

opentelemetry-collector-contrib/.github/CODEOWNERS

Line 23 in 73753b7

exporter/datadogexporter/ @open-telemetry/collector-contrib-approvers @KSerrania @ericmustin @mx-psi

Apparently, we might need to remove the group from the codeowners.

[jpkrohling]

Changed contrib as well, we need to make a decision about the auto-assignment script, which runs in parallel and may assign a different person as Assignees.

I see the assigned person as the one responsible for making sure the PR is moving (reviewed, up to date, merged). I wouldn't mind being assigned to issues in addition to get review assignments.

[jpkrohling]

The latest changes did have an effect, and I'm now getting manageable levels of notifications! Thank you!