Button press reward functions also reward pulling on button

[krzentner]

This rarely matters, but button-push family of tasks also reward pulling on the button. This makes it possible to get very high reward without ever succeeding at the task, which should probably be fixed.

Fortunately this rarely matters in practice, since most RL algorithms never attempt to pull on the button.

[reginald-mclean]

@krzentner any ideas on ways to fix this?

[krzentner]

I saw this happening in one of my experiments, but it's not obvious looking at the code what is wrong. The first step to fixing it would probably be reproducing the issue (perhaps by making a modified scripted policy that triggers the bug). The relevant line of the reward function is here.