- An implementation of Short Horizon Actor Critic (Xu; 2022) writen in Jax
- Simulation using the Mujoco MJX simulator
Run Time: 1 min jit, 2 min training
Known Issues: For some random seeds, you get drift in the cart position.
Run Time: 1 min jit, 2 min training
Known Issues: As seen in the rewards figure, training can be unstable.
- Having great difficulty applying SHAC to get Anymal to walk.
- Hypothesis: it's because quadruped gait is very contact-rich, leading to uninformative gradients.
![](https://private-user-images.githubusercontent.com/22626914/311497820-d774bea2-ef44-4370-8b77-b84594e780a4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQzODEwOTgsIm5iZiI6MTcyNDM4MDc5OCwicGF0aCI6Ii8yMjYyNjkxNC8zMTE0OTc4MjAtZDc3NGJlYTItZWY0NC00MzcwLThiNzctYjg0NTk0ZTc4MGE0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODIzVDAyMzk1OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZkYjM1ODY4NTkwZDRiM2JjNWNlOTc5MDA5OWQ0YjIxYTdhY2NlMDY3ZGZkNDQ5NjFmNjFlMDAzYjEzYzU2N2MmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.AZ7MBVMxy-tXmu0BZPbEo7STnzxbZn6c4rGOOlIJHr4)
![](https://private-user-images.githubusercontent.com/22626914/311497834-6262b083-a2dc-4402-ac2e-0d25d76f5cb4.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQzODEwOTgsIm5iZiI6MTcyNDM4MDc5OCwicGF0aCI6Ii8yMjYyNjkxNC8zMTE0OTc4MzQtNjI2MmIwODMtYTJkYy00NDAyLWFjMmUtMGQyNWQ3NmY1Y2I0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODIzVDAyMzk1OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk0OTIzNjRhOTllZTNiNWIzYWE5NjAwNjJmNmFhMDFiZGI1YmExODUwN2ZmMmYxZjg5Mjg5NTgxOWU0Zjg5MDAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.XC0WEOUwtFc7ByLPxKaWb_sNJ1XPZzzEbM0F4QlZE3w)
![](https://private-user-images.githubusercontent.com/22626914/311497891-a939d83f-2075-4866-8a7a-0893ef892fdf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQzODEwOTgsIm5iZiI6MTcyNDM4MDc5OCwicGF0aCI6Ii8yMjYyNjkxNC8zMTE0OTc4OTEtYTkzOWQ4M2YtMjA3NS00ODY2LThhN2EtMDg5M2VmODkyZmRmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODIzVDAyMzk1OFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTg3OWJkODdjYzI1N2JkMTBmYjdkOGMyMjM0MGMyZjU2ODg2NzBkNzU3NmQ4MjVhMmQwN2UxYzQzYWYwNWY4YjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.NEvBWzzwnYTyw6HyAkhWk1YKOWbvyGu6x8Xj08wxulg)
32-step rollout. Ground flashes red when step jacobian is greater than 10e2.
- pip install -r requirements.txt
- Add the parent folder of this repository to your PYTHONPATH environment variable.