Reinforcement Learning and Bipeds

Traditional matrices and path planning have stuck me with opportunity when considering biped motion.

I recently found a great tutorial with Matlab, reinforcement learning, and trying to teach a biped to walk:

https://www.mathworks.com/help/reinforcement-learning/ug/train-biped-robot-to-walk-using-reinforcement-learning-agents.html

In my own biped design(BASE3), I have been considering an inverted knee design like an Emu (https://ir.library.oregonstate.edu/downloads/g445cj94b, http://ames.caltech.edu/reher2019dynamic.pdf) for energy efficiency.

To that end, I have tried reconfiguring the sample model to invert the knee. In running the simulation on my own Matlab license, I found that I bumped up against the arbitrary 1,000 non-virtual block limit (blocks are the main elements used to build models in Matlab). In fact, I was just 60 or so blocks over. I did not feel like upgrading to a professional license. Instead, I opted to reduce the four-point feet to simply point feet.

I never actually could get the model to take more than a step or two, but did learn the importance of ankles in balance!