You don’t really say this in your article but are you pretty sure that you could’ve gotten the exact same results and probably with better convergence if you just use gradient, descent, and optimizers like Adam, etc?
To be clear I think GAs are way cooler though haha. So kudos to you for this awesome write up
Thanks! I do have a section on this in the article "Why genetic algorithms aren't state of the art"
"Physics simulation involves discontinuities (contacts, friction regimes), long rollouts, and chaotic dynamics where small parameter changes lead to large outcome differences. Even with simulator internals, differentiating through thousands of unstable timesteps would yield noisy, high-variance gradients. Evolution is simpler and more robust for this regime."
"The real tradeoff is sample-efficient but complex (RL) vs compute hungry but simple (GA). DQN extracts learning signal from every timestep and assigns credit to individual actions."
You don’t really say this in your article but are you pretty sure that you could’ve gotten the exact same results and probably with better convergence if you just use gradient, descent, and optimizers like Adam, etc?
To be clear I think GAs are way cooler though haha. So kudos to you for this awesome write up
Thanks! I do have a section on this in the article "Why genetic algorithms aren't state of the art"
"Physics simulation involves discontinuities (contacts, friction regimes), long rollouts, and chaotic dynamics where small parameter changes lead to large outcome differences. Even with simulator internals, differentiating through thousands of unstable timesteps would yield noisy, high-variance gradients. Evolution is simpler and more robust for this regime." "The real tradeoff is sample-efficient but complex (RL) vs compute hungry but simple (GA). DQN extracts learning signal from every timestep and assigns credit to individual actions."
DQN likely would have handled this much better.