AI's Non-Euclidean Geometry: when gradient descent's shortest path doesn't look like a straight line

Here's why Tensorflow and Pytorch version of "Gradient descent" treats the earth as though it's flat!

The claim is that both should change to using *natural* gradient descent, which takes into account the fact that belief spaces have non-Euclidean geometry to them. AI researchers use KL divergence all the time - they need to acknowledge (and build visual intuition for!) the geometry that KL divergence gives rise to ✊

Here's the link to the webpage where you can play with the gaussian belief space! hamishtodd1.github.io/fep/index.html To use it:
1. Click somewhere on the belief space to lay a point
2. Click somewhere else to lay another point
3. Click in a third place to get a third point
4. Press LEFT and RIGHT to get the circles to appear
5. You can also move the points to anywhere you like

Thank you again to Softmax! You can read about their work here https://www.softmax.com/

Скачать видео