Rants on AI and ML - 2

Essay: One Example Where Intuitive Learning Does Not Work And What It Might Mean

Since last year, Google AlphaZero's clone chess engine, the Leela Chess Zero (Lc0) has held a consistent spot in the top ten positions of Top Chess Engine Championship (TCEC). Strangely, Lc0 is a neural network (NN) based chess engine which DID NOT make use of the human knowledgebank of chess. Since early 2019, SugarNN, based on Lc0, has held the top spot in TCEC defeating traditial (commercial) engines like Komodo and Houdini, which draw heavily from human chess knowledge and experience.

When one learns chess, he's taught many "strategic rules": bishops are best placed on long diagonals, pawn pushes have to be carefully planned (they cannot be taken back), and intuitions: queen is strong in the endgame, two minor pieces roughly equal a rook and a pawn, etc. Furthermore, there's a consistent story running in the mind of a chess player consisting of events, plans, and tactics. Every move has a "meaning." This is why chess commentators can present a good overview of the possible thoughts on a player's mind -- a story for the game.

But when it comes to games by Lc0, most commentators are just dumbfolded and struggle to explain the rationale behind its moves. It is as though the game has no story -- it seems completely random. Yet, miraculously, Lc0 always manages to win! (Of course there's a predictable algorithm behind Lc0, but it's perhaps too complex to be woven into a story.)

This might shed some light on the human quest for knowledge. Even in a concrete field like engineering (my field), we do not operate in the "real world." Instead, all our research, theories, and explanations are in an "idealised world" with "good properties." In this idealised world, a piece of theory is like a story -- it builds on existing stories and extends them in a meaningful and intuitive way. Luckily, when applied to the real world, the idealised theories work well. But they're mere approximations -- they're only roughly correct.

Similar to chessplay by Lc0, things in the real world appear random and haphazard to us. When we try to weave a story behind them, it turns out to be a non-story because it's so convoluted. Hence, it is not surprising that "learning based systems" such a NNs, with no intuition and biases and no need for stories, perform much better than our techniques based on our idealised theories. Nevertheless, presently, when we build NNs, we incorporate our biases into them -- in the form of structure and data. Today, without them, NNs don't work well. But, one might intuit, risking failure, that the haphazard approach of NNs is probably better suited to handle the complexity of the real world than organized storytelling offered by idealised theories.

Could it be that we are better off designing systems that lack intuition (because of their complexity) but work in the real world? Such systems, presently based on NNs, have no stories to them. But objectively, they seem to work well compared to the systems that are built based on idealised theories. If so, perhaps in the future this gives us very little room for consistent stories and a lot of room for trial and error and data-crunching based research.

Rants on AI and ML - 1

Essay: Are we right in requiring neural network explanations?

One day many centuries ago, a man looked up at the sky and thought to himself -- "what if" the stars, sun, and the moon didn't control his life and fate (horoscopes)... what if stars and planets were beings with their own laws and lifecycle independent of humankind. This person, ladies and gentlemen, brought about a revolution in thinking -- he wanted to "know" the world he is living in. He wanted to "understand" it. And not merely be a part of it.

It is not surprising that a famous scientists when asked "Imagine the whole human civilization collapses and you could only transmit ONE message into the future, what would this message be?" said the message would be "The Universe is knowable," meaning that Universal laws can be found. That Universe does not run on magic.

Human curiosity and the hunger for knowledge is of immense importance in the saga of humankind. Scientific "explanations" have been a driving force behind engineering and sciences.

But, does it seem like our pet topic -- machine learning -- betrays explanations? At first, it seems so. It seems as though something "magical" is happening within these "black boxes" of neural networks (NNs). But imagine this -- if there were an algorithm which chose a "best-fit function," from an arbitrary list of functions, between inputs and outputs, would we ask this algorithm to explain its choice? No. We know how it works. Surprisingly, this is the case for NN as well.

However, the problem, it seems, is that we associate very personally with NNs. We think of NNs as a model of our brains rather than mathematical entities. Therefore, we require "explanations" from them as we would desire from our fellow human beings. In fact, NNs emerged from brain research and initial NN models were heavily inspired from brain structure. Hence, it is not strange that we associate personally so with them.

Furthermore, it is not even clear why such explanations for the operation of NNs, which are made to solve problems that are extremely complex to be solved using our current thinking tools, should even be possible.

I feel our energies are better spent in understanding NNs rather than require them to behave like us. Perhaps by asking for explanations we fall into the same trap as our ancestors did -- associating special meanings to stars and constellations rather than seeking to understand them.

Perhaps we do justice to our human endeavour by trying to understand these crucial aspects rather than trying to make NNs behave like human beings.