Interview with Pedro Domingos, the inventor of Markov Logic Network
When “The Master Algorithm” came out in 2015, it acted as a window into the field of AI and machine learning, proving to be a great introduction, especially to the “outside” world. It was authored by Pedro Domingos, a pioneer in the field and someone who has lived through the evolution of the field, from the early 90s to the present day in the 2020s. In a career that spans over three decades, Domingos has received several accolades, including the SIGKDD award, which is widely considered the highest honor in data science. He is also credited with the invention of Markov Logic Networks.
Analytics India Magazine was in conversation with Domingos recently, talking about topics ranging from the evolution of machine learning, his book(s), to AGI and Metaverse!
Evolution of AI
“When I was younger I came across this book on AI. It made me curious about machine learning – if you could make it work, it had the potential to take over the world. More later when I decided to get an official degree in this field there were hardly any colleges offering such courses.The field was very primitive which made it very difficult to make meaningful contributions unlike more mature fields like physics or biology. The University of California at Irvine was one of the few colleges that offered substantial machine learning courses, and that’s where I got a Ph.D.” he said.He counts Geoff Hinton and Tom Mitchell among his first heroes on the pitch.
Times have changed dramatically since then. As Domingos points out, the field of machine learning is changing dramatically and expanding. The other thing that marks the evolution of the field over the past three decades is the development of a whole new industry for AI. “If you look at the top ten companies in the world, seven of them would say that AI is essential to what they do,” Domingos pointed out.
While advancements in AI are encouraging, Domingos points out that there is a downside. “It becomes impossible to follow the terrain. It’s good that we have a thriving industry because of its impact factor, but on the other hand, it becomes much more difficult to change. We will have to fight this inertia,” he said.
The master algorithm
“I thought of writing such a book in the 90s with the explosion of data mining then underway. I thought people outside the field would benefit from knowing about the developments. My book was going to be aimed at people who weren’t necessarily working in AI and machine learning. In that sense, it was a challenge because a popular science book can’t be a list of topics and should instead contain an element of storytelling. And I didn’t know at the time what that story would be,” Domingos said.
Almost two decades later, two things persuaded him to finally write this book: the big data boom and the lack of knowledge about it that was causing costly mistakes. “There was a sense of urgency,” he said.
“What occurred to me was that machine learning is a quest for the ‘master algorithm.’ Such an algorithm can learn anything. So I introduced the book as a quest for a such a master algorithm. Through this book, I take people on a journey through the different approaches of different “tribes” as I would like to call them. These tribes are different schools of thought when it comes to machine learning. It is interesting to note that each of them thinks their path is the only right path when in reality all of them, individually, are just one piece of the puzzle. We need to find a unified theory of machine learning, building a standard like there is in physics or biology,” he explained. The five tribes of machine learning are Symbolists, Connectionists, Evolutionaries, Bayesians, and Analogies.
When asked which tribe he aligns with, Domingos replied, “I don’t align with any of them, which is unusual. But that’s also part of the reason I wrote this book, unlike someone else who thinks his paradigm is the only way. My doctoral dissertation was about unifying these paradigms, and much of what I’ve done throughout my career has revolved around the same. I strongly believe that we need to combine ideas from each of these paradigms to truly have a general-purpose learning algorithm. This is more apparent and true today than ever before.
On ethics and governance
“AI is a powerful technology, and like any other powerful technology, it creates a lot of problems. And when something starts to have a lot of impact in the real world, a lot of people start wanting to have a say in it – like politicians and those in other fields. It’s not surprising but still concerning. A lot of people who don’t understand AI very well are trying to make big decisions about it,” Domingos said.
“Westerners who were brought up on the Bible in Genesis are always worried that creation will turn against the Creator – will the machines turn against us – which is nonsense, but it gets so aired in Hollywood movies, for journalists etc. On the other hand Libertarians on the other hand are very concerned about freedom, fairness and equality. Often the concerns are well-meaning but misguided. Then we continue to solve a problem that didn’t exist in the first place. Let’s not jump the gun; trying to regulate something before you understand it is usually not a good idea,” Domingos said.
He further added that with strict data laws and regulations like the GDPR, the European government is more “happy with the trigger” but calls America a bit more “sensitive” in this regard, with more laws. lenient, with the exception of legal requirements on algorithms that make decisions, especially in the field of medicine.
Interestingly, not too long ago, Domingos published his views on the recently introduced “ethics review” section of papers submitted to the NeurIPS conference. “Since when have scientific conferences been tasked with policing the perceived ethics of technical papers?” he then tweeted.
He still holds his ground. “It could be a research paper on how to speed up an algorithm, but the authors should now have a discussion about the ethical consequences of making an algorithm fast, and that doesn’t make sense.”
Today, AI is intertwined with society and affects real people. What are Domingos thoughts on where should the line be drawn; on this, he said that regulations could be imposed on AI applications instead of the whole technology. “For me, trying to regulate AI is like trying to regulate mechanical engineering; what is the ethics of mechanical engineering? It does not mean anything. There are specific applications of machine or mechanical engineering that involve ethics and require regulation. And it’s the same with AI,” he said.
Prepare the world for Metaverse
Domingos thinks the idea of the metaverse is a desirable goal; although it seems like a long-term goal, it’s still very compelling. Calling it an extremely difficult problem, Domingos believes that there are several challenges to overcome, such as psychophysics.
The second challenge he talks about is identifying and getting this kind of technology to the right kind of people and industries, which, in turn, would give more resources to conduct more important research. “I think what Google did with Google Glass was a total mistake – they rolled it out as a consumer product even before the technology could fully mature. Such a product would have worked pretty well for, for example , a maintenance company, helping its staff to save a lot of time and resources.