Infographic: 2016 Review
Solving ‘Artificial General Intelligence’ by creating literate machines that can think, reason and communicate like humans
As we reflect back on a productive 2016, we have prepared an infographic to highlight our team’s work and achievements. We are on a journey towards creating Artificial General Intelligence and the end of the year provides a good opportunity to reflect on our journey so far.
About two years ago we saw that Deep and Reinforcement Learning research had led to great advances in areas such as vision and speech recognition. We posed an existential question to ourselves: Could we apply these same techniques to solve fundamental problems in language understanding, with the vision of creating a truly literate machine? A machine that could actually read, comprehend, synthesize, infer and make logical decisions like humans
In December 2015 we opened our lab in Montréal, the global epicentre of Deep Learning research. With our advisors Professors Yoshua Bengio (UdeM) and Richard Sutton (University of Alberta) we have established partnerships with leading academic institutions to help build upon and enable further research breakthroughs in the field.
We started out with just two team members and have now grown to more than thirty! We have recruited from around the world building an expert team. Our research scientists and engineers focus on applying Machine learning, Deep Learning, and Reinforcement Learning to Natural Language Processing.
Comprehension and communication
A major focus for our team is on building the skills and techniques to enable machines to interact with humans. Two teams are focused on comprehension and communication.
Our Machine Reading Comprehension team’s focus seeks to create machines that can ingest text and reason against its content. The team has been developing breakthroughs and we have published multiple peer-reviewed papers that highlight our work and achievements throughout the year. Our initial MRC work used simple texts, such as extracts from the CBT and CNN datasets. We showed that our agents were capable of extracting information and demonstrated the ability to predict missing content. More recently we have published our own dataset, NewsQA, the world’s largest human-generated corpus of Question and Answer pairs to date.
This team’s focus in 2017 and beyond is to build in more understanding skills and the ability for machines to synthesise, infer and reason upon text.
Our Dialogue team’s goal is to develop AI agents with human-level decision-making and communication skills. In 2016 the team has developed deep learning and reinforcement learning techniques for goal oriented dialogue systems. Recently, capabilities designed to equip machines with memory - and the ability to handle nuanced conversations that move from topic-to-topic. The team published Frames, a dataset of 20,000 turns, focussed on giving dialogue systems memory and decision-making capabilities.
Developing cognitive skills
Humans have capabilities that help them reason upon and interact with the world around us. We are curious and we seek out information; we like to tackle problems and we use common sense, creative thinking and critical analysis to achieve our goals.
In order to achieve Artificial General Intelligence, machines require the ability to model deeper level thinking capabilities. Our team is exploring and achieving inventive breakthroughs in starting to build these capabilities in machines. Through the work published in some of our papers, we have demonstrated the ability for machines to perform more effectively through separation of concerns and training agents to gather information efficiently.Towards literate machines
We are proud of our team’s work in 2016. We are still at the very beginning of our journey to machine literacy and we look forward to the long road ahead.