For the third year in a row, InstaDeep’s latest research findings across Deep Learning and Reinforcement Learning have been accepted to the Advances of Neural Information Processing Systems (NeurIPS) conference. The annual event aims to foster an exchange of research and neural information processing systems, and with no less than one paper accepted for the main conference, four workshop paper acceptances, and participation in the Flatland NeurIPS competition by providing baseline materials, InstaDeep’s contribution in 2020 is greater than ever and we are pleased to share an overview below.
Multi-Agent RL at Main Conference
The core focus at NeurIPS is peer-reviewed research which is presented and discussed in the general session alongside invited talks by leaders in their field. Additionally, the conference supports a range of workshops where further research is presented.
InstaDeep’s paper A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning is accepted to the main conference and will be presented on Thursday 10 December in Poster Session 6. This research’s main aim is to evaluate the use of empirical game-theoretic analysis as a viable tool for interpretability of practical networked MARL systems used for common-pool resource management e.g. water. The collaborative effort between Stellenbosch University, Wits University, University of Cape Town and InstaDeep contributes new insights into the consequences associated with certain design choices and provides an additional dimension of comparison between systems beyond efficiency, robustness, scalability and mean control performance.
The paper’s lead author, InstaDeep’s Arnu Pretorius, also made history with this research by being the only African-based researcher to have two first-authored papers accepted at NeurIPS. Read about the impressive feat here.
Workshop presentations
InstaDeep’s further contributions for 2020 are presented as part of specific workshops. These are:
- Designing a Prospective COVID-19 Therapeutic with Reinforcement Learning (Arxiv)
Workshops:- COVID-19 Symposium, spotlight talk. Date: 8-9th December
- Machine Learning for Structural Biology. Date: 11th December
- Machine Learning for Molecules. Date: 12th December
Following InstaDeep’s recently announced collaboration with next-generation immunotherapy company BioNTech, the joint research paper ‘Designing a Prospective COVID-19 Therapeutic with Reinforcement Learning’ has been accepted for three NeurIPS workshops, including a special invitation to contribute in the COVID19 Symposium. The research focuses on applying InstaDeep’s recently unveiled protein design platform DeepChain™ to address the world-wide Covid-19 pandemic. In the NeurIPS paper, the authors propose a novel approach to protein sequence design by combining computational biology with reinforcement learning. The method is applied to the task of designing a prospective therapeutic drug for treating those infected with Covid-19.
- Learning Compositional Neural Programs for Continuous Control (Arxiv)
Workshop: Interpretable Inductive Biases and Physically Structured Learning W
Date: 11th–12th December
This paper in collaboration with Google DeepMind, Sorbonne University and the French National Center for Research (CNRS), aims to bring more structure in robotics skills learning to improve interpretability and sample efficiency. The researchers propose a novel compositional approach to solve robotics manipulation tasks. By formalising the tasks to be mastered as programs, such as computer programs, that are learnt thanks to the Neural Programmer Interpreters (NPI) architecture, the agent learns first simple programs and then to compose them together to master more complex ones. In the paper, led by InstaDeep’s AI Research Engineer, Thomas Pierrot, the team experiments in an environment where the agent controls a robotic arm that faces four blocks with numbers and colours on a table. Through this approach, the research shows that the agent learns to master 20 simple programs such as stacking a block on top of another or moving a block to a specific zone. It then learns to compose these together to learn eight more complex ones such as stacking all the blocks together with colour ordering.
- Offline Reinforcement Learning Hands-On (Arxiv)
Workshop: Offline RL
Date: 12th of December
This research’s main aim is to provide insights to help practitioners build their intuition on today’s offline RL methods and their applicability in various settings. This is important as by taking advantage of large pre-collected datasets, offline RL can mitigate the technical concerns associated with online data acquisition, as it is often expensive or dangerous to interact with real environments. Over the last few years, a considerable amount of algorithmic solutions has been published in an attempt to solve the issues that arise when learning fully offline. However, the fast publication pace ends up being confusing for non-experts. In this paper we attempt to bring some clarity to practitioners in the field by benchmarking some of the most promising recent methods on a set of custom experiments that are designed to be clear and informative. Since the performance of an offline RL agent is fully dependent on the dataset available, we also explore some characteristics that make a dataset suitable for offline RL.
- Differentiable Histogram with Hard-Binning
Workshop: Black in AI
Date: December 7th
This initiative aims to train a model to output a sample distribution that is represented by a histogram, a simple yet powerful statistical tool for generating an approximate distribution of numerical data. However, it is not differentiable, hence cannot be used to teach a model that generates values that follow a distribution of interest. In this work, the team, led by InstaDeep’s Junior AI Researcher Ibrahim Saliu Yusuf, derived a functional form of a histogram which is differentiable and can be used to teach a model that generates values that follow any distribution that can be approximated by a histogram. The proposed differentiable functional form of the histogram was implemented using a stack of convolutional layers to create a differentiable histogram layer.
Workshop co-organiser
On the note of workshops! During NeurIPS 2020, InstaDeep’s AI Research Engineer Femi Azeez is also part of the organising committee for the workshop ‘Machine Learning for the Developing World’ (ML4D), a workshop focusing on how machine learning tools can help enhance preparedness for disease outbreaks, address the climate crisis, and improve countries’ ability to respond to emergencies. It will be hosted on Saturday, December 12th, more information on the session is available here.
Flatland baselines
In addition to the talk and workshops, NeurIPS is hosting a range of challenges, including a multi-agent Reinforcement Learning train optimisation challenge to tackle a major problem in the transportation world, namely how to efficiently manage dense traffic on complex railway networks. Following InstaDeep’s extensive experience in the field, we are contributing to the challenge by creating the baselines for the competition in partnership with Deutsche Bahn. It is a true honour to utilise our collaboration and results to date with the German railway company to facilitate transfer-learning through NeurIPS 2020. Read more about it here.
This years’ NeursIPS is the 34rth Annual edition and will be held virtually between 6-12th December. Click for more details and the full program.
