A breakthrough development in Deep Reinforcement Learning

Image by Vasundhara Srinivas on Unsplash 

 

Dr Natalia Ares and a team from Oxford and the University of Basel have been working on a machine learning approach which could significantly accelerate the development of quantum technologies by a breakthrough in the application of Deep Reinforcement Learning (DRL); a machine learning algorithm for decision-making, based on neurobiology, which teaches the computer to learn from its actions and rewards - in much the same way humans learn from experiences.

DRL is already used by a diverse range of industries, from finance to healthcare.  In their paper 'Deep Reinforcement Learning for efficient measurment of quantum devices' in NPJ Quantum Information, Dr Ares and her team successfully demonstrate that the machine learning approach can be harnessed to navigate complex parameter spaces using an automative decision process.  The DRL algorithm informs these decisions, and in their experiement, focus was placed on double quantum dot devices, for which they demonstrate the fully automatic identification of specific current features (called 'bias triangles').  It is usually difficult to automate such a task, because bias triangles are found in featureless regions of the parameter space, however, the Ares algorithm identified bias triangles in a mean time of no more than 30 minutes (and sometimes as short as 1 minute); the DRL agent probes the quantum chip, and in the light of the results makes a decision in real time within a set of actions.

The potential of DRL algorithms was demonstrated by playing games of strategy; the computer analysed all possible moves in games of 'Go' within seconds, and won every time against its human competitors.  In the work by Dr Ares' team and collaborators, the DRL algorithm succeeded in making successful strategic decisions in the measurement of a quantum chip controlled by voltages.  The possible actions for the agent are: increase or decrease voltage 1, increase or decrease voltage 2, and increase both voltage 1 and 2 or decrease voltage 1 and 2.  The Ares algorithm is able to play the game and devise efficient policies to obtain desired outcomes with 10 times less measurements.

The application of this approach heralds a new era in computing; although machine learning has already been identified as a method to advance the control of quantum chips (the building blocks of new quantum technologies), and to be robust against noise and stochastic processes present in many physical systems, harnessing DRL for the efficient measurement of quantum devices is currently unexplored.  Dr Ares and her team propose DRL for the efficient measurement of a double quantum dot device; which when embedded within an efficient algorithmic workflow results in a significant reduction of the measurement time when compared to current methods.

"Our findings could also be applicable to the real time control of other experiments, accelerating science discovery"

(Dr Natalia Ares)

The next stage for Dr Ares and her team is to automate and optimise all the necessary characterisation and tuning steps required to turn a quantum chip into an operational quantum circuit.  This is an unfeasible task for humans due to the increasing complexity of the circuits, but machine learning is already assisting with addressing this challenge.

As Dr Vu Nguyen says:"we are super excited to develop a reinforcement learning algorithm which assists experimentalists and automates the way to perform laboratory experiments".