Humans in the loop

2 August 2021 | Physical Sciences/Engineering


Computer circuit board

Machine learning (ML) is a data-driven form of Artificial Intelligence (AI) that enables a computer to replicate (and in some cases exceed) human decision-making. It’s used in everything from Netflix recommendations to credit decisions and even analysing medical images. But developing a machine learning model often requires extensive data annotation by a human operator, creating a major drain on time and resources. A UCLB spinout is helping companies to train ML models more quickly, by enabling computer systems and human experts to cooperate more efficiently.

Humanloop was created by UCL Professors David Barber and Emine Yilmaz with PhD students Raza Habib and Peter Hayes, based on research into AI uncertainty at UCL’s AI Centre. They were joined by entrepreneur and scientist Jordan Burgess. The team combine deep technical expertise with experience of working on major AI projects for the likes of Google, Microsoft and Amazon, as well as building other start-ups.

A bottleneck in adoption of ML

Through working with industry, the founders noticed that data annotation was slowing down the implementation of ML systems. Whereas humans can learn new concepts from just one or two examples, most AI systems need thousands of examples to reach adequate levels of performance. These data often need to be manually labelled by subject-matter experts, making it time-consuming and expensive.

“The range of applications for natural language understanding has grown rapidly in the last couple of years,” says Raza. “With that, the problems of data scarcity and annotating data have grown as well. I’ve met bankers, lawyers, doctors – really expensive, talented people inside companies labelling data for chatbots. We wanted to use some of our research to solve that.”

Teaching software to learn from humans faster

Humanloop uses probabilistic deep learning, which combines the understanding of uncertainty with machine learning, extracting patterns from unstructured data. Human subject-matter experts annotating data are incorporated into the training loop of the models only when the model decides it needs to increase its certainty for a particular knowledge space. This dramatically reduces the volume of labelling. The examples most likely to increase the model’s overall prediction certainty can be prioritised, and because assessment of the model’s learning is happening in real time, training can be stopped when the appropriate level of certainty is reached. Furthermore, new data can be added at any time to keep the model up to date.

The system means it’s faster and easier for companies to start getting the benefits of AI, explains Raza. “You can get things into production much quicker. Where previously it might take months or even a year to get a model up and running, with Humanloop this can be done in days.”

Game-changing technology

Humanloop has been attracting considerable interest from those in the know. The company was spun-out with assistance from UCL Business (UCLB), as part of UCLB’s Portico Ventures scheme which offers a simple framework for setting up high-growth technology-based businesses. In return for a small fixed-equity stake, UCLB grants an exclusive licence of any relevant software and know-how developed in the university. Portico Ventures spinouts are eligible to apply for investment from the UCL Technology Fund (UCLTF) and benefit from an ongoing association with UCL more broadly.

Professor Yilmaz says, “The advice and support from UCLB and UCLTF particularly in the early stages has been invaluable, and the Portico Ventures IP licence gave external investors confidence from the outset in the clear relationship between our spinout company and the university. UCLB has an excellent reputation in supporting spinouts with a clear framework, equitable terms, a strong business network and a can-do attitude.”

Marina Santilli, UCLB’s Associate Director for Physical Sciences and Engineering was quick to spot the company’s promise. “It was clear very early on that there was real opportunity in tackling the problem of labour-intensive labelling by applying the know-how developed by Professor Yilmaz and Professor Barber during their research. It also was a perfect fit for the Portico Ventures model,” she says. “The Humanloop founding team not only has real expertise at the deep science level, but also understood the need to talk to many businesses deploying ML solutions to make sure their product would meet the most pressing operational and financial needs of these companies.”

Her enthusiasm was shared by UCL Technology Fund (UCLTF), which is managed by AlbionVC in collaboration with UCLB. Recognising Humanloop’s potential, they made an early investment to get them up and running.

David Grimm, Investment Director for UCLTF said, “We were thrilled to support a highly talented team to create what could be a game-changing technology for improving performance in AI.”

Scaling up and solving new challenges

Acceptance onto the 2020 cohort of Y Combinator, one of the world’s most prestigious business accelerators, helped the founders define their offering and network with potential investors.

Humanloop brought a first version of their product to market in September 2020. Already it’s been used by several companies to automate tasks from content moderation to contract classification and social media monitoring.

They’ve been working with a small number of customers to develop and test the product. The next stage will be to distribute it much more widely, so that more people can get the value of what it offers, as well as turning their attention to new research challenges.

“We’re trying to build a company culture that’s still deeply research-oriented, but also has a product focus,” says Raza. “The ultimate goal is for a knowledge worker, such as an online moderator or lawyer, to teach a computer to solve some task by explaining to them exactly like you would teach a colleague. That’s the vision of the company.”

Further information: 


Portico Ventures

UCL Technology Fund