Software to scan phones for abuse images is itself open to abuse

‘Client-side scanning’ (CSS) is an image analysis technique based on ‘perceptual hashing’, and is mooted as a way to get around the legitimate problem of criminals hiding illegal content behind end-to-end encrypted phone apps.

Installed on a phone, CSS works by comparing digital signatures derived from images on the phone with an official database of known illegal images.

In the study, whose findings will be revealed in a paper presented at IEEE Security and Privacy next week, the Imperial team recreated algorithms that underpin CSS.

They then, without adding standard face detection or facial recognition algorithms, taught the existing perceptual hashing software to scan for specific faces in photos on the phone – effectively spying on private content – which it succeeded in identifying most of the time.

This added function was cloaked from detection because the modified CSS software could not be distinguished from the original CSS software.

“What our paper shows, is that the software could be built or tweaked to include other hidden features such as scanning private content from the phones using facial recognition,” said Imperial computer scientist Yves-Alexandre de Montjoye.

The UK is one of the countries considering mandating CSS, through the Online Safety Bill, according Imperial.

“It is our opinion that client-side scanning is not the innocuous single-purpose technology it has been described to Parliament as,” said de Montjoye. “We call on policymakers to thoroughly evaluate the pros and cons of client-side scanning, including the risk of it being abused, before passing laws mandating its installation.”

‘Hidden dual purpose deep hashing algorithms: when client-side scanning does facial recognition’ will be presented at IEEE Security and Privacy next week.