AI Weapons Detection System Concerns

The performance of an AI weapons detection system, installed across multiple US schools and at the Manchester arena, is being questioned after a student was stabbed.

Last year, on Halloween, a student at Proctor High School in Utica, New York, was repeatedly stabbed by another student, armed with a hunting knife. The victim suffered multiple stab wounds to his head, neck, face, shoulder, back and hand.

This brutal attack triggered an internal investigation into exactly how it was possible for this weapon to have been brought onto school premises and why it had not been detected by the state-of-the-art AI based Evolv Express weapons detection system, installed at great expense ($4 million) across 13 schools in the Uttica City School district. Brian Nolan, the local superintendent of schools in Uttica stated that:

“Through investigation it was determined the Evolv Weapon Detection System… was not designed to detect knives,”

What is the Evolv Express Weapon Detection System?

Evolv technology, who provided the AI based system used at this school, state that:

“The Evolv weapons detection system combines powerful sensor technology with proven artificial intelligence (AI), security ecosystem integrations, and comprehensive venue analytics to ensure safer, more accurate threat detection at an unprecedented speed and volume.”

The company promotes their weapon detection system stating that unlike traditional metal detectors their Evolv Express system uses advanced sensor technology and artificial intelligence to screen guests as they walk through at a natural pace, without stopping and without handing over their belongings. They highlight how their system is proven to operate up to ten times faster than traditional metal detectors and will alert operators to the presence of weapons (guns, knives, bombs) while recognising and ignoring personal items such as keys, coins and cell phones.

Evolv CEO Peter George has denied their system is like a metal detector, stating that:

“We’re a weapons detector, not a metal detector,”

But after the stabbing in New York the manufacturer quickly changed the promotional message presented from their website. Previously they had boldly claimed that using their system resulted in “Weapons Free Zones” but this claim was changed to “Safe Zones” and this has now been changed again to “Safer Zones.”

How Does AI Weapon Detection Work?

Evolv Technology are purposely vague about exactly how their system operates. The company CEO Peter George has defended the lack of transparency via a blog post in which he has said:

"It is well understood in the security industry that sharing details of a security system or operation puts people at risk"

"Marketing weapons detection security requires a delicate balance between educating stakeholders on new technology and not providing bad actors with the information they could use to do harm,"

"So, while public-facing marketing materials are intentionally not specific, we communicate all aspects of the Evolv Express system - including limitations and capabilities - with the trusted security professionals at our customers, partners, and prospects".

Artificial Intelligence (AI) is now pervading many areas of our daily lives. AI technologies are used in medicine, manufacturing, transportation, robotics, science, education, and much more. And AI technology has enabled the development of advanced algorithms that can automatically identify various dangerous items. The Evolv Technology CEO has repeatedly claimed their system scans for “signatures” for “all the guns, all the bombs, and all the large tactical knives” in the world.

AI Weapons Detection System Fails

The horrendous knife attack at Proctor High School in New York was not the first time the Evolv weapons detection system had failed. Earlier in the same month a law enforcement officer had walked right through the detection system at the school twice while in possession of his service revolver. Evolv Technology advised the school to increase the system sensitivity, which they did. This resulted in a student’s lunch box being detected - but not the hunting knife subsequently brought into the school and used by the violent attacker.

In another case, less than 3 months after Mifflin High School in Ohio installed the Evolv AI based weapons detection system a knife fight erupted between students who had passed through the scanners without their weapons being detected.

In December 2022 it was revealed that the National Center for Spectator Sports Safety and Security, a US government body, completed a confidential report showing that previous field tests on the scanners failed to detect knives and a handgun. And according to IPVM, a leading authority on physical security, although Evolv claimed the report was “fully independent” these known failings had been removed from the published results and there was no disclosure that the company had paid for the research. Evolv Technology has claimed the public version of the report had omitted information for security reasons.

A number of law firms have now announced investigations of Evolv Technology, looking into claims they have misrepresented their technology and its capabilities. Over-promising the capabilities provided by artificial intelligence based systems is recognised as an industry-wide problem in security.

BBC Investigates System Performance

The BBC has highlighted how independent security experts have expressed doubts about the claims made by Evolv Technology. A Freedom of Information Request, made by IPVM and shared with the BBC, revealed the private report in which system weaknesses were recorded.

For example, the report gave the Evolv system a knife detection score of just 1.3 out of 3. In a test involving 24 walkthroughs the Evolv Express system failed to detect large knives 42% of the time. Conor Healy, director of government research at IPVM, has said:

"For certain categories of knives, the system didn`t detect them at all when they were brought through. And that completely conflicts with what Evolv has told the public."

Security at Manchester Arena

On the 22nd of May 2017 a bomb at the Manchester Arena in the UK killed 22 people and injured 1,017, many of them children. The blast destroyed the venue’s foyer causing an estimated £28 million worth of damage.

ASM Global, which owns Manchester Arena, has partnered with Evolv Technology to fully rollout their Evolv Express security screening system at the venue. ASM Global director of safety security and risk Gary Simpson said:

“The detection technology has been used for some time in America but this is the first such deployment at an Arena in Europe. Given the positive experience at the AO Arena, Manchester we are planning a further roll out to other venues in Europe as part of our VenueShield programme.”

AI Based Security System Concerns

Airports with maximum security protocols are known to have reservations about the Evolv Express system. It has been acknowledged that the system is not an explosives detection machine and if a person walked through the scanner carrying a brick of C4 explosive the Evolv system will not raise an alarm.

Since the Evolv Express system has now been in use across a number of US schools for some time, valuable field data has been accumulated. For example, in Dorchester County Public Schools in Maryland the system had 250 false alarms for every real weapon detected over a period from September 2021 to June 2022.

Evolv Technology has told BBC news that they have advised venues of all “capabilities and limitations” of their system.

Dr. Marion Oswald, of the UK government`s Centre for Data Ethics and Innovation, told BBC News:

"There needs to be more public information and more independent evaluation of these systems before they are rolled out in the UK."

"At the end of the day, they are potentially replacing methods of metal detection and physical searches that have been tried and tested."

For more information on AI Weapons Detection System Concerns talk to Insight Security