Fabrizio Baiardi

Fabrizio Baiardi

Fabrizio Baiardi is a Full Professor at Università di Pisa where he has chaired one of the first degree on security of ICT infrastructures. His main research interests are formal approaches to risk assessment and resilience of critical ICT infrastructures. Fabrizio Baiardi has been involved in the risk assessment and management of several systems and of industrial control systems with SCADA components. He has authored several papers on ICT security and currently teaches university courses on security related topics.

The Illusion of Context Independent Anomaly Detection

An interesting review report from Algorithm Watch points out the complete failure of 3 automated systems in use by the Swiss police and judiciary to predict anomalous behaviour. As an example, one of the systems tries to predicts burglaries from past data, based on the assumption that burglars often operate in small areas. Hence, if a cluster of burglaries is detected in a neighbourhood, the police should patrol it more often to put an end to it. It seems that this analysis is inspired by one of the first big data analysis that revealed a cluster of infected people around infected water fountains in London at the beginning of the last century. Another system in use in Switzerland aims to predict the likelihood that a person will assault his/her intimate partner. The cost of each system is close to 100.000 euros.
 According to Algorithm Watch this failure is hardly a surprise because it has been anticipated by previous analyses. As an example, a 2019 report by the University of Hamburg has reported the lack of any evidence of the efficacy of predictive policing solutions. Furthermore, the false positive rate and the false negative one of these predictive systems both lie in the range 70–80%. Notice this holds even if fully accurate data on the problem of interest is used. Imagine what could happen if some database is poisoned with some carefully designed data.
 The failure of predictive AI points out a too optimistic attitude towards AI techniques and it should ring a bell for all those that believe that anomaly detection per se suffices to discover and signal attacks against an ICT system. Almost all of these systems are deployed under the assumption (hope??) that they can work without information on the target system, the applications and the security policies that have been adopted. The price to pay for the lack of this information are unacceptable rates of false positive and false negative. Experience confirms that these rates are dramatically reduced by taking into account constraints arising because of the system architecture, the interconnection topology and available applications.


More to explorer

Leave a Comment

Your email address will not be published. Required fields are marked *

it_IT en_US