Surveillance
Detecting ‘Aggressive Driving’ With Machine Learning and Edge Computing
A recent patent application has proposed a system for identifying ‘aggressive driving behavior' at junctions using machine learning algorithms deployed in civic edge computing devices.
In contrast to recent innovations of AI research into in-vehicle ‘road rage' analytics (primarily intended for the benefit of insurance companies), the system proposed is instead municipal in nature, and could be aimed at facilitating penalties for drivers that are not conforming to the ambient norms of ‘safe' driver behavior. It is also specifically intended to provide bad drivers with related in-car audiovisual alerts.
The patent was filed at the US Patent and Trademark Office on 29th April 2021 on behalf of the Board Of Regents of the University of Michigan, and the Denso corporation, a Japanese automotive components manufacturer owned by Toyota.
The UMich patent is not a proprietary, in-car system aimed at insurance oversight, nor designed solely to produce forensic data, but rather relies on well-resourced edge computing nodes deployed at traffic intersections to provide immediate and actionable feedback, by collating data from roadside edge computing resources and from sensors installed in nearby vehicles. Source: https://pdfaiw.uspto.gov/76/2021/50/012/1.pdf
The proposed framework is not entirely passive or dependent on civic equipment, but would also require in-vehicle technologies capable of turning cars into contributing sensors for the system. Effectively, this would turn every equipped vehicle into a traffic monitoring node, though the same oversight would also be turned on the driver themselves, with the additional benefit of access to the vehicle's onboard data streams. The schematic for the in-car installation (pictured below right) submitted in the patent includes direct sensor data from the vehicles brakes, throttle, steering wheel and odometer, as well as requiring access to maps data.
Both the roadside and in-vehicle installations feature CUDA-enabled GPU and local storage resources, and both feature cloud connectivity.
Warnings For Aggressive Drivers
According to the patent application, the system is designed to interact with a fractious driver:
‘When the system for predicting aggressive driving behavior 1 predicts aggressive driving in a subject vehicle, a warning message such as the warning in FIG. 8A may be output to the audio visual device 413 in the subject vehicle to warn an aggressive driver to slow down.'
Additional potential warnings include in-car alerts for speed and oncoming vehicles:
Warnings depicted in the patent application. Source: https://pdfaiw.uspto.gov/fdd/76/2021/50/012/0.pdf
The patent says that in another possible scenario, any available audio-visual devices (including smartphones owned by the driver) in the vehicle may be used to provide warnings, including audio warnings.
Driver-Specific Red Lights
Changing the frequencies and behavior of intersection traffic lights has long been known to be a method to reduce accidents, and the proposed system can be used to generate data that may change the way that traffic lights function at intersections at a very broad level.
However, the deep integration of infrastructure and vehicle data envisioned in the Michigan proposal offers another possibility: red lights that are dictated by the behavior of the driver, rather than run by rote or to schedule (see the prolix section [0157] in the patent application).
‘Additionally, if the driving pattern at one intersection (e.g., the intersection 200a) predicts and/or indicates aggressive driving in the vehicle 71, remedial measures such as the control of control elements may be employed at subsequent intersections (e.g., intersections 200b and 200c) to control and mitigate aggressive driving behavior. For example, if the system for predicting aggressive driving behavior 1 predicts that the vehicle 71 may run a red light at the intersection 200a, the system for predicting aggressive driving behavior 1 may output a control signal for controlling the state/phase of the traffic control signals 202b and 202c at the intersections 200b and 200c. In such manner, the traffic control signals 202b and 202c may change to red, for example, before the vehicle 71 passes through the intersection 200a to encourage the vehicle 71 to slow down and not attempt speed up in an effort to drive through intersections 200b and 200c before the signals 202b and 202c change to red.'
Inroads Into Driver Privacy
Proposals for in-vehicle driver monitoring systems over the last decade have exploited emotion recognition computer vision algorithms, together with other biometric indicators, to produce systems designed to have a ‘chilling effect' on obstreperous drivers, since errant driver behavior could affect driver insurance premiums under such schemes, or else be used as evidence in an accident investigation.
By contrast, the Michigan/Denso proposal seems intended for some form of state collaboration through changes in mandatory vehicle standards. However, given the mobility of the US population in normal times, and the fact that such a system would be most applicable for long-distance drivers, such as truck drivers, it is difficult to see how anything less than a federal implementation would be feasible, unless the framework is designed to become inactive over non-participating state borders, or across counties which do not support the scheme.
Though such innovations are generally driven by the increasing trend for insurance companies to reward policy holders willing to install monitoring equipment, the need for civic infrastructure does not exactly fit into that model, suggesting that such a system would need legislative support and federal or state funding.
An illustration of the machine learning architecture proposed for the system, using Recurrent Neural Networks (RNN). Though the framework also uses unsupervised learning and provides real-time feedback, it also makes provision for offline training to improve algorithms based on incoming data and events. The identification of aggressive driving behavior is facilitated by Dynamic Time Warping (DTW), an algorithm used in time series analysis to compare two temporal series or sequences of objects that may vary in speed.
Revenue Stream
The system proposed in the patent could be the first of its kind to offer real-time analysis of errant driver behavior in a civic framework that's capable of automatically issuing fines and penalties, or alerting authorities, in the event of truly dangerous driver incidents. However, the inventors concede that it could be misused as a cash-cow by revenue-hungry municipal authorities.
Neda Masoud, assistant professor at the Department of Civil and Environmental Engineering at the University of Michigan, told The Academic Times that ‘False alarms are a small price to pay for the enhanced level of safety that the technology can provide. That said, there should always be an opportunity for the wrongly accused to defend themselves against any false claims of aggressive driving,'.
The patent application acknowledges the possibility of aggressive driving events involving vehicles that are not equipped with on-board units (OBUs), which can be addressed by external observation of adjacent or nearby vehicles that do have the equipment, as well as the intersection installations. In such cases, identification would presumably default to other methods, such as automated license plate reading (though the patent application does not deal with this).
Intersections A High Risk
The Michigan patent addresses reckless behavior at intersections, since these are a locus of traffic incidents and infractions, while previous Chinese research along similar lines has used support vector machine (SVM) analysis to identify in-transit events, such as dangerous lane-changing. Another China-based research initiative exploited sensors in smartphones to detect erratic in-journey behavior.
The US National Highway Traffic Safety Administration estimated in 2010 that 40% of accidents that occurred in the United States in 2008 were related to activity at intersections.
