Engineers Create AI Railroad Trespassing Detection Tool

A team of engineers at Rutgers has developed an AI-enabled tool that can detect trespassing on railroad crossings, helping reduce the increasing number of fatalities taking place over the past ten years. 

The new research was published in the journal Accident Analysis & Prevention. 

Automatically Detecting Trespassing With AI

The team consisted of Asim Zaman, a Rutgers project engineer, and Xiang Liu, an associate professor in transportation engineering at the Rutgers School of Engineering. The pair developed an AI-aided framework that automatically detects railroad trespassing events. It also differentiates types of violators and generates video clips of the instances. The AI system relies on an object detection algorithm to process video data into a single dataset. 

“With this information we can answer numerous questions, like what time of day do people trespass the most, and do people go around the gates when they are coming down or going up?” said Zaman.

There has been a consistent rise in trespassing accidents in the United States over the last few years, with each year seeing hundreds of people killed. There have been many efforts to reduce these fatalities, but nothing has worked yet. 

The Federal Railroad Administration (FRA) estimated back in 2008 that around 500 people were killed annually trespassing on railroad rights-of-way. That number increased to 855 in 2018, according to the FRA. 

Zaman and Liu defined in their research that trespassers are unauthorized people or vehicles in an area of railroad or transit property not intended for public use, or people who enter a signalized grade crossing after it has been activated. 

Previous research in this area has mostly involved data derived from casualty information, but it didn’t take into account near-misses, which Zaman and Liu say can provide valuable insights into trespassing behavior. This could lead to the design of more effective control measures. 

The researchers tested their theory with video footage captured at a crossing in urban New Jersey. One of the problems with video systems at crossings is that they are not consistently reviewed due to the process being labor-intensive and expensive. 

Training the AI

Zaman and Liu trained the AI and deep-learning tool to analyze 1,632 hours of archival video footage from the study site. After 68 days of monitoring, they found 3,004 instances of trespassing, which averaged out to 44 per day. They also discovered that nearly 70 percent of the trespassers were men, and around a third trespassed before the train passed. Most violations took place on Saturdays around 5 p.m. 

According to Zaman, this type of granular data could be used by local authorities to place police officers near crossing during the times of peak violations, or it can help inform railway owners and decision makers of more effective crossing solutions. These types of solutions could include grade crossing elimination systems or advanced gates and signals. 

 “Everyone loves data, and that’s what we are providing,” said Zaman.

“We want to give the railroad industry and decision makers tools to harness the untapped potential of video surveillance infrastructure through the risk analysis of their data feeds in specific locations,” Liu added. 

The researchers are also conducting studies in Virginia and North Carolina. They were recently awarded a $583,000 grant from the U.S. Department of Transportation to expand to other states including Connecticut, Louisiana, and Massachusetts.