Computer Vision Solution for a Large Industrial Client

Download Case Study

96.0 KB

Employee safety is an important focus of every industrial plant which makes it essential for companies to track the compliance of personal safety. Many organizations are putting forth a significant effort into implementing automation systems that use AI to track and document whether employees are complying with personal safety requirements. This is a complex effort which involves many studies of AI. First Line Software has already taken part in one of those studies.

The Task

The client requested that First Line use the existing live video streaming capabilities of its CCTV cameras as the foundation for implementing an AI capability to monitor whether the company’s safety requirements were being met. This required that the AI system be able to identify specific plant production lines, employees, and key safety elements.

The Process

The First Line Software engineers created a convolutional neural network and taught it to identify the specific staff members, safety equipment elements (helmets, safety jackets, etc.) and the production line types, using the live stream from the CCTV cameras.

The pilot version of the system can track and react to the three most common scenarios of human behavior:

Whether the staff is wearing the necessary safety equipment (helmet and safety jackets)
Whether the individual has covered his helmet with a hood (which is strictly forbidden)
Whether an individual has attached a rope when performing high-altitude work

Dataset

The typical problem of this type of research task is the lack of exact cases and the scenarios which can be used to “teach” the neural network. In order to solve this challenge, First Line had to create all the potential positive and negative scenarios of an individual’s behavior. The process also involved analyzing and creating 56 models of human behavior that could be displayed on-site at the plant.

As you can see in the photos, some of the employees were fully equipped and met the safety requirements, while some part of employees did not. The system uses 12 pivot points to analyze each individual and additional pivot points to analyze the safety equipment. Each video frame has a specific color identification as well as the text frame. In addition, the viewer can see the in-plant location trigger.

Object recognition

The algorithm developed by First Line analyzes the data in three steps. The first step is to analyze the CCTV captured frame and analyze the human presence. If a human is detected the algorithm pushes the frame to the convolutional neural network. Once this step is completed the network uses the pivot points to identify the person and the safety equipment elements – helmets and/or a safety rope. Using the “support vector” algorithm, the program compares the captured image of the person with the scheme, which is stored in database. If the algorithm detects that the safety rules have been broken, it transmits a message to the management staff.

The technology

In order to achieve the best results, First Line used Mask R-CNN (Detection Platform) to analyze the captured frames. This framework was chosen because it was the best fit for the required task and can identify and highlight objects with the frames. We taught the neural network using the transfer learning script. This script was chosen since there are no requirements to collect statistics about the completed work or the number of workers at the plant, and no need to identify the specific department where each employee spends the majority of their time at work.

The result

First Line was able to achieve the sustainable analytics of the live video with the ability to detect the objects and classification of the same. The detection ratio varies from 77 to 100 percent. The pilot project showed great results and the customer is currently running in-debt testing on their premises.

Download Case Study

Client References

I want to express my gratitude for all of your team’s hard work, support and extra help. We are very grateful for all that you have done for us.
Senior Project Manager, a U.S. specialty digital design and web development agency

Your quality of work is exceptional, and your flexible approach has allowed us to build and deploy a large amount of complex functionality in a very short space of time. Kudos to everyone for their professionalism and for delivering results that exceeded our expectations.
CEO, leading social media analytics provider

We have never had a better release that we can think of. And that is all because of your hard and dedicated work. So here is a big THANK YOU to everyone from all of us!
Editor, leading Scandinavian newspaper

For a software vendor like ourselves, picking the right development partner is crucial. First Line’s technical expertise and solid project management, coupled with their proficiency in Agile methodologies, have made them an integral part of our product development group.
VP Product Development, leading enterprise software vendor

Our First Line Team has been doing a great job. All of the guys check in great code, and they have a forward thinking mindset, not a feature-to-feature approach. They are also very open in terms of sharing knowledge, which is critical to running productive distributed teams.
Application Development Director, global marketing company

We've all learned a lot from you, and it is great working with you.
Application Development Director, global marketing company

The First Line team has an excellent process and we can learn from your team and improve.
Product Owner, Strategic Software Consultancy

I would like to thank the First Line development team for their dedication and high level of quality in their work. Despite being thousands of miles apart, we have been able to work quickly, effectively, and intelligently. Your team has been a major part of why we have been successful thus far.
VP Engineering, e-learning startup

First Line is a great technology partner capable of delivering complex, ‘industrial grade’ projects. They have demonstrated strong technical competence and solid understanding of the media and publishing domain. We have also been very impressed by the management team's attention to our needs and commitment to our success.
CEO, leading European digital media company

Your effort is the #1 reason that this project is going so well. I've worked with a lot of teams over the years and there are very few teams that I trust and respect. When you are given a task or feature, I never have to worry whether or not it will be accomplished on time and I never worry about the quality - you always deliver.
Architect, Strategic Software Consultancy

The team we put together in the last half of this year is one of the most productive, skilled and enjoyable I’ve ever worked on. It’s great to see the product brought to life, and I’m proud to work with all you folks.
Senior Architect, Fortune 100 company

The First Line Software team is professional, results driven, and proactive. I know that when we provide requirements for new functionality, they will not only deliver on the scope of what has been asked but will look beyond and give insight into how we can make our product even better. It has been a privilege to work with First Line Software, and I look forward to continuing our partnership in future.
VP, global marketing company

related projects

all projects

Our customer asked our engineers to develop an online system that could identify and recognize military equipment in a photograph.

Show details

Wide-spread adoption of EHR systems brings along large amount of data available for reporting and other analytical insights. Health organizations routinely generate thousands of such analytical artifacts on daily or weekly basis.

Show details

Artificial Intelligence and Machine Learning are rapidly gaining momentum in Healthcare and - as some say - on the verge of becoming the most important breakthrough for healthcare since penicillin.

Show details