Tom Vermeulen
You can compare machine learning without data to a car without fuel — you won’t get far if you don’t have any. So, whenever we start a new AI project, we first look at the data, as our systems are only as good as the data we use to train the model.
We often have to go through a data annotation process before we can get started with the data our customers provide us. Data annotation is the process of labeling data with relevant tags to help machines recognize what is needed and ultimately use it to make predictions. It can be done for text, images, audio, or video material, either by a human or automatically using tools. In this article, we will dive deeper into how we approach image annotation specifically for object detection, illustrated by one of our customer cases at pcfruit (read the entire case here).
There are two critical challenges to data annotation. First, let’s dive into the cost of annotating data. The price can vary as data annotation can be done manually or automatically. When doing it entirely manually, it requires a lot of time as it is labor intensive and therefore is quite expensive. Our customers often execute the data labeling task internally to reduce costs. Still, often, this results in substantial delays in the project, as this requires a lot of time and undivided attention.
The second challenge is accuracy; human errors can creep in when manually labeling an entire dataset. This directly impacts data quality and, in turn, the prediction of the machine learning models.
When applied to our customer case at pcfruit, we faced both challenges. To provide more context, for this particular case, we had to label a ton of blueberry blossoms in order to train an AI model to detect these blossoms later on. When we say a ton of flowers, we are talking about 196 pictures with an average of 736 blossoms per picture — a grand total of 144.297 flowers to be segmented. As you can see in the picture, the blossoms are not so easily segmented, so a lot of time and effort would go into manually labeling them. As this footage is quite overwhelming, the question of accuracy also arises. So, we looked for another solution: a data annotation tool. Because why wouldn’t we use an ML-driven solution to solve an ML-driven problem?
As AI-solution builders, we couldn’t pass on the opportunity to have an ML-driven solution help us, so we dove into the wonderful world of ML-assisted annotation and chose Segments.ai, a specialized software, as our annotation tool.
The first thing that drew us to the platform was its user interface. Segments.ai offers a convenient user interface, already speeding up labeling drastically. Second, they provide intelligent labeling tools such as their Superpixel and Autosegment technology to mark objects and regions with unseen speed and accuracy. Third, by training your AI model with the already labeled data, you can upload your model predictions into Segments.ai and turn these predictions into perfectly labeled data. Last but not least, the platform is very engineering friendly with a best-practice API and SDK to push in and pull out data.
In the pcfruit case, the autosegment technology already sped up the process immensely. Already a good percentage of the blossoms were detected, so less manual work was required. In addition, when we want to move beyond our proof of concept, we can easily upload our model predictions into the tool to label new data even faster.
Data quality drastically improves when you combine the power of machine learning-assisted annotation with a human-in-the-loop approach. Integrating human judgment in the data labeling process reduces bias and improves data quality while still benefiting from an accelerated cycle time. Segments.ai offers a dedicated external annotation workforce to maximize annotation quality.
So, back to our customer case at pcfruit. In the end, labeling all images and segmenting every blossom pixel took approximately ten workdays. Imagine three scenarios: one where the client appointed one of their employees to mark the data, one where we had chosen one of our AI engineers to go through the data, or one where we outsourced the annotation task to Segments.ai’s dedicated workforce. You can probably already guess why we went for the third option. Let’s go over some of our reasons.
First, our client’s employees have probably never labeled data before; our AI engineer is trained to work on AI models; and Segments.ai’s annotators specialize in annotating in the platform. The annotators are trained and as a result, much faster in working with the platform. Second, hiring an external workforce is only a fraction of the cost of an AI engineer or an internal employee. Ultimately, we labeled all our data in 50% of the turnaround time for only one-tenth of the price. Of course, if we would have onboarded a larger labeling team, we would have received the labeled data even quicker.
Need to start labeling your datasets? Think of these best practices before getting started so you can feed your data-hungry models what they need to perform well:
Keep these in mind, and if you don’t know where to get started, reach out to us!
Written by
Tom Vermeulen
Want to know more?
