Brain Builder Knowledge Base
Understanding Brain Performance
Direct link to topic in this publication:
As you tag images and train your Brain, Brain Builder is constantly measuring how "smart" your Brain is. This is shown as a Brain Score at the top of the Workspace.
As you tag images, the Brain Score will update in real time so you can see when your Brain is ready to be deployed.
Your Brain can have one of four possible scores:
Still Learning—This means your Brain needs more images to get smarter. Keep tagging!
Low—Your Brain has enough images to calculate a score, but it needs more knowledge. Keep tagging images to help your Brain get smarter.
Good—Your Brain is getting smarter! This is a good time to test your Brain and determine whether you need to keep going. If you want to keep going, consider Brain Builder's suggestions about which classes could use more images.
Great—Your Brain is a genius! Put it to work in the real world with our API or SDK.
Clicking on the Brain Score opens a drop-down screen where you can get more insight into the performance of your Brain and the steps you can take to improve it.
At the top of the Brain Score drop-down, you'll see some of the key Brain Score indicators:
Number of Classes on which the Brain has been trained
The number of Images that the Brain has learned
The number of images that have been reserved for Validation (see Brain Score Methodology below for more details.)
Brain Builder reviews the performance of each class the Brain has learned and provides guidance on how to improve your Brain Score. Brain Builder may suggest you tag more images for specific classes to help improve performance of those classes and increase your overall Brain Score.
Below the Recommendations, you'll see the Accuracy calculated for each class. This lets you see how the specific classes are contributing to the overall accuracy and identify any classes that are performing more poorly than others.
If you select the Images button, the view will change to show display the actual validation images that are used to calculate the score and whether Brain Builder is recognizing them correctly.
This view lets you visually review the incorrect images to see if they share any common characteristics that may be adversely impacting performance.
The chart below the Correct/Incorrect images changes from class accuracy to image count per class. This will let you see if your data is evenly balanced between your classes.
Brain Score Methodology
The Brain Score is calculated based on a number of different factors.
Number of Images
Brain Builder will dynamically look at the number of images you have tagged for each Class and let you know if you need to tag more for any of the classes. This makes sure your Brain gets enough examples for each Class.
As you label images, some of them are used to test Brain accuracy.
Accuracy is measured by analyzing these testing images with your trained Brain and calculating the percent that the Brain gets right. For example, if the Brain's predictions match the tags for 9/10 testing images, then the accuracy will be 90%.
Brain Builder measures accuracy for the Brain as a whole and for each class on which it has been trained.
Some AI platforms use the same images for both training and testing performance. This is a big no-no in the AI world. The best practice among AI researchers is to always use separate data for training and testing.
By saving some of your tagged images and not training with them, Brain Builder is able to give you a more realistic accuracy score.
The Lifelong-AITM that powers Brain Builder offers additional indicators that help determine whether your Brain has been trained well. If those indicators reveal that some classes have been trained better than others, Brain Builder will guide you to label additional images for those classes.
What if Performance Doesn't Get Better
First, you should try Retraining your Brain to make sure your Brain is in sync with all of the classes and annotated images in the dataset.
If you've retrained your Brain, followed the Best Practices and the Brain Score Recommendations, but the Brain Score still isn't improving, the problem most likely stems from the images you're using to train the Brain. Don't forget these important tips:
The objects or scenes you're tagging should be clear and prominent in the images.
Each image should contain a single class.
Classes should be clearly distinct from each other.
If you're still not sure why it's not working well, don't struggle alone! Contact us at firstname.lastname@example.org and our team will be happy to take a look and share our expertise.
Next: Brain Testing >>