Brain Builder Knowledge Base
Bounding Box Annotation on Video
Direct link to topic in this publication:
Neurala's Brain Builder features a number of tools to quickly and efficiently tag videos without having to tag each frame from scratch. The AI Video Annotator is a time-saving tool that learns tags from a single video frame and intelligently applies them to subsequent frames. For example, if you have ten seconds of video at 30 fps you will end up with 300 images annotated in just a few minutes.
InstaTag (default)—This option works better when your video contains objects that move and overlap each other. It is also a faster technology.
PolyTag—This option works better when your video changes perspective or objects leave and enter the frame.
Both options generate bounding boxes around the objects of interest, despite starting with polygons.
Whether you're using InstaTag or PolyTag, the first step is tagging the first frame of the video (or the section of the video you wish to intelligently annotate). Use the Pen tool to outline the edges of any objects you want to tag.
This should be done in the same manner as if you were tagging a single image with Segmentation, making the most of the available tagging tools in the Workspace. Tighter outlines will result in more precise bounding boxes.
After you create a polygon, assign a class by selecting one or creating a new one from the Classes list on the right.
Continue creating and labeling polygons until the first frame is tagged to your satisfaction. Then, click the Annotator button. The following pop-up appears.
Use the Video Clip slider to define the start and end points within the video where you want to apply the annotator. The start point should be the frame you just tagged and the endpoint can be as much or as little as you wish. However, we recommend testing performance on a few seconds of video before doing longer clips.
Select which Annotation Technology you want to use. See Tagging Options above for a description of each.
Set a minimum size for the polygons the annotator generates. In some situations, the video annotator can generate numerous small false positive results, so setting this value can help reduce those.
Then, click Start to run the annotator.
Annotating Multiple Clips within a Video
There are a number of reasons you may want to create multiple annotation clips within the same video:
There are "cuts" in the video or significant changes in perspective
The objects you are annotating change their appearance (example: a person turning from front to back)
There are changes in the objects you wish to annotate in different parts of the video
The annotator works best when the objects you want to tag and the perspective are reasonably consistent throughout the clip you are annotating.
To annotate multiple clips within a single video, start by tagging the first frame of the clip you want to annotate, and then use the Video Clip slider in the Annotator window to define the start and end point of the clip you wish to annotate.
Click the Timeline button to see a timeline with all of the annotation clips you have created.
If you have overlapping clips, you can click the Eye buttons on the left side to show/hide each clip individually. The Garbage Can button can be used to delete video annotation clips that you don't want to keep. The more recent clip (higher in the timeline view) will take priority over earlier ones for the frames where they overlap.
Refining Video Annotations
The polygons created by the video annotator may not be perfect, but Brain Builder's tagging tools can be used to correct any details in each frame that need to be fixed.
The Addition and Subtraction tools are typically the most useful for refining the video annotator's polygons to make sure they are tightly bound around the objects of interest.
If you find yourself having to correct many frames in a row, it might be more efficient to re-run the video annotator on that portion of the video as a separate clip.
Next: Exporting Data >>