Brain Builder for AITRIOS
Using Brain Builder for AITRIOS is similar to using Neurala's standard Brain Builder product. The following documentation will be helpful in understanding how to use Brain Builder for AITRIOS.
- Using Brain Builder
- Creating Datasets and Uploading Images
- Training Classifiers and Anomaly Recognizers
- Training Detectors
- Evaluating Brain Performance
Brain Builder versions
There are different versions of Brain Builder for AITRIOS with different features and functionality:
- The Full version of Brain Builder includes the ability to train all available model types detailed in this documentation.
- Lite versions of Brain Builder may be configured to allow training of some of the available model types.
Brain Builder licensing
The ability to train different Brain types depends on the Brain Builder for AITRIOS license. If you do not see the type of Brain you would like to use in Brain Builder, please contact AITRIOS support for assistance.
βοΈ Exporting Brains for AITRIOS
The following model types can be exported from Brain Builder for use with AITRIOS:
- Static Classifier
- Anomaly Hi-Fi
- Detector
Though Learning Classifiers and Anomaly Recognizers are trained in Brain Builder as part of the process, they are not exportable.
Export process
- After the Brain has been trained to your satisfaction, navigate to the Export page within Brain Builder and click the Download Brain button.
- This will generate a zip file that includes the trained Brain and other key files that are necessary to use the Brain within the AITRIOS platform.
Detailed instructions for the AITRIOS platform are available on the Sony's AITRIOS Developer Site.
.
π Export File Information
Each exported Brain comes packaged with multiple files. The most important of these is the README.md file that includes detailed information about the contents of the exported zip file.
It may be helpful to understand the PPL Parameter and Command Parameter settings to fully leverage the Neurala Brain within AITRIOS. These are documented on the AITRIOS developer site:
- For information about the PPL Parameter, see Appendix D: PPL Parameter.
- For more detail on Command Parameters, see Appendix A on the Console manual.
The files and data packaged with the Neurala Brain can be used to adjust these parameters, as follows:
Anomaly Hi-Fi
The flatbuffer schema for the Neurala Anomaly Hi-Fi Vision and Sensing WASM application is:
// version 1.0.0
namespace NeuralaRecognizer;
table HifiResults {
width:uint;
height:uint;
anomalyScore:float;
heatmap:[float];
}
root_type HifiResults;
width
: Width of the heatmapheight
: Height of the heatmapheatmap
: Matrix of floating point values. Each value is a floating point in the range [0.0 - 1.0], where 0 means the area is normal, and non zero refers to the degree of anomaly. Each value in the heatmap corresponds to the anomaly score inside the image Region of interest. The region of interest is divided into a grid of size width x height and each value maps cells of that grid starting from top-left for the first value of the heatmap to the bottom right cell for the last value of the heatmap, going from left to right line by line.
The relationship between theheatmap
and the input tensor to the trained model is 4:1, meaning the 1x1 pixel of theheatmap
corresponds to an 4x4 area of the input tensor.anomalyScore
: Anomaly level for the whole image, between [0.0 - 1.0], where 0.0 is Normal and 1.0 is completely anomalous. Each value in theheatmap
has its own score, and their combined values are used to generate theanomalyScore
.
You may use this application in conjunction with flatbuffer compiler flatc
to generate bindings for your programming language.
PPL Parameters:
PPL Parameters are generated by BrainBuilder for your Brain during the Export phase. They can be updated as needed. You should be able to find the PPL Parameter JSON file in your brain zip file along with the corresponding WASM application.
Anomaly Hifi models have two PPL parameters:
pixelThreshold
: Controls when a heatmap element is declared anomalous. If the element value is above threshold, it is an anomalous, ortherwise it is considered normal and set to 0.0.imageThreshold
: Controls when an image is declared anomalous. If anomaly score is above image threshold, the image is an anomaly, otherwise it is considered normal.
Deserialization
For sample code to deserialize the Anomaly Hi-Fi model output obtained from AITRIOS, see Neurala Tools for AITRIOS.
Detector
PPL Parameters
dnn_output_detections
: Set to the same value asdetection_classes -> channels
in the deploy.json file that is exported with the Detection Brain.max_detections
: Thissets the upper limit to the number of detected Bounding boxes you want to get when the Detector runs. The value should be greater than 0 and less than or equal to the value ofdnn_output_detections
.threshold
: This represents a "sensitivity" value. Lower thresholds will allow more bounding box predictions to occur, while higher thresholds will result in fewer predictions.input_width
: Set to the value shown in the deploy.json file that is exported with the Detection Brain.input_height
: Set to the value shown in the deploy.json file that is exported with the Detection Brain.
Static Classifier
PPL Parameters
dnn_output_classes
: Set to the number of classes on which the Brain has been trained.max_predictions
: Setto the maximum number of predictions you want to receive from the Static Classifier. The value should be greater than 0 and less than or equal to the value ofdnn_output_classes
. Neurala recommends a value of 1 so that the classifier returns one prediction per image.
Additional Model Details
Input Tensor
Input Tensor
Brain Builder automatically resizes images to fit the "input tensor" (or input resolution) of each model. This is not something that the user needs to do. The input resolutions for each model type are:
- Static Classifier: 256x256
- Anomaly Hi-Fi: 256x256
- Detector: 320x320
Output Tensor
Output Tensor
Static Classifier | Anomaly Hi-Fi | Detector | |
---|---|---|---|
Shape | 0: Class confidences - shape (N classes, 1, 1) N = the number of classes to classify | 0: Anomaly map and score - shape (2, 64, 64) | 0: Array of [N][4] - shape (1, [N], 4) 1: Array of N integers - shape (1, [N]) 2: Array of N - shape (1, [N]) 3: Integer value of N - shape (1, ) Max number of detections: VIA <= 24.07: N = 100 VIA => 24.09: N = 300 |
Values | idx=0..N - Floating point class confidences for each class ID represented by idx. | idx=0,β¦.,4095: heatmap - Floating point values between 0 and 1 idx=4096,β¦,8191:anomalyScore - Floating point values between 0 and 1 | 0: Locations - Floating point values between 0 and 1 1: Classes - Floating point values each indicating the index of a class label from the labels file 2: Scores - Floating point values between 0 and 1 representing probability that a class was detected 3: Number of detections - Integer value refers to the total count of the objects detected within the image |
Description | Class ID (idx) corresponds to the index of class name in βlabel.txtβ. | anomalyScore is first element of the second half of (2, 64, 64) with idx=4096. idx=4097..8191 are filled with meaningless zeros. | 0: Locations - The inner arrays representing bounding boxes in the form [top, left, bottom, right] 1: Classes - Each detected object is assigned a class label indicating what type of object it is 2: Scores - These scores help in determining the reliability of the detection. Higher scores indicate greater confidence in the detection result. 3: Number of detections - It indicates how many distinct objects the model has identified |
Updated about 2 months ago