Compass Unified Parser is designed to convert models of various frameworks into a floating-point intermediate representation (IR). This IR is a standard IR designed by ARM China for the neural network compilation of the Zhouyi series. device.
Parser’s processing flow and design philosophy
The main goal of Parser is to convert a trained model into a floating-point IR and feed it to OPT (optimizer). The following is the main processing flow of Parser:
- A model is passed to Parser through a unified configuration file.
- The configuration parser parses the configuration file and submits a task to the corresponding model reader according to different configurations.
- A supported model reader will take over the input model and complete the reading of the model:
- Parse the model file (eg protobuf/flattenbuf/json or some proprietary format) and build a raw graph representation.
- Convert the nodes in the original graph representation to internal unified nodes, for example:
- Merge several TensorFlow nodes into one GRUSeq node.
- will caffe
detectionoutputNode converted to
- The model reader generates an internal unified graph representation, which is then passed to the front-end optimizer.
- The main operation object of the front-end optimizer is the unified graph representation. It will merge or eliminate some nodes, for example:
addto a node.
batchnormnode to node.
- Eliminate some useless nodes, for example: a swap dimension unchanged
- After optimization, Parser will perform at least one shape inference to obtain the shapes of all tensors.
- Do some extra processing, such as:
- Add some postprocessing nodes for some models.
- Serialized to an IR file.
Node the design of
Inside Parser, similar to other frameworks, we use
Node To represent a model, use a linked list to represent a graph.
Graph Only all nodes are saved, and the topological relationship between nodes is saved in a
Node and another
Node To represent the layer concept (layer) in IR,
Node can be accessed by calling
serialize method to serialize into a string.
About Parser Design
- Parser only supports a graph with a fixed shape (static graph), and will perform several shape derivations during the entire parsing and conversion process.
- After every graph operation, such as merging, transforming, and eliminating nodes, we hope to perform a shape derivation, unless you know and guarantee that all shapes are correct.
- Shape inference is done because graph operations may change the graph topology and may also result in changes in shape.
- If some parameters depend on the shape, then please put the processing of these parameters after the shape derivation or in the derivation phase.
- The optimized processing only points out the unified graph representation, and does not support the original graph representation. Therefore, models from all frameworks can benefit from these optimizations.
Parser is part of the Compass AIPUBuilder (NN-Compiler) compiler. You can refer to the following Compass AIPUBuilder guide to install AIPUBuilder. After completing the installation of AIPUBuilder, Parser will also be installed and can be used directly.
You can also passCompass_IntegrationInstructions in to compile an AIPUBuilder that includes a Parser.For instructions on using AIPUBuilder, please refer toMiniPkgThe manual inside: Zhouyi_Compass_Software_Programming_Guide_61010011_0205_01_en.pdf.
In addition to this, Parser can run alone.As long as the following dependencies are met, it can be run directly
main.pyfile to run Parser.
- python (3.8 or higher)
- onnx (> 12)
- tensorflow (== 2.6)
Parser is driven by the configuration file as input, you can use the following example to run Parser
python3 main.py -c my_config.ini
Configuration file format
All options must be in
Common Inside the section:
The shape of the input tensor. Regular models have only one input tensor, such as:
input_shape=[1,224,224,3]If you have multiple input tensors, separate them with commas, such as:
Enter a name for the model
The framework of the input model, the default is tensorflow, currently supports:
Classification of models, for example:
If your model is
object_detectionand you are using the official model, you can choose the following two post-processing methods, we will add the corresponding post-processing nodes at the end:
The file path of the input model, currently supports tensorflow frozen pb, tflite, caffe and onnx formats.
The name of the input node (or tensor), if there are multiple inputs, use English commas
Output node (or tensor) name, if there are multiple outputs, use English commas
Configuration file example
[Common] input_shape = [1,224,224,3] model_name = resnet50 model_domain = image_classification detection_postprocess = input_model = resnet50/frozen.pb input = Placeholder output = resnet_v1_50/predictions/Reshape
For more examples please refer to examples.
Run the example
First, you need to download the corresponding original model.you can passexamplesThe download_model.sh script below to download.
Then configure the corresponding input and output in the example.cfg file
[Common] model_type = tensorflow model_name = gru_l model_domain = image_classification input_model = ./GRU_L.pb input = Mfcc:0 input_shape = [1, 49, 10] output = labels_softmax:0 output_dir = ./
python3 run_example.py --framework [specify example] --input_data [specify feed data]
#Compass #Unified #Parser #Homepage #Documentation #Downloads #Model #Parser #News Delivery