Model Compression

In PaddleDetection, a complete tutorial and benchmarks for model compression based on PaddleSlim are provided. Currently supported methods:

• prunning
• quantitative
• distillation
• The joint strategy

It is recommended that you use a combination of prunning and distillation training, or use prunning and quantization for test model compression. The following takes YOLOv3 as an example to carry out cutting, distillation and quantization experiments.

Experimental Environment

• Python 3.7+
• PaddlePaddle >= 2.1.0
• PaddleSlim >= 2.1.0
• CUDA 10.1+
• cuDNN >=7.6.5

Version Dependency between PaddleDetection, Paddle and PaddleSlim Version

Install PaddleSlim

• Method 1: Install it directly：

pip install paddleslim -i https://pypi.tuna.tsinghua.edu.cn/simple

• Method 2: Compile and install：

git clone https://github.com/PaddlePaddle/PaddleSlim.git
cd PaddleSlim
python setup.py install

Quick Start

Train

python tools/train.py -c configs/{MODEL.yml} --slim_config configs/slim/{SLIM_CONFIG.yml}

• -c: Specify the model configuration file.
• --slim_config: Specify the compression policy profile.

Evaluation

python tools/eval.py -c configs/{MODEL.yml} --slim_config configs/slim/{SLIM_CONFIG.yml} -o weights=output/{SLIM_CONFIG}/model_final

• -c: Specify the model configuration file.
• --slim_config: Specify the compression policy profile.
• -o weights: Specifies the path of the model trained by the compression algorithm.

Test

python tools/infer.py -c configs/{MODEL.yml} --slim_config configs/slim/{SLIM_CONFIG.yml} \
    -o weights=output/{SLIM_CONFIG}/model_final
    --infer_img={IMAGE_PATH}

• -c: Specify the model configuration file.
• --slim_config: Specify the compression policy profile.
• -o weights: Specifies the path of the model trained by the compression algorithm.
• --infer_img: Specifies the test image path.

Full Chain Deployment

the model is derived from moving to static

python tools/export_model.py -c configs/{MODEL.yml} --slim_config configs/slim/{SLIM_CONFIG.yml} -o weights=output/{SLIM_CONFIG}/model_final

• -c: Specify the model configuration file.
• --slim_config: Specify the compression policy profile.
• -o weights: Specifies the path of the model trained by the compression algorithm.

prediction and deployment

• Paddle-Inference Prediction：
  • Python Deployment
  • C++ Deployment
  • TensorRT Predictive Deployment Tutorial
• Server deployment: UsedPaddleServing
• Mobile deployment: UsePaddle-Lite Deploy it on the mobile terminal.

Benchmark

Prunning

Pascal VOC Benchmark

COCO Benchmark

Description:

• Currently, all models except RCNN series models are supported.
• The SD855 predicts the delay for deployment using Paddle Lite, using the ARM8 architecture and using 4 Threads (4 Threads) to reason the delay.

Quantitative

COCO Benchmark

Description:

• The above V100 prediction delay non-quantified model is tested by TensorRT FP32, and the quantified model is tested by TensorRT INT8, and both of them include NMS time.
• The SD855 predicts the delay for deployment using PaddleLite, using the ARM8 architecture and using 4 Threads (4 Threads) to reason the delay.

Distillation

COCO Benchmark

• Please refer to the specific distillation methodDistillation Policy Document

Distillation Prunning Combined Strategy

COCO Benchmark