tensorRT ipynb

+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "name": "Untitled0.ipynb",
+      "provenance": [],
+      "private_outputs": true,
+      "collapsed_sections": []
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "accelerator": "GPU"
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "8k6IDrTDGZ5W"
+      },
+      "source": [
+        "## Pytorch 추론시간 단축을 위한 정밀도 감소 - TensorRT / torch2trt library\n",
+        "\n",
+        "### 환경설정\n",
+        "\n",
+        "- [Nvidia developer portal](https://developer.nvidia.com/nvidia-tensorrt-7x-download) 회원가입\n",
+        "- TensorRT 7.0.0.11 for Ubuntu 1804 and CUDA 10.0 DEB local repo package 다운로드 후 colab 업로드\n",
+        "\n",
+        "### TensorRT 7.0 을 colab 에 설치하는 방법"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "ApojKcRcGf2j"
+      },
+      "source": [
+        "!nvcc --version"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "NZD-bw7YIlHL"
+      },
+      "source": [
+        "### colab CUDA 10.1 삭제"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "jJQrjrcfIm8u"
+      },
+      "source": [
+        "# remove CUDA 10.1\n",
+        "\n",
+        "!sudo apt-get --purge remove cuda nvidia* libnvidia-*\n",
+        "!sudo dpkg -l | grep cuda- | awk '{print $2}' | xargs -n1 dpkg --purge\n",
+        "!sudo apt-get remove cuda-*\n",
+        "!sudo apt autoremove\n",
+        "!sudo apt-get update"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "tRhvY4fZIygS"
+      },
+      "source": [
+        "### CUDA 10.0 설치 - 입력 세번 필요 (Y, 31, 1)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "73tZNq2WI41u"
+      },
+      "source": [
+        "# Installing CUDA 10.0\n",
+        "\n",
+        "!wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb\n",
+        "!sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb\n",
+        "!sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub\n",
+        "!sudo apt-get update\n",
+        "!wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb\n",
+        "!sudo apt install -y ./nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb\n",
+        "!sudo apt-get update\n",
+        "\n",
+        "# Install NVIDIA driver\n",
+        "!sudo apt-get -y installnvidia-driver-418\n",
+        "\n",
+        "# Install development and runtime libraries (~4GB)\n",
+        "!sudo apt-get install -y \\\n",
+        "    cuda-10-0 \\\n",
+        "    libcudnn7=7.6.2.24-1+cuda10.0  \\\n",
+        "    libcudnn7-dev=7.6.2.24-1+cuda10.0 --allow-change-held-packages"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "5HAS_SQ7LBgj"
+      },
+      "source": [
+        "!nvcc --version"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "emv-dfsgLEE9"
+      },
+      "source": [
+        "### TensorRT 설치 - colab .deb 파일 경로 확인"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "YI96JETCLJ3I"
+      },
+      "source": [
+        "# install tensorrt\n",
+        "!sudo dpkg -i \"/content/drive/My Drive/capstone1/CAN/nv-tensorrt-repo-ubuntu1804-cuda10.0-trt7.0.0.11-ga-20191216_1-1_amd64.deb\"\n",
+        "!sudo apt-key add /var/nv-tensorrt-repo-cuda10.0-trt7.0.0.11-ga-20191216/7fa2af80.pub\n",
+        "\n",
+        "!sudo apt-get update\n",
+        "\n",
+        "!sudo apt-get install libnvinfer7=7.0.0-1+cuda10.0 libnvonnxparsers7=7.0.0-1+cuda10.0 libnvparsers7=7.0.0-1+cuda10.0 libnvinfer-plugin7=7.0.0-1+cuda10.0 libnvinfer-dev=7.0.0-1+cuda10.0 libnvonnxparsers-dev=7.0.0-1+cuda10.0 libnvparsers-dev=7.0.0-1+cuda10.0 libnvinfer-plugin-dev=7.0.0-1+cuda10.0 python-libnvinfer=7.0.0-1+cuda10.0 python3-libnvinfer=7.0.0-1+cuda10.0\n",
+        "\n",
+        "!sudo apt-mark hold libnvinfer7 libnvonnxparsers7 libnvparsers7 libnvinfer-plugin7 libnvinfer-dev libnvonnxparsers-dev libnvparsers-dev libnvinfer-plugin-dev python-libnvinfer python3-libnvinfer\n",
+        "\n",
+        "!sudo apt-get install tensorrt"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "DcpWP5snL2yy"
+      },
+      "source": [
+        "### TensorRT 설치 확인"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "uOW85nnxLdSB"
+      },
+      "source": [
+        "!dpkg -l | grep TensorRT"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "k7xHrhZuNb0U"
+      },
+      "source": [
+        "### 런타임 다시시작하기"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "8EQ__AkjMH9G"
+      },
+      "source": [
+        "### [torch2trt](https://github.com/NVIDIA-AI-IOT/torch2trt) 라이브러리 설치"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "k7IztAyvMGlP"
+      },
+      "source": [
+        "cd /content/drive/My\\ Drive/capstone1/CAN"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "qR0V215wMHXF"
+      },
+      "source": [
+        "!pip3 install pycuda\n",
+        "!git clone https://github.com/NVIDIA-AI-IOT/torch2trt"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "fG9UtL6sNAgM"
+      },
+      "source": [
+        "cd /content/drive/My\\ Drive/capstone1/CAN/torch2trt"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "iMEc08ilNDW8"
+      },
+      "source": [
+        "!python setup.py install"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "LpK7xRFdNVbr"
+      },
+      "source": [
+        "import tensorrt\n",
+        "import torch2trt"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "M1DDuNL4Npn-"
+      },
+      "source": [
+        "### 추론시간 비교하기"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "9WddajL5Pj6E"
+      },
+      "source": [
+        "cd /content/drive/My\\ Drive/capstone1/CAN"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "PgKGWmSqNr4z"
+      },
+      "source": [
+        "import utils\n",
+        "import numpy as np\n",
+        "import importlib\n",
+        "importlib.reload(utils)\n",
+        "\n",
+        "utils.run_benchmark('./weights/fed_avg_50_0.9688.pth')"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "qh7_UaFsRykh"
+      },
+      "source": [
+        "- torch: 464 msc / 96.87 % / 0.05931\n",
+        "- trt: 354 msc / 96.87 % / 0.05931\n",
+        "- trt float 16: 323 msc / 96.87 % / 0.05931\n",
+        "- trt int8 strict: 401 msc / 96.86 % / 06405\n",
+        "\n",
+        "\n",
+        "#### int8 양자화 실패한 이유 추측\n",
+        "- input 이 0.0 ~ 1.0 이기 때문에 계산이 전부 float 연산\n",
+        "- nvidia tensorRT  가 정확도가 너무 낮아질 경우 양자화 무시\n",
+        "- 사용법 미숙, 모델이 너무 간단함\n"
+      ]
+    }
+  ]
+}
\ No newline at end of file