TensorRT Container Setup

Prerequisites

Before starting, ensure your system meets these requirements:

  • Linux operating system (Ubuntu 20.04 or later recommended)
  • Apropriate NVIDIA GPU
  • Apropriate NVIDIA driver
  • At least 10GB of free disk space

Table of Contents

  1. Install NVIDIA Driver
  2. Install Docker
  3. Install NVIDIA Container Toolkit
  4. Install TensorRT Container
  5. Verification
  6. Troubleshooting

Install NVIDIA Driver

  1. Check if NVIDIA driver is installed: 
    nvidia-smi
  2. Install NVIDIA drivers: 
    sudo apt install nvidia-driver-###  # Use latest recommended version
sudo reboot

Install Docker

  1. Check if docker is install and remove old versions (if any): 
    dpkg -l | grep -i docker
  2. Install Docker Engine: 
    sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
  3. Verify Docker installation: 
    sudo docker run hello-world

Install NVIDIA Container Toolkit

  1. Install NVIDIA Container Toolkit (if necessary):

Install TensorRT Container

  1. Pull the TensorRT container with TensorFlow support: 
    sudo docker pull nvcr.io/nvidia/tensorflow:##.##-tf#-py3
  2. Run the container: 
    sudo docker run --gpus all -it --rm nvcr.io/nvidia/tensorflow:##.##-tf#-py3

Verification

  1. Inside the container, verify TensorFlow can see the GPU: 
    import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))
  2. Verify TensorRT: 
    import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt
print(tf.__version__)

Troubleshooting

Common Issues and Solutions

  1. Docker permission denied

  2. NVIDIA driver not found 
    sudo reboot
  3. Container fails to start with GPU error 
    # Check if NVIDIA runtime is properly configured
sudo docker info | grep -i runtime

Additional Resources

Note: Version numbers in this guide may need to be updated based on your system requirements and the latest available versions.