tritonserver/README.md

tritonserver

Based on nvcr.io/nvidia/tritonserver container, but this add Miniconda3, Golang, Singularity

Usage

TRITON_RELEASE="25.01"
docker run \
    -d \
    -it \
    --restart=always \
    --net=host \
    --gpus=all \
    -v ~/tritonserver-workspace:/workspace \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --name tritonserver \
    --privileged \
    git.athichal.com/tonkaew131/tritonserver:${TRITON_RELEASE}-py3-sdk
# Connect into the container
docker exec -it tritonserver bash

Mounting

Directory	Uses
/workspace	Default workspace
/root/.cache/huggingface	Huggingface cache

Building

TRITON_RELEASE="25.01"
docker build \
    -t git.athichal.com/tonkaew131/tritonserver:${TRITON_RELEASE}-py3-sdk \
    --build-arg TRITON_RELEASE=${TRITON_RELEASE} \
    .

Pushing the image

docker image push \
    git.athichal.com/tonkaew131/tritonserver:${TRITON_RELEASE}-py3-sdk