46 lines
1.2 KiB
Markdown
46 lines
1.2 KiB
Markdown
# tritonserver
|
|
|
|
Based on `nvcr.io/nvidia/tritonserver` container, but this add ~~[Miniconda3](https://www.anaconda.com/docs/getting-started/miniconda/main)~~, [Golang](https://go.dev/), [Singularity](https://docs.sylabs.io/guides/4.3/user-guide/)
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
TRITON_RELEASE="25.01"
|
|
docker run \
|
|
-d \
|
|
-it \
|
|
--restart=always \
|
|
--net=host \
|
|
--gpus=all \
|
|
-v ~/tritonserver-workspace:/workspace \
|
|
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
|
--name tritonserver \
|
|
--privileged \
|
|
git.athichal.com/tonkaew131/tritonserver:${TRITON_RELEASE}-py3-sdk
|
|
# Connect into the container
|
|
docker exec -it tritonserver bash
|
|
```
|
|
|
|
## Mounting
|
|
|
|
| Directory | Uses |
|
|
| -------------------------- | ----------------- |
|
|
| `/workspace` | Default workspace |
|
|
| `/root/.cache/huggingface` | Huggingface cache |
|
|
|
|
## Building
|
|
|
|
```bash
|
|
TRITON_RELEASE="25.01"
|
|
docker build \
|
|
-t git.athichal.com/tonkaew131/tritonserver:${TRITON_RELEASE}-py3-sdk \
|
|
--build-arg TRITON_RELEASE=${TRITON_RELEASE} \
|
|
.
|
|
```
|
|
|
|
## Pushing the image
|
|
|
|
```bash
|
|
docker image push \
|
|
git.athichal.com/tonkaew131/tritonserver:${TRITON_RELEASE}-py3-sdk
|
|
``` |