Prerequisites

  • Architecture: x86_64
  • Chip models: MThreads MTT S5000
  • Host driver: 5.2.0-server
  • Container toolkit (optional) : KUAE Cloud Native Toolkits (MT Container Toolkit) >= 2.1.0

Image contents

Built on

harbor.baai.ac.cn/flagos-runtime/flagos-runtime-mthreads-musa5.2.0:2.2.0 open_in_new

Python

3.10

Application package

vllm==0.20.2+flagos

vllm-plugin-fl==0.2.2

Launch

Published: harbor.baai.ac.cn/flagos-app/vllm0.20.2-mthreads-musa5.2.0:2.2.0-0.2.2

The image name is long — assign it to a variable first:

  IMG=harbor.baai.ac.cn/flagos-app/vllm0.20.2-mthreads-musa5.2.0:2.2.0-0.2.2

The two approaches below are alternatives — pick the one that matches how your host runs containers:

With the container toolkit

Start an interactive shell:

  docker run --rm -it \
  --runtime mthreads \
  --env MTHREADS_VISIBLE_DEVICES=all \
  $IMG bash

Start the app with its default settings:

  docker run --rm -it \
  --runtime mthreads \
  --env MTHREADS_VISIBLE_DEVICES=all \
  $IMG

Pass arguments to the launcher:

  docker run --rm -it \
  --runtime mthreads \
  --env MTHREADS_VISIBLE_DEVICES=all \
  $IMG vllm-serve --model <path> --port 9000

Without a toolkit — plain docker / podman

Start an interactive shell:

  docker run --rm -it \
  --device /dev/mtgpu.0 \
  --device /dev/dri \
  -v /usr/bin/mthreads-gmi:/usr/bin/mthreads-gmi:ro \
  $IMG bash

Start the app with its default settings:

  docker run --rm -it \
  --device /dev/mtgpu.0 \
  --device /dev/dri \
  -v /usr/bin/mthreads-gmi:/usr/bin/mthreads-gmi:ro \
  $IMG

Pass arguments to the launcher:

  docker run --rm -it \
  --device /dev/mtgpu.0 \
  --device /dev/dri \
  -v /usr/bin/mthreads-gmi:/usr/bin/mthreads-gmi:ro \
  $IMG vllm-serve --model <path> --port 9000
Edit this page

Last updated 28 Sep 2026, 14:37 +0800. history