sglang0.5.18-kunlunxin-xre5.37.1

Prerequisites link

• Architecture: x86_64
• Chip models: Kunlunxin P800
• Host driver: 5.37.1
• Container toolkit (optional) ⓘ: xpu_container >= 1.0.13

Image contents link

Built on link

Python link

3.10

Application package link

sglang==0.5.18+flagos

sglang-fl==0.1.dev1+g7fb22a0c2

Environment link

• TORCHINDUCTOR_COMPILE_THREADS=1
• SGLANG_IS_FLASHINFER_AVAILABLE=false
• SGLANG_SKIP_SGL_KERNEL_VERSION_CHECK=1

Launch link

Published: harbor.baai.ac.cn/flagos-app/sglang0.5.18-kunlunxin-xre5.37.1:2.1.2-0.1.dev1_g7fb22a0c2

The image name is long — assign it to a variable first:

  IMG=harbor.baai.ac.cn/flagos-app/sglang0.5.18-kunlunxin-xre5.37.1:2.1.2-0.1.dev1_g7fb22a0c2
  

The two approaches below are alternatives — pick the one that matches how your host runs containers:

With the container toolkit link

Start an interactive shell:

  docker run --rm -it \
  --runtime xpu \
  -e CXPU_VISIBLE_DEVICES=0 \
  $IMG bash
  

Start the app with its default settings:

  docker run --rm -it \
  --runtime xpu \
  -e CXPU_VISIBLE_DEVICES=0 \
  $IMG
  

Pass arguments to the launcher:

  docker run --rm -it \
  --runtime xpu \
  -e CXPU_VISIBLE_DEVICES=0 \
  $IMG sglang-serve --model-path <path> --port 9000
  

Without a toolkit — plain docker / podman link

Start an interactive shell:

  docker run --rm -it \
  --device /dev/xpu0 \
  --device /dev/xpuctrl \
  $IMG bash
  

Start the app with its default settings:

  docker run --rm -it \
  --device /dev/xpu0 \
  --device /dev/xpuctrl \
  $IMG
  

Pass arguments to the launcher:

  docker run --rm -it \
  --device /dev/xpu0 \
  --device /dev/xpuctrl \
  $IMG sglang-serve --model-path <path> --port 9000