Curl 聊天客户端#
请参阅 trtllm-serve 文档 了解如何启动服务器。
源代码 NVIDIA/TensorRT-LLM。
1#! /usr/bin/env bash
2
3curl http://localhost:8000/v1/chat/completions \
4 -H "Content-Type: application/json" \
5 -d '{
6 "model": "TinyLlama-1.1B-Chat-v1.0",
7 "messages":[{"role": "system", "content": "You are a helpful assistant."},
8 {"role": "user", "content": "Where is New York?"}],
9 "max_tokens": 16,
10 "temperature": 0
11 }'