# Run Batch命令 - 飞桨大语言模型推理部署工具包

跳转至

# run-batch：批处理

## 说明

批量运行推理任务。支持从本地文件或远程 URL 读取输入请求，并将推理结果输出到文件或上传至远程 URL

## 用法

      fastdeploy run-batch --model MODEL --input-file INPUT --output-file OUTPUT [参数]

## 参数

参数 | 说明 | 默认值
--- | --- | ---
-i, --input-file | 单个输入文件的路径或 URL。目前支持本地文件路径或 HTTP/HTTPS 协议。如果是文件路径，文件里每行一个请求；如果指定了 URL，文件应可通过 HTTP GET 访问。 | None
-o, --output-file | 单个输出文件的路径或 URL。目前支持本地文件路径或网络 URL（HTTP/HTTPS）。如果是输出文件路径，每行一个响应；如果指定了 URL，文件应可通过 HTTP PUT 上传。 | None
--output-tmp-dir | 在将输出文件上传到输出 URL 之前，用于存放输出文件的临时目录。 | None
--model | 模型路径 | None

更多参数说明见：FastDeploy 参数文档

## 示例

    fastdeploy run-batch -i Input.json -o Output.json --model baidu/ERNIE-4.5-0.3B-Paddle

## 输入文件格式示例（Input.json）

    {"custom_id": "req-00001", "method": "POST", "url": "/v1/chat/completions", "body": {"messages": [{"role": "user", "content": "Tell me a fun fact. (id=1)"}], "temperature": 0.7, "max_tokens": 50}}
    {"custom_id": "req-00002", "method": "POST", "url": "/v1/chat/completions", "body": {"messages": [{"role": "user", "content": "What's the weather like today? (id=2)"}], "temperature": 0.7, "max_tokens": 50}}

## 输出文件格式示例（Output.json）

    {"id":"fastdeploy-84601f40de3e48aeb3fe4d2ca328c32e","custom_id":"req-00001","response":{"status_code":200,"request_id":"fastdeploy-batch-0c18b71f5349453eaf00ae04659a21a0","body":{"id":"chatcmpl-024e9267-3d44-4594-91da-b5033c856da9","object":"chat.completion","created":1761203881,"model":"/root/PaddlePaddle/ERNIE-4.5-0.3B-Paddle","choices":[{"index":0,"message":{"role":"assistant","content":"Here's a fun fact about a funny animal:\n\n**The Elephant in the Room**\n\nElephants are known for their ability to inflate themselves with air pressure. Imagine a giant elephant standing upright, its trunk filling","multimodal_content":null,"reasoning_content":null,"tool_calls":null,"prompt_token_ids":null,"completion_token_ids":null,"prompt_tokens":null,"completion_tokens":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":18,"total_tokens":68,"completion_tokens":50,"prompt_tokens_details":{"cached_tokens":0}}}},"error":null}
    {"id":"fastdeploy-04cdfbd5b51e43408971b16be4439888","custom_id":"req-00002","response":{"status_code":200,"request_id":"fastdeploy-batch-dd7a9bebd2964acba6713c5dcb4b4aa6","body":{"id":"chatcmpl-452e9d0b-6c04-4b6f-9a2c-7d961f2dc605","object":"chat.completion","created":1761203881,"model":"/root/PaddlePaddle/ERNIE-4.5-0.3B-Paddle","choices":[{"index":0,"message":{"role":"assistant","content":"根据您提供的查询语句“What's the weather like today? (id=2)”，我需要先了解您想要查询的内容。请问您想查询的是某个特定天气状况（如晴天、下雨、阴天等）还是当前具体","multimodal_content":null,"reasoning_content":null,"tool_calls":null,"prompt_token_ids":null,"completion_token_ids":null,"prompt_tokens":null,"completion_tokens":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":20,"total_tokens":70,"completion_tokens":50,"prompt_tokens_details":{"cached_tokens":0}}}},"error":null}