# Reasoning Output - FastDeploy: Large Language Model Deployment

Skip to content

简体中文

# Reasoning Outputs

Reasoning models return an additional `reasoning_content` field in their output, which contains the reasoning steps that led to the final conclusion.

## Supported Models

Model Name | Parser Name | Enable thinking by Default | Tool Calling | Thinking switch parameters
--- | --- | --- | --- | ---
baidu/ERNIE-4.5-VL-424B-A47B-Paddle | ernie-45-vl | ✅ | ❌ | "chat_template_kwargs":{"enable_thinking": true/false}
baidu/ERNIE-4.5-VL-28B-A3B-Paddle | ernie-45-vl | ✅ | ❌ | "chat_template_kwargs":{"enable_thinking": true/false}
baidu/ERNIE-4.5-21B-A3B-Thinking | ernie-x1 | ✅ Not supported for turning off | ✅ | ❌
baidu/ERNIE-4.5-VL-28B-A3B-Thinking | ernie-45-vl-thinking | ✅ Not recommended to turn off | ✅ | "chat_template_kwargs": {"options": {"thinking_mode": "open/close"}}

The reasoning model requires a specified parser to extract reasoning content. Referring to the `thinking switch parameters` of each model can turn off the model's thinking mode.

Interfaces that support toggling the reasoning mode: 1. `/v1/chat/completions` requests in OpenAI services. 2. `/v1/chat/completions` requests in the OpenAI Python client. 3. `llm.chat` requests in Offline interfaces.

For reasoning models, the length of the reasoning content can be controlled via `reasoning_max_tokens`. Add `"reasoning_max_tokens": 1024` to the request.

### Quick Start

When launching the model service, specify the parser name using the `--reasoning-parser` argument. This parser will process the model's output and extract the `reasoning_content` field.

    python -m fastdeploy.entrypoints.openai.api_server \
        --model /path/to/your/model \
        --enable-mm \
        --tensor-parallel-size 8 \
        --port 8192 \
        --quantization wint4 \
        --reasoning-parser ernie-45-vl

Next, make a request to the model that should return the reasoning content in the response. Taking the baidu/ERNIE-4.5-VL-28B-A3B-Paddle model as an example：

    curl -X POST "http://0.0.0.0:8192/v1/chat/completions" \
    -H "Content-Type: application/json" \
    -d '{
      "messages": [
        {"role": "user", "content": [
          {"type": "image_url", "image_url": {"url": "https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
          {"type": "text", "text": "Which era does the cultural relic in the picture belong to"}
        ]}
      ],
      "chat_template_kwargs":{"enable_thinking": true},
      "reasoning_max_tokens": 1024
    }'

The `reasoning_content` field contains the reasoning steps to reach the final conclusion, while the `content` field holds the conclusion itself.

### Streaming chat completions

Streaming chat completions are also supported for reasoning models. The `reasoning_content` field is available in the `delta` field in `chat completion response chunks`

    from openai import OpenAI
    # Set OpenAI's API key and API base to use vLLM's API server.
    openai_api_key = "EMPTY"
    openai_api_base = "http://localhost:8192/v1"
    client = OpenAI(
        api_key=openai_api_key,
        base_url=openai_api_base,
    )
    chat_response = client.chat.completions.create(
        messages=[
            {"role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://paddlenlp.bj.bcebos.com/datasets/paddlemix/demo_images/example2.jpg"}},
            {"type": "text", "text": "Which era does the cultural relic in the picture belong to"}]}
        ],
        model="vl",
        stream=True,
        extra_body={
          "chat_template_kwargs":{"enable_thinking": True},
          "reasoning_max_tokens": 1024
        }
    )
    for chunk in chat_response:
        if chunk.choices[0].delta is not None:
            print(chunk.choices[0].delta, end='')
            print("\n")

## Tool Calling

The reasoning content is also available when both tool calling and the reasoning parser are enabled. Additionally, tool calling only parses functions from the `content` field, not from the `reasoning_content`.

Model request example:

    curl -X POST "http://0.0.0.0:8390/v1/chat/completions" \
    -H "Content-Type: application/json" \
    -d '{
      "messages": [
        {
          "role": "user",
          "content": "Get the current weather in BeiJing"
        }
      ],
      "tools": [
        {
          "type": "function",
          "function": {
            "name": "get_weather",
            "description": "Determine weather in my location",
            "parameters": {
              "type": "object",
              "properties": {
                "location": {
                  "type": "string",
                  "description": "The city and state e.g. San Francisco, CA"
                },
                "unit": {
                  "type": "string",
                  "enum": [
                    "c",
                    "f"
                  ]
                }
              },
              "additionalProperties": false,
              "required": [
                "location",
                "unit"
              ]
            },
            "strict": true
          }
        }],
        "stream": false
    }'

Model output example

    {
        "choices": [
            {
                "index": 0,
                "message": {
                    "role": "assistant",
                    "content": "",
                    "reasoning_content": "The user asks about ...",
                    "tool_calls": [
                        {
                            "id": "chatcmpl-tool-311b9bda34274722afc654c55c8ce6a0",
                            "type": "function",
                            "function": {
                                "name": "get_weather",
                                "arguments": "{\"location\": \"BeiJing\", \"unit\": \"c\"}"
                            }
                        }
                    ]
                },
                "finish_reason": "tool_calls"
            }
        ]
    }

More reference documentation related to tool calling usage： Tool Calling