<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Nginx on Ambertu</title><link>https://ambertu.com/tags/nginx/</link><description>Recent content in Nginx on Ambertu</description><generator>Hugo</generator><language>zh-CN</language><copyright>2026 Ambertu · 粤ICP备18071259号</copyright><lastBuildDate>Fri, 04 Sep 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://ambertu.com/tags/nginx/index.xml" rel="self" type="application/rss+xml"/><item><title>Caddy 还是 nginx 给 API 中转做反代：证书自动化是决定性差异</title><link>https://ambertu.com/posts/caddy-vs-nginx-api-gateway-tls/</link><pubDate>Fri, 04 Sep 2026 00:00:00 +0000</pubDate><guid>https://ambertu.com/posts/caddy-vs-nginx-api-gateway-tls/</guid><description>反代能力上 Caddy 和 nginx 都够用，真正的分水岭是证书生命周期谁来管。对 API 中转站来说证书过期不是「用户看到警告」，而是所有 SDK 客户端硬失败的全站中断。本文含一个共享域名下 ACME 验证被吞掉的真实坑。</description><content:encoded><![CDATA[<p>给大模型 API 中转选反向代理，网上的对比文章大多在比性能、比配置语法、比模块生态。这些差异对一个 API 网关来说其实都不重要——<strong>真正的分水岭是证书生命周期由谁管理。</strong></p>
<p>原因很简单：对普通网站，证书过期是&quot;用户看到一个可怕的警告，但能点继续&quot;；对 API 网关，证书过期是<strong>所有客户端硬失败的全站中断</strong>。Python 的 <code>httpx</code>、<code>requests</code>、OpenAI SDK 全都会直接抛 <code>SSLError</code>，没有&quot;点继续&quot;这个选项。而且报错文本是 <code>certificate verify failed</code>，第一眼看着像客户端环境问题，会把人引到完全错误的方向排查。</p>
<h2 id="一张手动证书的真实时间线">一张手动证书的真实时间线</h2>
<p>我手上一台机器用的是 nginx 加商业 DV 证书（TrustAsia 签发），实测有效期：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">notBefore = Jun 18 08:00:00 2026 GMT
</span></span><span class="line"><span class="cl">notAfter  = Sep 16 07:59:59 2026 GMT
</span></span></code></pre></div><p>正好 90 天。查证书那天是 9 月 3 日，<strong>距到期 13 天</strong>。</p>
<p>这台机器上跑着 API 服务和一个静态站，证书是手工申请、手工安装的。它现在没坏，问题是<strong>它的存续完全依赖&quot;有人记得&quot;</strong>——记得到期日、记得去续、记得装上去、记得 reload。这三个月里只要有一次没记住，就是一次全站中断。</p>
<figure>
<svg viewBox="0 0 800 260" role="img" aria-label="对比图：手动证书在 90 天有效期末尾依赖人工记得续期，存在中断风险窗口；ACME 自动续期在生命周期约三分之二处自动更新，永远不会接近到期日" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="c1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <text x="0" y="14" font-size="10.5" font-family="monospace" fill="currentColor" opacity="0.6">90 天证书生命周期</text>
  <!-- manual -->
  <text x="0" y="52" font-size="10.5" font-family="monospace" fill="currentColor">手动</text>
  <rect x="70" y="38" width="600" height="20" fill="none" stroke="currentColor" stroke-width="1.3"/>
  <rect x="583" y="38" width="87" height="20" fill="currentColor" opacity="0.22"/>
  <text x="70" y="76" font-size="9" font-family="monospace" fill="currentColor" opacity="0.65">签发</text>
  <text x="670" y="76" font-size="9" font-family="monospace" fill="currentColor" opacity="0.65" text-anchor="end">到期</text>
  <line x1="583" y1="90" x2="583" y2="62" stroke="currentColor" stroke-width="1.2" marker-end="url(#c1)"/>
  <text x="583" y="104" font-size="9.5" font-family="monospace" fill="currentColor" text-anchor="middle" font-weight="bold">今天 · 剩 13 天</text>
  <text x="676" y="52" font-size="9.5" font-family="monospace" fill="currentColor" font-weight="bold">⚠</text>
  <text x="70" y="122" font-size="9.5" font-family="monospace" fill="currentColor" opacity="0.75">灰色区间是风险窗口：全靠有人记得。忘一次 = 一次全站中断。</text>
  <line x1="0" y1="140" x2="780" y2="140" stroke="currentColor" stroke-width="1" opacity="0.2"/>
  <!-- auto -->
  <text x="0" y="180" font-size="10.5" font-family="monospace" fill="currentColor">ACME</text>
  <rect x="70" y="166" width="600" height="20" fill="none" stroke="currentColor" stroke-width="1.3"/>
  <line x1="470" y1="160" x2="470" y2="192" stroke="currentColor" stroke-width="1.4" stroke-dasharray="4 3"/>
  <text x="470" y="152" font-size="9.5" font-family="monospace" fill="currentColor" text-anchor="middle">约 2/3 处自动续期</text>
  <path d="M 470 202 L 560 202" stroke="currentColor" stroke-width="1.2" marker-end="url(#c1)" opacity="0.6"/>
  <text x="568" y="206" font-size="9.5" font-family="monospace" fill="currentColor" opacity="0.75">还有 30 天缓冲用于重试</text>
  <text x="70" y="232" font-size="9.5" font-family="monospace" fill="currentColor" opacity="0.75">永远不会接近右端。续期失败还有一个月时间告警和补救。</text>
</svg>
<figcaption>
两者的差别不是"自动比手动省事"，而是<strong>失败模式完全不同</strong>：
手动方案的失败是静默的、突然的、全量的；ACME 方案即使续期失败，也还有约 30 天窗口留给告警和人工介入。
</figcaption>
</figure>
<p>顺带说一个正在发生的趋势，它会让手动方案越来越站不住：<strong>证书有效期正在被行业强制缩短。</strong> 按 CA/Browser Forum 已通过的决议，TLS 证书最长有效期会分阶段从原来的一年多压到 200 天、100 天，最终到 47 天量级。手动续期在一年一次的年代还算可行，压到两个月一次就必然出事——<strong>这不是效率问题，是可靠性问题。</strong></p>
<h2 id="两者的实际差异">两者的实际差异</h2>
<table>
	<thead>
			<tr>
					<th>维度</th>
					<th>Caddy</th>
					<th>nginx</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td><strong>证书管理</strong></td>
					<td><strong>内置 ACME，默认自动申请与续期</strong></td>
					<td>本身不管，需要 certbot + 定时器</td>
			</tr>
			<tr>
					<td>反代能力</td>
					<td>够用</td>
					<td>够用</td>
			</tr>
			<tr>
					<td>流式 / SSE</td>
					<td>对 <code>text/event-stream</code> 默认刷新</td>
					<td>需显式关 <code>proxy_buffering</code> 和 <code>gzip</code></td>
			</tr>
			<tr>
					<td>复杂 rewrite / 正则路由</td>
					<td>能做，但表达力弱一些</td>
					<td><strong>更强，且久经考验</strong></td>
			</tr>
			<tr>
					<td>模块生态</td>
					<td>较小</td>
					<td><strong>大得多</strong></td>
			</tr>
			<tr>
					<td>配置语法</td>
					<td>Caddyfile，简洁</td>
					<td>熟悉的人多</td>
			</tr>
			<tr>
					<td>内存占用</td>
					<td>略高（Go 运行时）</td>
					<td>略低</td>
			</tr>
			<tr>
					<td>出网要求</td>
					<td><strong>需要能访问 ACME 目录</strong></td>
					<td>无</td>
			</tr>
	</tbody>
</table>
<p>看这张表要抓重点：<strong>除了第一行和最后一行，其余差异对一个 API 网关都不构成决策依据。</strong> 反代能力两边都够，性能差异在 LLM 场景里被上游生成耗时完全淹没——你的瓶颈是模型在想，不是代理在转发。</p>
<p>Caddy 的代价也要说清，别只看好处：</p>
<ul>
<li><strong>需要出网访问 ACME 目录。</strong> 内网隔离环境、或出网被严格管控的机器上，自动签发根本走不通，这时 Caddy 的核心优势直接消失。</li>
<li><strong><code>on_demand_tls</code> 是个陷阱。</strong> 配错了会给任意 Host 头签发证书，等于把签发配额敞开给别人刷。要用就必须配 <code>ask</code> 端点做白名单校验。</li>
<li><strong>复杂路由场景 nginx 更稳。</strong> 如果你已经有一大堆 nginx 的 <code>location</code> 规则和正则，重写成 Caddyfile 的风险可能高于收益。</li>
</ul>
<h2 id="不换-nginx-的话certbot-要配对">不换 nginx 的话，certbot 要配对</h2>
<p>大多数人的现实是 nginx 配置已经很复杂、不想动。那就把证书自动化补上，同样能解决问题：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 首次签发。--deploy-hook 是关键，否则证书换了但 nginx 还在用内存里的旧的</span>
</span></span><span class="line"><span class="cl">certbot certonly <span class="se">\
</span></span></span><span class="line"><span class="cl">  --webroot -w /var/www/acme <span class="se">\
</span></span></span><span class="line"><span class="cl">  -d ambertu.com -d www.ambertu.com <span class="se">\
</span></span></span><span class="line"><span class="cl">  --deploy-hook <span class="s2">&#34;systemctl reload nginx&#34;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 确认自动续期的定时器真的在跑 —— 这一步很多人跳过</span>
</span></span><span class="line"><span class="cl">systemctl list-timers <span class="p">|</span> grep certbot
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 演练一次，别等到真到期才发现签发链路是坏的</span>
</span></span><span class="line"><span class="cl">certbot renew --dry-run
</span></span></code></pre></div><p>三个容易漏的点：</p>
<ul>
<li><strong><code>--deploy-hook</code> 不能省。</strong> 证书文件换了，但 nginx 不会自己重新读——不 reload 就等于没续。</li>
<li><strong><code>systemctl list-timers</code> 必须确认。</strong> certbot 的自动续期靠 systemd timer（或 cron），装了 certbot 不代表定时器启用了。</li>
<li><strong><code>--dry-run</code> 要定期跑。</strong> 签发链路可能因为 DNS 变更、防火墙调整、challenge 路径被占而悄悄失效，<code>--dry-run</code> 是唯一能提前发现的手段。</li>
</ul>
<h2 id="一个共享域名下的真实坑acme-验证被吞掉">一个共享域名下的真实坑：ACME 验证被吞掉</h2>
<p>这个坑值得单独讲，因为它只在&quot;一个域名下住着多个应用&quot;时出现，而这种架构相当常见。</p>
<p>HTTP-01 验证要求 <code>http://your-domain/.well-known/acme-challenge/&lt;token&gt;</code> 能被访问到。问题是：<strong>这个路径会命中哪个 <code>location</code>？</strong></p>
<figure>
<svg viewBox="0 0 820 300" role="img" aria-label="示意图：共享域名下 ACME 验证路径的两种命中情况。未加专用 location 时验证路径落入站点根目录并被静态站的 try_files 返回 404 或被部署脚本清除；加了专用 location 后验证路径指向独立稳定目录，签发成功" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="c2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <text x="0" y="14" font-size="10.5" font-family="monospace" fill="currentColor" opacity="0.6">GET /.well-known/acme-challenge/TOKEN 会命中哪个 location？</text>
  <!-- bad -->
  <text x="0" y="46" font-size="10.5" font-family="monospace" fill="currentColor" opacity="0.6">❌ 没有专用 location</text>
  <g font-family="monospace" font-size="10" fill="currentColor">
    <rect x="0" y="58" width="130" height="36" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="65" y="80" text-anchor="middle">certbot</text>
    <line x1="130" y1="76" x2="196" y2="76" stroke="currentColor" stroke-width="1.2" marker-end="url(#c2)"/>
    <rect x="198" y="58" width="150" height="36" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="273" y="80" text-anchor="middle">nginx location /</text>
    <line x1="348" y1="76" x2="414" y2="76" stroke="currentColor" stroke-width="1.2" marker-end="url(#c2)"/>
    <text x="381" y="68" text-anchor="middle" font-size="8.5" opacity="0.65">try_files</text>
    <rect x="416" y="58" width="240" height="36" fill="currentColor" fill-opacity="0.1" stroke="currentColor" stroke-width="1.3"/>
    <text x="536" y="76" text-anchor="middle" font-size="9">站点根目录/.well-known/…</text>
    <text x="536" y="88" text-anchor="middle" font-size="8.5" opacity="0.7">文件不在这里</text>
    <line x1="656" y1="76" x2="700" y2="76" stroke="currentColor" stroke-width="1.2" marker-end="url(#c2)"/>
    <text x="712" y="80" font-weight="bold">404</text>
  </g>
  <text x="0" y="120" font-size="9.5" font-family="monospace" fill="currentColor" opacity="0.75">签发失败，报错只说"验证不通过"，不会告诉你是哪个 location 吃掉了请求。</text>
  <text x="0" y="138" font-size="9.5" font-family="monospace" fill="currentColor" opacity="0.75">即使把 webroot 指对了，下一次部署静态站又会把 .well-known 目录清掉。</text>
  <line x1="0" y1="156" x2="800" y2="156" stroke="currentColor" stroke-width="1" opacity="0.2"/>
  <!-- good -->
  <text x="0" y="188" font-size="10.5" font-family="monospace" fill="currentColor" opacity="0.6">✓ 加一个专用 location，指向独立目录</text>
  <g font-family="monospace" font-size="10" fill="currentColor">
    <rect x="0" y="200" width="130" height="36" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="65" y="222" text-anchor="middle">certbot</text>
    <line x1="130" y1="218" x2="196" y2="218" stroke="currentColor" stroke-width="1.2" marker-end="url(#c2)"/>
    <rect x="198" y="200" width="150" height="36" fill="currentColor" fill-opacity="0.1" stroke="currentColor" stroke-width="1.5"/>
    <text x="273" y="216" text-anchor="middle" font-size="8.5">location</text>
    <text x="273" y="229" text-anchor="middle" font-size="8">/.well-known/acme-challenge/</text>
    <line x1="348" y1="218" x2="414" y2="218" stroke="currentColor" stroke-width="1.2" marker-end="url(#c2)"/>
    <text x="381" y="210" text-anchor="middle" font-size="8.5" opacity="0.65">前缀更长</text>
    <rect x="416" y="200" width="240" height="36" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="536" y="218" text-anchor="middle" font-size="9">/var/www/acme/.well-known/…</text>
    <line x1="656" y1="218" x2="700" y2="218" stroke="currentColor" stroke-width="1.2" marker-end="url(#c2)"/>
    <text x="712" y="222" font-weight="bold">200</text>
  </g>
  <text x="0" y="262" font-size="9.5" font-family="monospace" fill="currentColor" opacity="0.75">前缀 location 更长，优先命中；目录独立于任何应用的部署产物，部署脚本碰不到它。</text>
  <text x="0" y="284" font-size="9.5" font-family="monospace" fill="currentColor" font-weight="bold">这个 location 必须在 80 端口的 server 块里 —— 别被 301 到 443 拦掉（见下）。</text>
</svg>
<figcaption>
共享域名下这个坑有两重：<strong>验证路径可能落到错误的目录，而正确的目录又可能被部署脚本清空。</strong>
一个指向独立目录的专用 <code>location</code> 同时解决两个问题。
</figcaption>
</figure>
<p>正确的配法：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="c1"># 80 端口的 server 块里 —— 顺序很重要
</span></span></span><span class="line"><span class="cl"><span class="k">server</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kn">listen</span> <span class="mi">80</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">server_name</span> <span class="s">ambertu.com</span> <span class="s">www.ambertu.com</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># ACME 验证必须放在 301 之前，否则请求会被跳转走
</span></span></span><span class="line"><span class="cl">    <span class="kn">location</span> <span class="s">/.well-known/acme-challenge/</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="kn">root</span> <span class="s">/var/www/acme</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="kn">location</span> <span class="s">/</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="kn">return</span> <span class="mi">301</span> <span class="s">https://</span><span class="nv">$host$request_uri</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p><strong>顺序是关键。</strong> 如果 80 端口的 server 块里只有一句 <code>return 301 https://...</code>，那么 ACME 的验证请求会被 301 走。Let&rsquo;s Encrypt 会跟随跳转到 443，所以这种情况有时也能成——但如果 443 上的 <code>location /</code> 又用 <code>try_files ... =404</code> 兜底，请求最终还是 404。加一个显式的前缀 <code>location</code> 才是确定的做法，不依赖跳转链路碰巧走对。</p>
<p>再说一遍那个连带的坑：<strong>验证目录要放在任何应用的部署产物之外。</strong> 如果你把 webroot 指向静态站的根目录，而部署脚本用的是 <code>rsync --delete</code> 式的镜像同步，<code>.well-known</code> 会在下一次发版时被删掉——证书在三个月后续期失败，而当时没有人会想起两个月前改过部署脚本。</p>
<h2 id="混合方案">混合方案</h2>
<p>如果 nginx 配置太复杂不想重写，但又想要自动证书，可以让 Caddy 只做 TLS 终止：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">外网 → Caddy :443（自动证书） → nginx :8080（全部现有路由逻辑） → 各应用
</span></span></code></pre></div><p>Caddyfile 简单到几行：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-caddyfile" data-lang="caddyfile"><span class="line"><span class="cl"><span class="gh">ambertu.com</span>, <span class="gh">www.ambertu.com</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="k">reverse_proxy</span> <span class="n">127.0.0.1</span><span class="p">:</span><span class="mi">8080</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="k">flush_interval</span> <span class="s">-1</span><span class="c1">          # 保住 SSE 流式
</span></span></span><span class="line"><span class="cl">        <span class="k">header_up</span> <span class="s">X-Real-IP</span> <span class="se">{remote_host}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p>代价是多一跳（本机回环，延迟可忽略），以及 nginx 那边要改成监听 8080 且信任来自本机的 <code>X-Real-IP</code>——这一步别搞错，否则所有客户端 IP 都会变成 <code>127.0.0.1</code>，按 IP 的限流会一起误伤。</p>
<h2 id="无论用哪个把到期日变成告警">无论用哪个：把到期日变成告警</h2>
<p>这是这篇文章最该带走的一条。<strong>再好的自动化也可能静默失效</strong>，所以到期监控是独立于方案选择的必需品：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="cp">#!/bin/sh
</span></span></span><span class="line"><span class="cl"><span class="c1"># /usr/local/bin/cert-expiry-check.sh</span>
</span></span><span class="line"><span class="cl"><span class="nv">DOMAIN</span><span class="o">=</span><span class="s2">&#34;</span><span class="si">${</span><span class="nv">1</span><span class="k">:-</span><span class="nv">ambertu</span><span class="p">.com</span><span class="si">}</span><span class="s2">&#34;</span>
</span></span><span class="line"><span class="cl"><span class="nv">THRESHOLD</span><span class="o">=</span><span class="m">20</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nv">END</span><span class="o">=</span><span class="k">$(</span><span class="nb">echo</span> <span class="p">|</span> openssl s_client -connect <span class="s2">&#34;</span><span class="nv">$DOMAIN</span><span class="s2">:443&#34;</span> -servername <span class="s2">&#34;</span><span class="nv">$DOMAIN</span><span class="s2">&#34;</span> 2&gt;/dev/null <span class="se">\
</span></span></span><span class="line"><span class="cl">      <span class="p">|</span> openssl x509 -noout -enddate <span class="p">|</span> cut -d<span class="o">=</span> -f2<span class="k">)</span>
</span></span><span class="line"><span class="cl"><span class="o">[</span> -z <span class="s2">&#34;</span><span class="nv">$END</span><span class="s2">&#34;</span> <span class="o">]</span> <span class="o">&amp;&amp;</span> <span class="o">{</span> <span class="nb">echo</span> <span class="s2">&#34;无法获取证书：</span><span class="nv">$DOMAIN</span><span class="s2">&#34;</span><span class="p">;</span> <span class="nb">exit</span> 2<span class="p">;</span> <span class="o">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nv">DAYS</span><span class="o">=</span><span class="k">$((</span> <span class="o">(</span> <span class="k">$(</span>date -d <span class="s2">&#34;</span><span class="nv">$END</span><span class="s2">&#34;</span> +%s<span class="k">)</span> <span class="o">-</span> <span class="k">$(</span>date +%s<span class="k">)</span> <span class="o">)</span> <span class="o">/</span> <span class="m">86400</span> <span class="k">))</span>
</span></span><span class="line"><span class="cl"><span class="nb">echo</span> <span class="s2">&#34;</span><span class="nv">$DOMAIN</span><span class="s2"> 证书剩余 </span><span class="nv">$DAYS</span><span class="s2"> 天（到期 </span><span class="nv">$END</span><span class="s2">）&#34;</span>
</span></span><span class="line"><span class="cl"><span class="o">[</span> <span class="s2">&#34;</span><span class="nv">$DAYS</span><span class="s2">&#34;</span> -lt <span class="s2">&#34;</span><span class="nv">$THRESHOLD</span><span class="s2">&#34;</span> <span class="o">]</span> <span class="o">&amp;&amp;</span> <span class="o">{</span> <span class="nb">echo</span> <span class="s2">&#34;低于 </span><span class="nv">$THRESHOLD</span><span class="s2"> 天，需要处理&#34;</span><span class="p">;</span> <span class="nb">exit</span> 1<span class="p">;</span> <span class="o">}</span>
</span></span><span class="line"><span class="cl"><span class="nb">exit</span> <span class="m">0</span>
</span></span></code></pre></div><p>两点设计考虑：</p>
<ul>
<li><strong>从外部探测，而不是读本地证书文件。</strong> 读文件只能证明&quot;磁盘上那份还有效&quot;，探端口才能证明&quot;nginx 实际吐出去的那份还有效&quot;。两者不一致正是&quot;续了但忘了 reload&quot;这个经典故障的样子。</li>
<li><strong>阈值设 20 天而不是 3 天。</strong> 要留出的不是&quot;续期需要的时间&quot;，而是&quot;发现问题、排查、找人、处理&quot;的时间。</li>
</ul>
<p>配个定时任务每天跑一次，非零退出就告警。这十几行东西能挡住的，是那种&quot;周末证书过期、周一早上所有客户端集体报 SSL 错误&quot;的事故。</p>
<h2 id="相关阅读">相关阅读</h2>
<ul>
<li><a href="/posts/llm-api-timeout-retry-double-billing/">大模型 API 中转的超时与重试：一次 504 可能让你双倍计费</a> —— 另一类会让全部请求一起失败的配置问题</li>
<li><a href="/posts/llm-api-sse-proxy-buffering/">反代把大模型 API 的 SSE 流式响应缓冲了：五个缓冲点与三层排查法</a> —— 换成 Caddy 后 SSE 的行为差异</li>
<li><a href="/posts/api-gateway-real-ip-ratelimit/">API 中转站的限流为什么不能信 X-Forwarded-For</a> —— 混合方案里 <code>X-Real-IP</code> 那一跳为什么必须配对</li>
</ul>
]]></content:encoded></item><item><title>大模型 API 中转的超时与重试：一次 504 可能让你双倍计费</title><link>https://ambertu.com/posts/llm-api-timeout-retry-double-billing/</link><pubDate>Fri, 04 Sep 2026 00:00:00 +0000</pubDate><guid>https://ambertu.com/posts/llm-api-timeout-retry-double-billing/</guid><description>nginx 的 proxy_read_timeout 默认 60 秒，而长上下文生成常常超过它。超时返回 504 后客户端 SDK 会自动重试，上游却已经生成并计费——一次超时变成两次账单。本文拆解这条链路并给出正确的超时分层。</description><content:encoded><![CDATA[<p>nginx 的 <code>proxy_read_timeout</code> 默认是 <strong>60 秒</strong>，而长上下文或推理模型的单次生成经常超过这个数。超时之后 nginx 返回 504，看起来只是&quot;一次失败&quot;——但真实后果比这严重：<strong>上游很可能已经把生成做完并计了费，而客户端 SDK 收到 504 会自动重试，于是上游又生成一次、又计一次费。</strong></p>
<p>客户端最终只拿到一个响应，你的账单上是两笔。更麻烦的是这件事在监控里几乎看不出来：错误率只涨了一点，成本却在悄悄翻倍。</p>
<h2 id="先纠正一个常见的误解">先纠正一个常见的误解.</h2>
<p>网上很多文章会告诉你 nginx 的 <code>proxy_next_upstream</code> 会把超时请求转给下一个上游、导致重复计费。<strong>这个说法对 LLM API 来说是不准确的</strong>，值得先说清楚，否则你会去改一个本来没问题的配置。</p>
<p><code>proxy_next_upstream</code> 的默认值确实是 <code>error timeout</code>，但 nginx 从 <strong>1.9.13</strong> 起有一条额外规则：</p>
<blockquote>
<p>非幂等方法（POST、LOCK、PATCH）的请求，<strong>一旦已经发送给上游，就不会</strong>再转给下一台服务器——除非你显式加上 <code>non_idempotent</code> 选项。</p>
</blockquote>
<p>LLM 的 chat completions 走的是 POST，所以 nginx 默认行为是安全的。<strong>真正的重复计费来源有两个，都不是 nginx 的默认配置：</strong></p>
<ol>
<li><strong>有人手工加了 <code>non_idempotent</code></strong>——一些&quot;提升可用性&quot;的高可用教程会教这个，在普通 Web 服务上无害，在按 token 计费的接口上是直接烧钱。</li>
<li><strong>客户端 SDK 自己在重试</strong>——这是绝大多数情况的元凶。OpenAI 官方 Python SDK 的 <code>max_retries</code> <strong>默认是 2</strong>，重试触发条件包含连接错误、408、409、429 和所有 5xx。nginx 吐的 504 正好落在 5xx 里。</li>
</ol>
<h2 id="双倍计费是怎么发生的">双倍计费是怎么发生的</h2>
<figure>
<svg viewBox="0 0 820 340" role="img" aria-label="时序图：客户端请求经 nginx 转发给上游，上游生成耗时 75 秒；nginx 在 60 秒时触发 proxy_read_timeout 返回 504，但上游继续生成完并计费；客户端 SDK 收到 504 后自动重试，上游第二次生成并再次计费，最终客户端得到一个响应而产生两笔账单" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="t1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <g font-family="monospace" font-size="11" fill="currentColor">
    <text x="40" y="16" text-anchor="middle">客户端</text>
    <text x="300" y="16" text-anchor="middle">nginx</text>
    <text x="600" y="16" text-anchor="middle">上游 API</text>
  </g>
  <line x1="40"  y1="26" x2="40"  y2="300" stroke="currentColor" stroke-width="1" opacity="0.3" stroke-dasharray="3 4"/>
  <line x1="300" y1="26" x2="300" y2="300" stroke="currentColor" stroke-width="1" opacity="0.3" stroke-dasharray="3 4"/>
  <line x1="600" y1="26" x2="600" y2="300" stroke="currentColor" stroke-width="1" opacity="0.3" stroke-dasharray="3 4"/>
  <g font-family="monospace" font-size="9.5" fill="currentColor">
    <!-- t0 -->
    <text x="0" y="52" opacity="0.6">t=0s</text>
    <line x1="40" y1="48" x2="296" y2="48" stroke="currentColor" stroke-width="1.2" marker-end="url(#t1)"/>
    <line x1="300" y1="48" x2="596" y2="48" stroke="currentColor" stroke-width="1.2" marker-end="url(#t1)"/>
    <text x="450" y="41" text-anchor="middle" opacity="0.7">POST /v1/chat/completions</text>
    <!-- generating 1 -->
    <rect x="586" y="56" width="28" height="70" fill="currentColor" opacity="0.2"/>
    <text x="626" y="80">生成中</text>
    <text x="626" y="94" opacity="0.7">第 1 次</text>
    <text x="626" y="112" font-weight="bold">已计费 ①</text>
    <!-- timeout -->
    <text x="0" y="136" opacity="0.6">t=60s</text>
    <line x1="296" y1="132" x2="44" y2="132" stroke="currentColor" stroke-width="1.4" marker-end="url(#t1)"/>
    <text x="170" y="125" text-anchor="middle" font-weight="bold">504 Gateway Timeout</text>
    <text x="170" y="146" text-anchor="middle" opacity="0.7">proxy_read_timeout 到点</text>
    <text x="626" y="140" opacity="0.7">上游不知道</text>
    <text x="626" y="154" opacity="0.7">nginx 放弃了，</text>
    <text x="626" y="168" opacity="0.7">继续把生成做完</text>
    <!-- retry -->
    <text x="0" y="196" opacity="0.6">t=60s</text>
    <line x1="40" y1="192" x2="296" y2="192" stroke="currentColor" stroke-width="1.4" marker-end="url(#t1)"/>
    <line x1="300" y1="192" x2="596" y2="192" stroke="currentColor" stroke-width="1.4" marker-end="url(#t1)"/>
    <text x="450" y="185" text-anchor="middle" font-weight="bold">SDK 自动重试（max_retries=2）</text>
    <!-- generating 2 -->
    <rect x="586" y="200" width="28" height="70" fill="currentColor" opacity="0.2"/>
    <text x="626" y="224">生成中</text>
    <text x="626" y="238" opacity="0.7">第 2 次</text>
    <text x="626" y="256" font-weight="bold">又计费 ②</text>
    <!-- success -->
    <text x="0" y="288" opacity="0.6">t=135s</text>
    <line x1="596" y1="284" x2="304" y2="284" stroke="currentColor" stroke-width="1.2" marker-end="url(#t1)"/>
    <line x1="296" y1="284" x2="44" y2="284" stroke="currentColor" stroke-width="1.2" marker-end="url(#t1)"/>
    <text x="170" y="277" text-anchor="middle" opacity="0.7">200 OK</text>
  </g>
  <line x1="0" y1="306" x2="800" y2="306" stroke="currentColor" stroke-width="1" opacity="0.2"/>
  <text x="0" y="326" font-size="10.5" font-family="monospace" fill="currentColor" font-weight="bold">客户端拿到 1 个响应 · 账单上 2 笔 · 监控里只是错误率轻微上升</text>
</svg>
<figcaption>
链条的关键一环是<strong>上游并不知道 nginx 已经放弃</strong>——它按自己的节奏把生成做完。是否计费取决于供应商策略，
但<strong>安全假设是"已经计了"</strong>：连接中断后仍收费的供应商很常见，你无法事后核对。
</figcaption>
</figure>
<h2 id="流式请求几乎不会撞上这个问题">流式请求几乎不会撞上这个问题</h2>
<p>这是很多人不知道的一点，也是&quot;默认就该用流式&quot;的一个纯运维理由。<code>proxy_read_timeout</code> 的官方定义是：</p>
<blockquote>
<p>该超时只作用于<strong>两次连续读操作之间</strong>，而不是整个响应的传输时长。</p>
</blockquote>
<p>所以：</p>
<ul>
<li><strong>非流式请求</strong>：从发出到收到完整 JSON，中间是一个长长的空隙。生成花 75 秒，就是一个 75 秒的读间隔 → 撞上 60 秒超时。</li>
<li><strong>流式请求</strong>：token 一个接一个到达，每两次读之间只隔几百毫秒 → 无论总时长多久都不会触发超时。</li>
</ul>
<figure>
<svg viewBox="0 0 780 220" role="img" aria-label="对比图：非流式请求在整个生成期间没有任何数据到达，形成一个 75 秒的读间隔从而在 60 秒处触发超时；流式请求每几百毫秒到达一个 token，读间隔始终很小，永不触发超时" style="width:100%;height:auto;color:currentColor">
  <text x="0" y="14" font-size="10.5" font-family="monospace" fill="currentColor" opacity="0.6">nginx 眼中的「两次读操作之间的间隔」</text>
  <!-- timeout threshold -->
  <line x1="130" y1="30" x2="130" y2="180" stroke="currentColor" stroke-width="0" />
  <line x1="530" y1="26" x2="530" y2="186" stroke="currentColor" stroke-width="1.4" stroke-dasharray="5 4"/>
  <text x="538" y="40" font-size="9.5" font-family="monospace" fill="currentColor" font-weight="bold">60s 超时线</text>
  <g font-family="monospace" font-size="10.5" fill="currentColor">
    <text x="0" y="66">非流式</text>
    <line x1="130" y1="70" x2="700" y2="70" stroke="currentColor" stroke-width="1" opacity="0.25"/>
    <rect x="130" y="60" width="10" height="20" fill="currentColor" opacity="0.75"/>
    <rect x="620" y="60" width="80" height="20" fill="currentColor" opacity="0.4"/>
    <path d="M 142 90 L 616 90" stroke="currentColor" stroke-width="1.2" marker-end="url(#t1)"/>
    <text x="370" y="106" text-anchor="middle" font-size="9.5" font-weight="bold">一个 75s 的读间隔 → 在 60s 处被切断</text>
    <text x="0" y="152">流式</text>
    <line x1="130" y1="156" x2="700" y2="156" stroke="currentColor" stroke-width="1" opacity="0.25"/>
  </g>
  <g fill="currentColor" opacity="0.75">
    <rect x="130" y="146" width="8" height="20"/><rect x="152" y="146" width="8" height="20"/>
    <rect x="174" y="146" width="8" height="20"/><rect x="196" y="146" width="8" height="20"/>
    <rect x="218" y="146" width="8" height="20"/><rect x="240" y="146" width="8" height="20"/>
    <rect x="262" y="146" width="8" height="20"/><rect x="284" y="146" width="8" height="20"/>
    <rect x="306" y="146" width="8" height="20"/><rect x="328" y="146" width="8" height="20"/>
    <rect x="350" y="146" width="8" height="20"/><rect x="372" y="146" width="8" height="20"/>
    <rect x="394" y="146" width="8" height="20"/><rect x="416" y="146" width="8" height="20"/>
    <rect x="438" y="146" width="8" height="20"/><rect x="460" y="146" width="8" height="20"/>
    <rect x="482" y="146" width="8" height="20"/><rect x="504" y="146" width="8" height="20"/>
    <rect x="548" y="146" width="8" height="20"/><rect x="570" y="146" width="8" height="20"/>
    <rect x="592" y="146" width="8" height="20"/><rect x="614" y="146" width="8" height="20"/>
    <rect x="636" y="146" width="8" height="20"/><rect x="658" y="146" width="8" height="20"/>
  </g>
  <text x="130" y="188" font-size="9.5" font-family="monospace" fill="currentColor" font-weight="bold">每个间隔都远小于 60s → 穿过超时线也毫发无伤</text>
  <text x="0" y="212" font-size="10" font-family="monospace" fill="currentColor" opacity="0.6">本图为示意，用于说明超时的计量方式</text>
</svg>
<figcaption>
同一个总时长、同一个超时配置，结果完全不同。<strong>非流式请求才是超时的重灾区。</strong>
如果你的中转站还有大量非流式调用，这就是优先要治的地方——而不是一味把超时往上调。
</figcaption>
</figure>
<h2 id="正确的超时分层">正确的超时分层</h2>
<p>关键是<strong>区分&quot;连不上&quot;和&quot;生成慢&quot;</strong>：前者应该快速失败，后者必须耐心等。用一个统一的大数字把三个超时都拉长，是把两种完全不同的故障混在一起处理。</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="k">location</span> <span class="s">/v1/</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_pass</span> <span class="s">http://upstream_pool</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># 连不上 → 快速失败。请求还没发出去，此时重试是安全的
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_connect_timeout</span>  <span class="s">5s</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># 发送请求体 → 大 prompt 也就几百 KB，给足但不必夸张
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_send_timeout</span>     <span class="s">60s</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># 读响应 → 生成慢是正常的，不是故障。这里必须放宽
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_read_timeout</span>     <span class="s">600s</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># LLM 是非幂等的：绝不要开 non_idempotent
</span></span></span><span class="line"><span class="cl">    <span class="c1"># 保守起见连 timeout 重试也去掉，只保留连接阶段的重试
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_next_upstream</span>    <span class="s">error</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_next_upstream_tries</span> <span class="mi">2</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_next_upstream_timeout</span> <span class="s">10s</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># 流式必须配套，否则响应被攒起来又变回一个长读间隔
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_buffering</span>        <span class="no">off</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">gzip</span>                   <span class="no">off</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_http_version</span>     <span class="mi">1</span><span class="s">.1</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_set_header</span>       <span class="s">Connection</span> <span class="s">&#34;&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><figure>
<svg viewBox="0 0 740 200" role="img" aria-label="超时分层示意：连接超时 5 秒对应连不上的情况且重试安全，发送超时 60 秒，读取超时 600 秒对应生成慢的情况不应视为故障，客户端 SDK 重试次数应设为 0 或 1" style="width:100%;height:auto;color:currentColor">
  <text x="0" y="14" font-size="10.5" font-family="monospace" fill="currentColor" opacity="0.6">按故障性质分层，而不是统一拉长</text>
  <g font-family="monospace" font-size="10.5" fill="currentColor">
    <rect x="0" y="30" width="700" height="34" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="12" y="52">proxy_connect_timeout</text>
    <text x="200" y="52" font-weight="bold">5s</text>
    <text x="250" y="52" opacity="0.7">连不上 → 快速失败</text>
    <text x="500" y="52" opacity="0.7">请求未发出，重试安全 ✓</text>
    <rect x="0" y="72" width="700" height="34" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="12" y="94">proxy_send_timeout</text>
    <text x="200" y="94" font-weight="bold">60s</text>
    <text x="250" y="94" opacity="0.7">上传 prompt</text>
    <text x="500" y="94" opacity="0.7">已发出，重试即重复计费 ⚠</text>
    <rect x="0" y="114" width="700" height="34" fill="currentColor" fill-opacity="0.1" stroke="currentColor" stroke-width="1.5"/>
    <text x="12" y="136">proxy_read_timeout</text>
    <text x="200" y="136" font-weight="bold">600s</text>
    <text x="250" y="136" opacity="0.7">生成慢 → 不是故障</text>
    <text x="500" y="136" font-weight="bold">这一层千万别按 60s 算 ⚠</text>
    <rect x="0" y="156" width="700" height="34" fill="none" stroke="currentColor" stroke-width="1.3" stroke-dasharray="4 3"/>
    <text x="12" y="178">SDK max_retries</text>
    <text x="200" y="178" font-weight="bold">0 或 1</text>
    <text x="250" y="178" opacity="0.7">客户端侧</text>
    <text x="500" y="178" opacity="0.7">默认 2，是双倍计费主因</text>
  </g>
</svg>
<figcaption>灰底那一行是唯一需要显著放宽的；虚线那一行不在服务端，<strong>但它才是账单翻倍的直接原因</strong>，容易被漏掉。</figcaption>
</figure>
<h2 id="客户端侧要做的事">客户端侧要做的事</h2>
<p>服务端配对了，客户端如果还在盲目重试，问题只是变少而不是消失。</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">openai</span> <span class="kn">import</span> <span class="n">OpenAI</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">client</span> <span class="o">=</span> <span class="n">OpenAI</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">    <span class="n">base_url</span><span class="o">=</span><span class="s2">&#34;https://your-gateway/v1&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">api_key</span><span class="o">=</span><span class="s2">&#34;...&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="n">max_retries</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>   <span class="c1"># 默认是 2，对按 token 计费的接口必须显式关掉</span>
</span></span><span class="line"><span class="cl">    <span class="n">timeout</span><span class="o">=</span><span class="mf">600.0</span><span class="p">,</span>   <span class="c1"># 客户端超时也要放宽，否则轮到它先放弃</span>
</span></span><span class="line"><span class="cl"><span class="p">)</span>
</span></span></code></pre></div><p>关掉自动重试之后，重试逻辑要自己写，并且<strong>只在明确安全的错误上重试</strong>：</p>
<table>
	<thead>
			<tr>
					<th>错误</th>
					<th>能否重试</th>
					<th>原因</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>连接失败 / DNS 失败</td>
					<td><strong>可以</strong></td>
					<td>请求没发出去，上游没开始生成</td>
			</tr>
			<tr>
					<td><code>429</code> 限流</td>
					<td><strong>可以</strong></td>
					<td>上游明确拒绝了，没有生成</td>
			</tr>
			<tr>
					<td><code>400</code> 参数错误</td>
					<td>不该重试</td>
					<td>重试一万次也是同样结果</td>
			</tr>
			<tr>
					<td><code>401</code> / <code>403</code></td>
					<td>不该重试</td>
					<td>凭据问题</td>
			</tr>
			<tr>
					<td><code>504</code> / <code>502</code></td>
					<td><strong>危险</strong></td>
					<td>上游可能已经生成完了，重试就是第二笔账单</td>
			</tr>
			<tr>
					<td>流式中途断开</td>
					<td><strong>危险</strong></td>
					<td>已经生成了一部分，且已产生费用</td>
			</tr>
	</tbody>
</table>
<h2 id="更彻底的办法在中转层做请求去重">更彻底的办法：在中转层做请求去重</h2>
<p>如果你控制中转层（自建 new-api 之类就是这种情况），可以在网关上加一层短窗口去重，把重复计费在源头掐掉：</p>
<p>思路是给每个请求算一个指纹——模型名 + 消息体 + 关键参数的哈希——配上客户端标识，在 Redis 里存一个几十秒的短 key。窗口内出现相同指纹时，不再打上游，直接返回上一次的结果或一个明确的&quot;重复请求&quot;错误。</p>
<p>这个做法的取舍要讲清楚：</p>
<ul>
<li><strong>好处</strong>：无论客户端怎么重试，上游只被打一次，账单可控。</li>
<li><strong>代价</strong>：LLM 本身是有随机性的（temperature &gt; 0 时同样输入本该有不同输出），去重会让窗口内的重复请求拿到相同结果。对聊天场景通常无害，对&quot;故意重新生成一次&quot;的功能就是 bug。</li>
<li><strong>所以</strong>：窗口要短（几十秒量级），并且给客户端一个显式绕过的办法（比如一个请求头或一个随机 nonce 参数）。</li>
</ul>
<h2 id="怎么发现自己已经在被双倍计费">怎么发现自己已经在被双倍计费</h2>
<p>从日志里找证据，不用等账单。特征是：<strong>同一客户端在很短时间内发出内容完全相同的请求，且其中至少一个返回了 504。</strong></p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 找 504：如果这个数不是 0，就值得往下查</span>
</span></span><span class="line"><span class="cl">awk <span class="s1">&#39;$9==504 {print $1, $7}&#39;</span> /var/log/nginx/access.log <span class="p">|</span> sort <span class="p">|</span> uniq -c <span class="p">|</span> sort -rn <span class="p">|</span> head
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 找 504 之后紧跟着重试的模式：同 IP、同路径、几秒内连续多次</span>
</span></span><span class="line"><span class="cl">awk <span class="s1">&#39;$9==504 || $9==200 {print $1, $4, $9, $7}&#39;</span> /var/log/nginx/access.log <span class="se">\
</span></span></span><span class="line"><span class="cl">  <span class="p">|</span> grep <span class="s1">&#39;/v1/chat/completions&#39;</span> <span class="p">|</span> tail -50
</span></span></code></pre></div><p>更可靠的信号是把<strong>上游用量与自己记录的成功响应数对账</strong>：如果上游统计的请求数明显高于你返回给客户端的成功数，差额基本就是被白烧掉的重试。</p>
<h2 id="相关阅读">相关阅读</h2>
<ul>
<li><a href="/posts/llm-api-sse-proxy-buffering/">反代把大模型 API 的 SSE 流式响应缓冲了：五个缓冲点与三层排查法</a> —— 本文说流式不容易超时，前提是流式真的没被缓冲掉</li>
<li><a href="/posts/caddy-vs-nginx-api-gateway-tls/">Caddy 还是 nginx 给 API 中转做反代：证书自动化是决定性差异</a> —— 另一类会让全部请求一起失败的运维故障</li>
<li><a href="/posts/new-api-binary-vs-docker/">new-api 部署选型：2G 小内存机器上裸二进制比 Docker 省多少</a> —— 超时配置所在的那层反代怎么部署</li>
<li><a href="/posts/api-gateway-database-choice-small-vps/">给 API 中转站选数据库：SQLite / MySQL / PostgreSQL 在小机器上的取舍</a> —— 本文说的&quot;拿用量日志对账&quot;，那张表本身也会变成问题</li>
</ul>
]]></content:encoded></item><item><title>API 中转站的限流为什么不能信 X-Forwarded-For</title><link>https://ambertu.com/posts/api-gateway-real-ip-ratelimit/</link><pubDate>Thu, 03 Sep 2026 00:00:00 +0000</pubDate><guid>https://ambertu.com/posts/api-gateway-real-ip-ratelimit/</guid><description>X-Forwarded-For 是可追加的头，客户端能自己塞——按它做限流等于没限。本文讲清追加与覆写的本质区别、后端为什么必须只监听 127.0.0.1，以及多层反代下用 real_ip_recursive 从右往左取值的正确姿势。</description><content:encoded><![CDATA[<p>自建大模型 API 中转站几乎一定要做按 IP 限流：防注册刷号、防单账号打满上游配额。而这里有一个安静失效的陷阱——<strong>如果你的限流按 <code>X-Forwarded-For</code> 计数，那它等于没做</strong>。攻击者每个请求塞一个不同的伪造 IP，计数器永远打不满。</p>
<p>根本原因一句话：<strong><code>X-Forwarded-For</code> 是追加语义，<code>X-Real-IP</code> 是覆写语义。</strong> 前者保留客户端自带的值，后者丢弃。这个区别决定了哪个可信。</p>
<h2 id="追加与覆写的区别">追加与覆写的区别</h2>
<p>nginx 里这两行长得很像，行为完全不同：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="c1"># 追加：把 $remote_addr 拼在客户端已有的 XFF 后面
</span></span></span><span class="line"><span class="cl"><span class="k">proxy_set_header</span> <span class="s">X-Forwarded-For</span> <span class="nv">$proxy_add_x_forwarded_for</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 覆写：无条件用 $remote_addr 替换，客户端塞的值被丢掉
</span></span></span><span class="line"><span class="cl"><span class="k">proxy_set_header</span> <span class="s">X-Real-IP</span> <span class="nv">$remote_addr</span><span class="p">;</span>
</span></span></code></pre></div><p><code>$proxy_add_x_forwarded_for</code> 这个变量的定义就是&quot;客户端请求里的 XFF 值 + <code>, </code> + <code>$remote_addr</code>&quot;。所以攻击者只要在请求里带上 <code>X-Forwarded-For: 1.2.3.4</code>，上游收到的就是：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">X-Forwarded-For: 1.2.3.4, 203.0.113.77
</span></span><span class="line"><span class="cl">                 ↑伪造      ↑真实（nginx 追加的）
</span></span></code></pre></div><p>而<strong>绝大多数框架和教程都教你取 XFF 的第一个值</strong>当客户端 IP——理由听起来也很合理：&ldquo;最左边是最原始的客户端&rdquo;。这个说法在没有攻击者的世界里成立，在真实互联网上是漏洞。</p>
<figure>
<svg viewBox="0 0 800 300" role="img" aria-label="对比图：使用 X-Forwarded-For 时攻击者伪造的 IP 被追加保留，应用取第一个值导致限流计数落在伪造 IP 上；使用 X-Real-IP 时 nginx 覆写该头，应用取到真实 IP，限流生效" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="b1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <g font-family="monospace" font-size="10.5" fill="currentColor">
    <!-- top lane -->
    <text x="0" y="16" opacity="0.6">❌ 按 X-Forwarded-For 限流</text>
    <rect x="0" y="30" width="120" height="44" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="60" y="49" text-anchor="middle">攻击者</text>
    <text x="60" y="64" text-anchor="middle" font-size="9" opacity="0.65">自带 XFF: 1.2.3.4</text>
    <line x1="120" y1="52" x2="196" y2="52" stroke="currentColor" stroke-width="1.2" marker-end="url(#b1)"/>
    <text x="158" y="44" text-anchor="middle" font-size="9" opacity="0.65">追加</text>
    <rect x="198" y="30" width="150" height="44" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="273" y="49" text-anchor="middle">nginx</text>
    <text x="273" y="64" text-anchor="middle" font-size="8.5" opacity="0.65">$proxy_add_x_f_f</text>
    <line x1="348" y1="52" x2="424" y2="52" stroke="currentColor" stroke-width="1.2" marker-end="url(#b1)"/>
    <rect x="426" y="30" width="230" height="44" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="541" y="48" text-anchor="middle" font-size="9.5">XFF: 1.2.3.4, 203.0.113.77</text>
    <text x="541" y="64" text-anchor="middle" font-size="9" opacity="0.65">应用取 [0] → 1.2.3.4</text>
    <line x1="656" y1="52" x2="700" y2="52" stroke="currentColor" stroke-width="1.2" marker-end="url(#b1)"/>
    <text x="712" y="49" font-size="10" font-weight="bold">限流</text>
    <text x="712" y="63" font-size="10" font-weight="bold">失效</text>
    <text x="198" y="98" font-size="9.5" opacity="0.7">每个请求换一个伪造 IP → 计数器永远打不满</text>
    <line x1="0" y1="120" x2="780" y2="120" stroke="currentColor" stroke-width="1" opacity="0.2"/>
    <!-- bottom lane -->
    <text x="0" y="150" opacity="0.6">✓ 按 X-Real-IP 限流</text>
    <rect x="0" y="164" width="120" height="44" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="60" y="183" text-anchor="middle">攻击者</text>
    <text x="60" y="198" text-anchor="middle" font-size="9" opacity="0.65">自带 Real-IP: 1.2.3.4</text>
    <line x1="120" y1="186" x2="196" y2="186" stroke="currentColor" stroke-width="1.2" marker-end="url(#b1)"/>
    <text x="158" y="178" text-anchor="middle" font-size="9" font-weight="bold">覆写</text>
    <rect x="198" y="164" width="150" height="44" fill="currentColor" fill-opacity="0.1" stroke="currentColor" stroke-width="1.3"/>
    <text x="273" y="183" text-anchor="middle">nginx</text>
    <text x="273" y="198" text-anchor="middle" font-size="8.5" opacity="0.65">$remote_addr</text>
    <line x1="348" y1="186" x2="424" y2="186" stroke="currentColor" stroke-width="1.2" marker-end="url(#b1)"/>
    <rect x="426" y="164" width="230" height="44" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="541" y="182" text-anchor="middle" font-size="9.5">X-Real-IP: 203.0.113.77</text>
    <text x="541" y="198" text-anchor="middle" font-size="9" opacity="0.65">伪造值已被丢弃</text>
    <line x1="656" y1="186" x2="700" y2="186" stroke="currentColor" stroke-width="1.2" marker-end="url(#b1)"/>
    <text x="712" y="183" font-size="10" font-weight="bold">限流</text>
    <text x="712" y="197" font-size="10" font-weight="bold">生效</text>
    <text x="198" y="232" font-size="9.5" opacity="0.7">客户端塞什么都没用，值由反代单方面决定</text>
  </g>
  <line x1="0" y1="252" x2="780" y2="252" stroke="currentColor" stroke-width="1" opacity="0.2"/>
  <text x="0" y="274" font-size="10" font-family="monospace" fill="currentColor" opacity="0.7">关键差异：追加保留了客户端的输入，覆写没有。可信的是覆写。</text>
  <text x="0" y="292" font-size="10" font-family="monospace" fill="currentColor" opacity="0.7">前提：nginx 必须是唯一入口 —— 见下一节</text>
</svg>
<figcaption>两条路径的唯一区别是 nginx 那一格用了追加还是覆写。<strong>「取最左边是最原始客户端」这条常见说法，恰恰是最左边那个值完全由攻击者控制。</strong></figcaption>
</figure>
<h2 id="前提反代必须是唯一入口">前提：反代必须是唯一入口</h2>
<p><code>X-Real-IP</code> 可信有一个隐含前提——<strong>所有流量都必须经过反代</strong>。如果后端端口自己也在公网上听着，攻击者直连后端，想塞什么头就塞什么头，覆写逻辑根本不会执行。</p>
<p>这个疏漏比想象中普遍。写这篇文章的时候我顺手扫了自己的机器，发现两个问题：一个 Go 服务监听了 <code>0.0.0.0</code> 而不是 <code>127.0.0.1</code>，公网直接可达；更糟的是数据库端口也在安全组里开着，从外网能直接拿到 MariaDB 的协议握手包和版本号。两个都是历史遗留，跑了很久没人注意——<strong>因为从正常入口访问一切都是对的，问题只在你主动去扫端口时才会现形。</strong></p>
<figure>
<svg viewBox="0 0 760 250" role="img" aria-label="示意图：后端只监听 127.0.0.1 时攻击者无法绕过 nginx；后端监听 0.0.0.0 且安全组开放时攻击者可直连后端并自带任意请求头" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="b2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <g font-family="monospace" font-size="10.5" fill="currentColor">
    <text x="0" y="16" opacity="0.6">✓ 后端只监听 127.0.0.1</text>
    <rect x="0" y="30" width="110" height="38" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="55" y="53" text-anchor="middle">客户端</text>
    <line x1="110" y1="49" x2="186" y2="49" stroke="currentColor" stroke-width="1.2" marker-end="url(#b2)"/>
    <rect x="188" y="30" width="150" height="38" fill="currentColor" fill-opacity="0.1" stroke="currentColor" stroke-width="1.3"/>
    <text x="263" y="53" text-anchor="middle">nginx :443</text>
    <line x1="338" y1="49" x2="414" y2="49" stroke="currentColor" stroke-width="1.2" marker-end="url(#b2)"/>
    <text x="376" y="41" text-anchor="middle" font-size="9" opacity="0.65">覆写头</text>
    <rect x="416" y="30" width="180" height="38" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="506" y="53" text-anchor="middle" font-size="9.5">127.0.0.1:8081</text>
    <path d="M 55 68 L 55 92 L 506 92" fill="none" stroke="currentColor" stroke-width="1.2" stroke-dasharray="4 3" opacity="0.5"/>
    <text x="616" y="53" font-size="9.5" opacity="0.7">唯一入口</text>
    <text x="230" y="108" font-size="9.5" opacity="0.7">直连尝试 → 连接被拒，没有第二条路</text>
    <line x1="0" y1="128" x2="740" y2="128" stroke="currentColor" stroke-width="1" opacity="0.2"/>
    <text x="0" y="156" opacity="0.6">❌ 后端监听 0.0.0.0 且安全组开放</text>
    <rect x="0" y="170" width="110" height="38" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="55" y="193" text-anchor="middle">攻击者</text>
    <path d="M 110 189 L 200 189 L 200 216 L 414 216" fill="none" stroke="currentColor" stroke-width="1.4" marker-end="url(#b2)"/>
    <text x="238" y="234" font-size="9.5" font-weight="bold">绕过 nginx，自带任意 X-Real-IP</text>
    <rect x="188" y="170" width="150" height="38" fill="none" stroke="currentColor" stroke-width="1.3" stroke-dasharray="4 3" opacity="0.45"/>
    <text x="263" y="193" text-anchor="middle" opacity="0.45">nginx :443</text>
    <rect x="416" y="170" width="180" height="38" fill="currentColor" fill-opacity="0.1" stroke="currentColor" stroke-width="1.4"/>
    <text x="506" y="188" text-anchor="middle" font-size="9.5">0.0.0.0:8980</text>
    <text x="506" y="202" text-anchor="middle" font-size="8.5" opacity="0.7">公网可达</text>
    <text x="616" y="193" font-size="9.5" font-weight="bold">全部防护</text>
    <text x="616" y="206" font-size="9.5" font-weight="bold">被跳过</text>
  </g>
</svg>
<figcaption>下半图里被跳过的不只是 IP 覆写——TLS、限流、鉴权、路径前缀剥离，凡是在 nginx 层做的全部失效。<strong>后端监听地址是一道比任何头处理都更靠前的防线。</strong></figcaption>
</figure>
<p>自查两条命令，五秒钟的事：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># 本机看监听地址：凡是 0.0.0.0 或 * 开头的都要问一句&#34;为什么&#34;</span>
</span></span><span class="line"><span class="cl">ss -tlnp
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># 从外网验证（换成你的公网 IP 和端口），能连上就是有问题</span>
</span></span><span class="line"><span class="cl">curl -sv -m <span class="m">5</span> http://YOUR_PUBLIC_IP:8080/ 2&gt;<span class="p">&amp;</span><span class="m">1</span> <span class="p">|</span> head -5
</span></span></code></pre></div><p>第二条必须<strong>从机器外面</strong>跑。在机器上自己 curl 自己永远是通的，测不出安全组的实际策略。</p>
<p>修法按优先级：</p>
<ol>
<li><strong>应用只监听本地</strong>——大多数框架都能配，Go 里就是 <code>http.ListenAndServe(&quot;127.0.0.1:8081&quot;, h)</code> 而不是 <code>&quot;:8081&quot;</code></li>
<li><strong>安全组只开 22 / 80 / 443</strong>——这是最后一道闸，也是最不容易被应用改动破坏的一道</li>
<li><strong>数据库一律不开公网</strong>——需要远程访问就走 SSH 隧道：<code>ssh -L 3306:127.0.0.1:3306 user@host</code></li>
</ol>
<h2 id="正确的限流写法">正确的限流写法</h2>
<h3 id="nginx-层限流天然安全">nginx 层限流：天然安全</h3>
<p>如果限流在 nginx 做，用 <code>$binary_remote_addr</code> 就好——它取的是 TCP 连接的对端地址，<strong>不是任何请求头</strong>，客户端无法伪造：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="c1"># http 块
</span></span></span><span class="line"><span class="cl"><span class="k">limit_req_zone</span> <span class="nv">$binary_remote_addr</span> <span class="s">zone=api:10m</span> <span class="s">rate=10r/s</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">location</span> <span class="s">/v1/</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kn">limit_req</span> <span class="s">zone=api</span> <span class="s">burst=20</span> <span class="s">nodelay</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">limit_req_status</span> <span class="mi">429</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_pass</span> <span class="s">http://127.0.0.1:3000/v1/</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p>用 <code>$binary_remote_addr</code> 而不是 <code>$remote_addr</code> 是内存考虑：前者是 4 字节（IPv4），后者是字符串，在 10m 的 zone 里能存的条目数差好几倍。</p>
<h3 id="应用层限流只信覆写过的头">应用层限流：只信覆写过的头</h3>
<p>如果限流逻辑在应用里（比如要按用户 + IP 组合计数），那就只认 <code>X-Real-IP</code>：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="c1">// ✓ 只信反代覆写的头，且兜底用 TCP 对端地址</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">clientIP</span><span class="p">(</span><span class="nx">r</span><span class="w"> </span><span class="o">*</span><span class="nx">http</span><span class="p">.</span><span class="nx">Request</span><span class="p">)</span><span class="w"> </span><span class="kt">string</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="k">if</span><span class="w"> </span><span class="nx">ip</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">r</span><span class="p">.</span><span class="nx">Header</span><span class="p">.</span><span class="nf">Get</span><span class="p">(</span><span class="s">&#34;X-Real-IP&#34;</span><span class="p">);</span><span class="w"> </span><span class="nx">ip</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="s">&#34;&#34;</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="k">return</span><span class="w"> </span><span class="nx">ip</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">host</span><span class="p">,</span><span class="w"> </span><span class="nx">_</span><span class="p">,</span><span class="w"> </span><span class="nx">_</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">net</span><span class="p">.</span><span class="nf">SplitHostPort</span><span class="p">(</span><span class="nx">r</span><span class="p">.</span><span class="nx">RemoteAddr</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="k">return</span><span class="w"> </span><span class="nx">host</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p><strong>绝对不要</strong>写成这样：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="c1">// ❌ 取 XFF 第一个值 —— 完全由客户端控制</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">clientIP</span><span class="p">(</span><span class="nx">r</span><span class="w"> </span><span class="o">*</span><span class="nx">http</span><span class="p">.</span><span class="nx">Request</span><span class="p">)</span><span class="w"> </span><span class="kt">string</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="k">if</span><span class="w"> </span><span class="nx">xff</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">r</span><span class="p">.</span><span class="nx">Header</span><span class="p">.</span><span class="nf">Get</span><span class="p">(</span><span class="s">&#34;X-Forwarded-For&#34;</span><span class="p">);</span><span class="w"> </span><span class="nx">xff</span><span class="w"> </span><span class="o">!=</span><span class="w"> </span><span class="s">&#34;&#34;</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="k">return</span><span class="w"> </span><span class="nx">strings</span><span class="p">.</span><span class="nf">Split</span><span class="p">(</span><span class="nx">xff</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;,&#34;</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="w">  </span><span class="c1">// 攻击者说什么就是什么</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="o">...</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>我自己的 pet 服务在这一处踩过，最后在 nginx 配置里留了行注释提醒自己：<strong><code>X-Real-IP</code> 不能少，注册限流只认这个头，因为 XFF 客户端能自己塞。</strong> 这类结论写在配置文件旁边比写在文档里有用得多——改配置的人一定会看到它。</p>
<h2 id="前面还有-cdn-怎么办">前面还有 CDN 怎么办</h2>
<p>如果套了 CDN，<code>$remote_addr</code> 是 CDN 节点的 IP，直接覆写 <code>X-Real-IP</code> 会把全站流量都算成同几个 CDN 节点，限流立刻误伤所有人。这时候要用 nginx 的 <code>realip</code> 模块，而不是自己解析 XFF：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="c1"># 声明哪些来源是可信代理（填 CDN 的回源网段，各家文档都有公布）
</span></span></span><span class="line"><span class="cl"><span class="k">set_real_ip_from</span> <span class="n">203.0.113.0</span><span class="s">/24</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="k">set_real_ip_from</span> <span class="n">198.51.100.0</span><span class="s">/24</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">real_ip_header</span>    <span class="s">X-Forwarded-For</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="k">real_ip_recursive</span> <span class="no">on</span><span class="p">;</span>          <span class="c1"># 这一行是安全的关键
</span></span></span></code></pre></div><p>配好之后 <code>$remote_addr</code> 会被改写成真实客户端 IP，前面所有基于 <code>$remote_addr</code> 的限流、<code>X-Real-IP</code> 覆写都自动正确，不用改一行业务代码。</p>
<p><strong><code>real_ip_recursive on</code> 为什么是安全的关键</strong>，值得画一下：</p>
<figure>
<svg viewBox="0 0 760 230" role="img" aria-label="示意图：real_ip_recursive 开启时 nginx 从 X-Forwarded-For 最右侧开始向左扫描，跳过属于可信代理网段的地址，取第一个非可信地址作为真实客户端 IP，因此最左侧攻击者伪造的值永远不会被取到" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="b3" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <text x="0" y="16" font-size="10.5" font-family="monospace" fill="currentColor" opacity="0.6">X-Forwarded-For 的实际内容</text>
  <g font-family="monospace" font-size="10.5" fill="currentColor">
    <rect x="0" y="34" width="180" height="42" fill="none" stroke="currentColor" stroke-width="1.3" stroke-dasharray="4 3"/>
    <text x="90" y="52" text-anchor="middle" font-size="9.5">1.2.3.4</text>
    <text x="90" y="68" text-anchor="middle" font-size="9" opacity="0.7">攻击者伪造</text>
    <text x="190" y="59" opacity="0.5">,</text>
    <rect x="204" y="34" width="200" height="42" fill="currentColor" fill-opacity="0.12" stroke="currentColor" stroke-width="1.5"/>
    <text x="304" y="52" text-anchor="middle" font-size="9.5">203.0.113.77</text>
    <text x="304" y="68" text-anchor="middle" font-size="9" opacity="0.75">真实客户端</text>
    <text x="414" y="59" opacity="0.5">,</text>
    <rect x="428" y="34" width="200" height="42" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="528" y="52" text-anchor="middle" font-size="9.5">203.0.113.9</text>
    <text x="528" y="68" text-anchor="middle" font-size="9" opacity="0.7">CDN 节点（可信）</text>
  </g>
  <path d="M 620 100 L 540 100" fill="none" stroke="currentColor" stroke-width="1.4" marker-end="url(#b3)"/>
  <text x="628" y="104" font-size="9.5" font-family="monospace" fill="currentColor">从右开始扫</text>
  <g font-family="monospace" font-size="9.5" fill="currentColor">
    <text x="428" y="126">① 在 set_real_ip_from 里 → 跳过</text>
    <text x="204" y="146">② 不在可信列表里 → 取它，扫描停止 ✓</text>
    <text x="0" y="166" opacity="0.6">③ 永远扫不到这里</text>
  </g>
  <line x1="0" y1="184" x2="740" y2="184" stroke="currentColor" stroke-width="1" opacity="0.2"/>
  <text x="0" y="206" font-size="10" font-family="monospace" fill="currentColor" opacity="0.75">扫描一旦遇到非可信地址就停止，所以左侧的伪造值不可能被取到。</text>
  <text x="0" y="224" font-size="10" font-family="monospace" fill="currentColor" opacity="0.75">安全性完全依赖 set_real_ip_from 填得准 —— 填宽了等于自己把门打开。</text>
</svg>
<figcaption><strong>方向就是安全性本身。</strong>从右往左扫、遇到非可信地址即停，意味着攻击者在左边塞多少个伪造 IP 都到不了。反过来从左往右取第一个，就是本文开头那个漏洞。</figcaption>
</figure>
<p>两个配置陷阱：</p>
<ul>
<li><strong><code>real_ip_recursive off</code>（默认值）在有 CDN 时是错的。</strong> 关闭递归时 nginx 只取 XFF 的<strong>最后一个</strong>地址，那就是 CDN 节点自己，等于没解析。</li>
<li><strong><code>set_real_ip_from</code> 千万别填 <code>0.0.0.0/0</code>。</strong> 把所有来源都标为可信，递归就会一路扫到最左边——正好扫到攻击者伪造的那个值，比不配还糟。见过有人为了&quot;省事&quot;这么写。</li>
</ul>
<h2 id="一份自查清单">一份自查清单</h2>
<table>
	<thead>
			<tr>
					<th>检查项</th>
					<th>命令 / 位置</th>
					<th>期望</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>后端监听地址</td>
					<td><code>ss -tlnp</code></td>
					<td>只有反代和 SSH 在 <code>0.0.0.0</code>，业务端口都在 <code>127.0.0.1</code></td>
			</tr>
			<tr>
					<td>外网可达端口</td>
					<td>从<strong>外部机器</strong> <code>curl</code> 各端口</td>
					<td>只有 22 / 80 / 443 通</td>
			</tr>
			<tr>
					<td>数据库端口</td>
					<td><code>ss -tlnp | grep -E '3306|5432'</code></td>
					<td>绑 <code>127.0.0.1</code>，且安全组不开</td>
			</tr>
			<tr>
					<td>应用取 IP 的逻辑</td>
					<td>搜代码里的 <code>X-Forwarded-For</code></td>
					<td>不存在取 <code>[0]</code> 的写法</td>
			</tr>
			<tr>
					<td>nginx 是否覆写</td>
					<td>搜配置 <code>X-Real-IP</code></td>
					<td>每个 <code>proxy_pass</code> 的 location 都有</td>
			</tr>
			<tr>
					<td>CDN 场景</td>
					<td>搜配置 <code>real_ip_recursive</code></td>
					<td>为 <code>on</code>，且 <code>set_real_ip_from</code> 是具体网段</td>
			</tr>
	</tbody>
</table>
<h2 id="相关阅读">相关阅读</h2>
<ul>
<li><a href="/posts/llm-api-sse-proxy-buffering/">反代把大模型 API 的 SSE 流式响应缓冲了：五个缓冲点与三层排查法</a> —— 同一份 nginx 配置里的另一个坑</li>
<li><a href="/posts/new-api-binary-vs-docker/">new-api 部署选型：2G 小内存机器上裸二进制比 Docker 省多少</a> —— 后端监听 <code>127.0.0.1</code> 在两种部署形态下怎么配</li>
</ul>
]]></content:encoded></item><item><title>反代把大模型 API 的 SSE 流式响应缓冲了：五个缓冲点与三层排查法</title><link>https://ambertu.com/posts/llm-api-sse-proxy-buffering/</link><pubDate>Thu, 03 Sep 2026 00:00:00 +0000</pubDate><guid>https://ambertu.com/posts/llm-api-sse-proxy-buffering/</guid><description>大模型 API 中转后流式输出变成一次性返回，首字节延迟等于整个生成时长？问题几乎不在应用。本文列出请求路径上真实存在的五个缓冲点，给出一套从应用往外逐层二分的排查方法。</description><content:encoded><![CDATA[<p>自建大模型 API 中转后最常见的故障是：<strong>后端明明是流式的，经过反向代理就变成一次性返回</strong>——客户端等十几秒没有任何输出，然后整段文字一次吐完。首字节延迟几乎等于整个生成时长，流式的意义全部丧失。</p>
<p>这个问题九成不在应用代码里，而在 <strong>nginx 的 <code>proxy_buffering</code> 默认是开启的</strong>。但真正麻烦的地方是：请求路径上会缓冲的地方不止 nginx 一个，实际有五个，而且<strong>其中一个是你用来排查的 curl 自己</strong>。如果不知道这一点，你会在错误的层上改半天配置。</p>
<h2 id="先分清-sse-和-websocket">先分清 SSE 和 WebSocket</h2>
<p>这是配置写错的根源。大模型 API 的流式输出用的是 <strong>SSE（Server-Sent Events）</strong>：一个普通的 HTTP 响应，<code>Content-Type: text/event-stream</code>，靠 chunked 编码持续往下写。<strong>它不涉及协议升级。</strong></p>
<p>而网上大量&quot;流式接口反代配置&quot;教程给的是 WebSocket 的写法：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="c1"># ❌ 这是 WebSocket 的配置，SSE 不需要，而且硬编码有害
</span></span></span><span class="line"><span class="cl"><span class="k">proxy_set_header</span> <span class="s">Upgrade</span> <span class="nv">$http_upgrade</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="k">proxy_set_header</span> <span class="s">Connection</span> <span class="s">&#34;upgrade&#34;</span><span class="p">;</span>
</span></span></code></pre></div><p>问题出在第二行<strong>硬编码</strong>了 <code>&quot;upgrade&quot;</code>：它会给经过这个 location 的<strong>所有</strong>请求都带上 <code>Connection: upgrade</code>，包括根本没有 <code>Upgrade</code> 头的普通请求和 SSE 请求。<code>Connection</code> 是逐跳（hop-by-hop）首部，被这么滥用会破坏到上游的 keepalive，某些上游实现还会因此行为异常。</p>
<p>正确的写法是用 <code>map</code> 按客户端是否真的请求升级来取值：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="c1"># http 块内
</span></span></span><span class="line"><span class="cl"><span class="k">map</span> <span class="nv">$http_upgrade</span> <span class="nv">$connection_upgrade</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kn">default</span> <span class="s">upgrade</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">&#39;&#39;</span>      <span class="s">close</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># location 内
</span></span></span><span class="line"><span class="cl"><span class="k">proxy_http_version</span> <span class="mi">1</span><span class="s">.1</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="k">proxy_set_header</span> <span class="s">Upgrade</span>    <span class="nv">$http_upgrade</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="k">proxy_set_header</span> <span class="s">Connection</span> <span class="nv">$connection_upgrade</span><span class="p">;</span>
</span></span></code></pre></div><p>如果这个 location 只服务 SSE、完全没有 WebSocket，那更简单——两行都不要写，改成让 nginx 复用到上游的连接：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="k">proxy_http_version</span> <span class="mi">1</span><span class="s">.1</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="k">proxy_set_header</span> <span class="s">Connection</span> <span class="s">&#34;&#34;</span><span class="p">;</span>
</span></span></code></pre></div><h2 id="请求路径上真实存在的五个缓冲点">请求路径上真实存在的五个缓冲点</h2>
<figure>
<svg viewBox="0 0 800 240" role="img" aria-label="从应用到客户端的请求路径上有五个缓冲点：应用是否调用 flush、nginx 的 proxy_buffering、nginx 的 gzip 压缩、CDN 层缓冲、以及客户端 curl 自身的输出缓冲" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="a1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <text x="0" y="14" font-size="11" font-family="monospace" fill="currentColor" opacity="0.6">token 的流动方向 →</text>
  <g font-family="monospace" font-size="11.5" fill="currentColor">
    <rect x="0" y="42" width="118" height="42" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="59" y="68" text-anchor="middle">应用</text>
    <line x1="118" y1="63" x2="198" y2="63" stroke="currentColor" stroke-width="1.2" marker-end="url(#a1)"/>
    <rect x="200" y="42" width="118" height="42" fill="currentColor" fill-opacity="0.09" stroke="currentColor" stroke-width="1.3"/>
    <text x="259" y="68" text-anchor="middle">nginx</text>
    <line x1="318" y1="63" x2="398" y2="63" stroke="currentColor" stroke-width="1.2" marker-end="url(#a1)"/>
    <rect x="400" y="42" width="118" height="42" fill="none" stroke="currentColor" stroke-width="1.3" stroke-dasharray="4 3"/>
    <text x="459" y="62" text-anchor="middle">CDN</text>
    <text x="459" y="77" text-anchor="middle" font-size="9" opacity="0.6">若有</text>
    <line x1="518" y1="63" x2="598" y2="63" stroke="currentColor" stroke-width="1.2" marker-end="url(#a1)"/>
    <rect x="600" y="42" width="118" height="42" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="659" y="68" text-anchor="middle">客户端</text>
  </g>
  <g font-family="monospace" font-size="9.5" fill="currentColor">
    <text x="0" y="112" opacity="0.55">默认</text>
    <text x="0" y="130">看代码</text>
    <text x="0" y="158" opacity="0.55">开关</text>
    <text x="0" y="176">flusher.Flush()</text>
    <text x="200" y="112" opacity="0.55">默认</text>
    <text x="200" y="130" font-weight="bold">缓冲 ⚠ 主因</text>
    <text x="200" y="158" opacity="0.55">开关</text>
    <text x="200" y="176">proxy_buffering off</text>
    <text x="200" y="194">gzip off ← 第二个</text>
    <text x="400" y="112" opacity="0.55">默认</text>
    <text x="400" y="130">多数缓冲 ⚠</text>
    <text x="400" y="158" opacity="0.55">开关</text>
    <text x="400" y="176">控制台流式配置</text>
    <text x="600" y="112" opacity="0.55">默认</text>
    <text x="600" y="130">curl 自己缓冲 ⚠</text>
    <text x="600" y="158" opacity="0.55">开关</text>
    <text x="600" y="176">curl -N</text>
  </g>
  <line x1="0" y1="212" x2="780" y2="212" stroke="currentColor" stroke-width="1" opacity="0.2"/>
  <text x="0" y="232" font-size="10" font-family="monospace" fill="currentColor" opacity="0.6">共 5 个缓冲点：应用 flush · proxy_buffering · gzip · CDN · 客户端</text>
</svg>
<figcaption>nginx 那一格是灰底的，因为它同时贡献了两个缓冲点：<code>proxy_buffering</code> 和 <code>gzip</code>。<strong>最容易误导排查的是最右边那格</strong>——不加 <code>-N</code> 的 curl 自己会缓冲输出，你会以为是服务端问题。</figcaption>
</figure>
<p>第五个缓冲点最坑人，所以单独强调：<strong><code>curl</code> 默认会缓冲输出</strong>。用不带 <code>-N</code> 的 curl 去验证流式，看到的永远是&quot;一次性返回&quot;，无论服务端配得对不对。排查流式问题的第一条纪律是所有 curl 都加 <code>-N</code>（等价于 <code>--no-buffer</code>）。</p>
<h2 id="症状长什么样">症状长什么样</h2>
<figure>
<svg viewBox="0 0 720 180" role="img" aria-label="示意图：缓冲开启时客户端长时间无输出然后一次性收到全部内容；缓冲关闭时 token 从一开始就持续到达" style="width:100%;height:auto;color:currentColor">
  <text x="0" y="14" font-size="11" font-family="monospace" fill="currentColor" opacity="0.6">客户端收到数据的时间轴（示意）</text>
  <g font-family="monospace" font-size="10.5" fill="currentColor">
    <text x="0" y="48">缓冲开启</text>
    <line x1="130" y1="52" x2="660" y2="52" stroke="currentColor" stroke-width="1" opacity="0.25"/>
    <rect x="560" y="38" width="100" height="18" fill="currentColor" opacity="0.75"/>
    <text x="130" y="76" opacity="0.6">↑ 全程无输出 ——————————————————→ 一次吐完</text>
    <text x="0" y="118">缓冲关闭</text>
    <line x1="130" y1="122" x2="660" y2="122" stroke="currentColor" stroke-width="1" opacity="0.25"/>
    <rect x="140" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="162" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="184" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="206" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="228" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="250" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="272" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="294" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="316" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="338" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="360" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <rect x="382" y="108" width="14" height="18" fill="currentColor" opacity="0.75"/>
    <text x="140" y="146" opacity="0.6">↑ 首字节很快，token 持续到达</text>
  </g>
  <text x="0" y="174" font-size="10" font-family="monospace" fill="currentColor" opacity="0.6">本图为示意，非实测数据</text>
</svg>
<figcaption>判据不是"感觉慢"，而是一个可量化的信号：<strong>首字节时间约等于总时长</strong>。下一节给出怎么量。</figcaption>
</figure>
<p>用 curl 的计时字段可以把这件事量化，不用凭感觉：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">curl -N -o /dev/null -s <span class="se">\
</span></span></span><span class="line"><span class="cl">  -w <span class="s2">&#34;首字节 %{time_starttransfer}s / 总计 %{time_total}s\n&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -H <span class="s2">&#34;Authorization: Bearer </span><span class="nv">$KEY</span><span class="s2">&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -H <span class="s2">&#34;Content-Type: application/json&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  -d <span class="s1">&#39;{&#34;model&#34;:&#34;gpt-4o-mini&#34;,&#34;stream&#34;:true,&#34;messages&#34;:[{&#34;role&#34;:&#34;user&#34;,&#34;content&#34;:&#34;数到二十&#34;}]}&#39;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl">  https://your-domain/v1/chat/completions
</span></span></code></pre></div><ul>
<li><code>首字节 ≈ 总计</code> → 被完整缓冲了</li>
<li><code>首字节 ≪ 总计</code> → 流式正常</li>
</ul>
<p>另一个免费信号在响应头里：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">curl -N -D - -o /dev/null -s ... <span class="p">|</span> grep -iE <span class="s1">&#39;content-length|transfer-encoding|content-type&#39;</span>
</span></span></code></pre></div><p><strong>出现 <code>Content-Length</code> 就说明响应被完整缓冲过</strong>——只有知道全部长度才能写出这个头。正常流式应该是 <code>Transfer-Encoding: chunked</code>（HTTP/1.1）或 HTTP/2 的分帧，加上 <code>Content-Type: text/event-stream</code>。</p>
<h2 id="三层二分排查法">三层二分排查法</h2>
<p>不要直接去猜哪一层，按下面这棵树走，两三条命令就能定位。核心思路是<strong>从最里层往外逐层加一跳</strong>，第一次出问题的那跳就是元凶。</p>
<figure>
<svg viewBox="0 0 780 350" role="img" aria-label="排查决策树：先用 curl -N 直连应用确认后端本身流式，再经 nginx，再经 CDN，每一步不流式就定位到该层" style="width:100%;height:auto;color:currentColor">
  <defs>
    <marker id="a2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto">
      <path d="M 0 1 L 10 5 L 0 9 z" fill="currentColor"/>
    </marker>
  </defs>
  <g font-family="monospace" font-size="10.5" fill="currentColor">
    <rect x="180" y="14" width="270" height="38" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="315" y="31" text-anchor="middle">① curl -N 直连 127.0.0.1:3000</text>
    <text x="315" y="45" text-anchor="middle" opacity="0.6">绕过全部反代</text>
    <line x1="200" y1="52" x2="120" y2="88" stroke="currentColor" stroke-width="1.2" marker-end="url(#a2)"/>
    <text x="140" y="74" text-anchor="end" opacity="0.75">不流式</text>
    <rect x="10" y="90" width="200" height="38" fill="currentColor" fill-opacity="0.09" stroke="currentColor" stroke-width="1.3"/>
    <text x="110" y="107" text-anchor="middle">应用没有 flush</text>
    <text x="110" y="121" text-anchor="middle" opacity="0.6">改代码，见下节</text>
    <line x1="360" y1="52" x2="430" y2="88" stroke="currentColor" stroke-width="1.2" marker-end="url(#a2)"/>
    <text x="404" y="74" opacity="0.75">流式 ✓</text>
    <rect x="330" y="90" width="270" height="38" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="465" y="107" text-anchor="middle">② curl -N 经 nginx（内网 IP）</text>
    <text x="465" y="121" text-anchor="middle" opacity="0.6">加上反代这一跳</text>
    <line x1="350" y1="128" x2="230" y2="164" stroke="currentColor" stroke-width="1.2" marker-end="url(#a2)"/>
    <text x="252" y="150" text-anchor="end" opacity="0.75">不流式</text>
    <rect x="60" y="166" width="230" height="52" fill="currentColor" fill-opacity="0.09" stroke="currentColor" stroke-width="1.3"/>
    <text x="175" y="183" text-anchor="middle">nginx 缓冲</text>
    <text x="175" y="197" text-anchor="middle" opacity="0.6">proxy_buffering / gzip</text>
    <text x="175" y="211" text-anchor="middle" opacity="0.6">最常见的一档</text>
    <line x1="520" y1="128" x2="570" y2="164" stroke="currentColor" stroke-width="1.2" marker-end="url(#a2)"/>
    <text x="548" y="150" opacity="0.75">流式 ✓</text>
    <rect x="450" y="166" width="270" height="38" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="585" y="183" text-anchor="middle">③ curl -N 经公网域名</text>
    <text x="585" y="197" text-anchor="middle" opacity="0.6">加上 CDN 这一跳</text>
    <line x1="480" y1="204" x2="400" y2="242" stroke="currentColor" stroke-width="1.2" marker-end="url(#a2)"/>
    <text x="420" y="228" text-anchor="end" opacity="0.75">不流式</text>
    <rect x="250" y="244" width="200" height="38" fill="currentColor" fill-opacity="0.09" stroke="currentColor" stroke-width="1.3"/>
    <text x="350" y="261" text-anchor="middle">CDN 在缓冲</text>
    <text x="350" y="275" text-anchor="middle" opacity="0.6">去控制台配流式</text>
    <line x1="650" y1="204" x2="680" y2="242" stroke="currentColor" stroke-width="1.2" marker-end="url(#a2)"/>
    <text x="670" y="228" opacity="0.75">流式 ✓</text>
    <rect x="530" y="244" width="240" height="52" fill="none" stroke="currentColor" stroke-width="1.3"/>
    <text x="650" y="261" text-anchor="middle">服务端没问题</text>
    <text x="650" y="275" text-anchor="middle" opacity="0.6">查客户端：漏了 -N，</text>
    <text x="650" y="289" text-anchor="middle" opacity="0.6">或 SDK 未开 stream</text>
  </g>
  <text x="0" y="330" font-size="10" font-family="monospace" fill="currentColor" opacity="0.6">每一步都必须带 -N，否则测的是 curl 自己的缓冲</text>
</svg>
<figcaption>这套方法的价值在于<strong>每次只增加一跳</strong>，所以定位是确定的，不需要靠猜。灰底的三个是终点结论。</figcaption>
</figure>
<h2 id="nginx-侧的完整配置">nginx 侧的完整配置</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-nginx" data-lang="nginx"><span class="line"><span class="cl"><span class="k">location</span> <span class="s">/v1/</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_pass</span> <span class="s">http://127.0.0.1:3000/v1/</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># 关掉响应缓冲 —— 解决问题的主开关
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_buffering</span>    <span class="no">off</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_request_buffering</span> <span class="no">off</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_cache</span>        <span class="no">off</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># gzip 压缩也会攒数据，流式 location 里必须关
</span></span></span><span class="line"><span class="cl">    <span class="kn">gzip</span>               <span class="no">off</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># chunked 需要 HTTP/1.1；清空 Connection 让 nginx 复用上游连接
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_http_version</span> <span class="mi">1</span><span class="s">.1</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_set_header</span>   <span class="s">Connection</span> <span class="s">&#34;&#34;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1"># 长生成会超过默认的 60s，放宽读超时
</span></span></span><span class="line"><span class="cl">    <span class="kn">proxy_read_timeout</span> <span class="s">600s</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_send_timeout</span> <span class="s">600s</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_set_header</span>   <span class="s">Host</span>      <span class="nv">$host</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kn">proxy_set_header</span>   <span class="s">X-Real-IP</span> <span class="nv">$remote_addr</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p>几个容易漏的点：</p>
<ul>
<li><strong><code>gzip off</code> 不是可选项。</strong> nginx 的 gzip 模块要攒够数据才能压缩，开着它等于重新引入缓冲。很多人只关了 <code>proxy_buffering</code> 就以为完事，结果症状减轻但没消失，就是 gzip 在作祟。</li>
<li><strong><code>proxy_read_timeout</code> 默认 60 秒。</strong> 长上下文生成很容易超，超了会收到截断的响应或 504，且现象酷似&quot;流式坏了&quot;。</li>
<li><strong><code>proxy_request_buffering off</code></strong> 管的是请求方向，上传大 prompt 时有用，和响应流式无关但顺手一起配。</li>
</ul>
<h3 id="不想改-nginx-配置用响应头">不想改 nginx 配置？用响应头</h3>
<p>如果你没有 nginx 的配置权限，或者不想为一个接口改全局配置，<strong>应用可以自己让 nginx 关掉缓冲</strong>——返回这个响应头：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">X-Accel-Buffering: no
</span></span></code></pre></div><p>nginx 识别这个头并对该响应禁用缓冲。这是 nginx 特有的机制，非常实用：<strong>开关从运维侧移到了应用侧</strong>，随代码走，不会因为换机器忘配而复发。</p>
<h2 id="caddy-侧">Caddy 侧</h2>
<p>Caddy 的 <code>reverse_proxy</code> 对 <code>text/event-stream</code> 会自动刷新，多数情况下开箱即用。如果确实需要强制：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-caddyfile" data-lang="caddyfile"><span class="line"><span class="cl"><span class="gh">your-domain.com</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="k">reverse_proxy</span> <span class="n">127.0.0.1</span><span class="p">:</span><span class="mi">3000</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="k">flush_interval</span> <span class="s">-1</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p><code>flush_interval -1</code> 表示每次写入立即刷新。另外 Caddy 默认会对文本类型启用压缩，如果显式配了 <code>encode gzip</code>，记得把流式路径排除掉，理由和 nginx 的 <code>gzip off</code> 一样。</p>
<h2 id="应用侧反代配对了也可能不流式">应用侧：反代配对了也可能不流式</h2>
<p>如果排查树的第一步就不通过，那是应用自己没有 flush。反代关掉缓冲只是&quot;不主动攒&quot;，它没法把应用攒着不发的数据变出来。Go 里的标准写法：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-go" data-lang="go"><span class="line"><span class="cl"><span class="kd">func</span><span class="w"> </span><span class="nf">stream</span><span class="p">(</span><span class="nx">w</span><span class="w"> </span><span class="nx">http</span><span class="p">.</span><span class="nx">ResponseWriter</span><span class="p">,</span><span class="w"> </span><span class="nx">r</span><span class="w"> </span><span class="o">*</span><span class="nx">http</span><span class="p">.</span><span class="nx">Request</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">flusher</span><span class="p">,</span><span class="w"> </span><span class="nx">ok</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="nx">w</span><span class="p">.(</span><span class="nx">http</span><span class="p">.</span><span class="nx">Flusher</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="k">if</span><span class="w"> </span><span class="p">!</span><span class="nx">ok</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nx">http</span><span class="p">.</span><span class="nf">Error</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;streaming unsupported&#34;</span><span class="p">,</span><span class="w"> </span><span class="nx">http</span><span class="p">.</span><span class="nx">StatusInternalServerError</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="k">return</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">w</span><span class="p">.</span><span class="nf">Header</span><span class="p">().</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;Content-Type&#34;</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;text/event-stream&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">w</span><span class="p">.</span><span class="nf">Header</span><span class="p">().</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;Cache-Control&#34;</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;no-cache&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">w</span><span class="p">.</span><span class="nf">Header</span><span class="p">().</span><span class="nf">Set</span><span class="p">(</span><span class="s">&#34;X-Accel-Buffering&#34;</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;no&#34;</span><span class="p">)</span><span class="w"> </span><span class="c1">// 顺手把 nginx 也关掉</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="k">for</span><span class="w"> </span><span class="nx">chunk</span><span class="w"> </span><span class="o">:=</span><span class="w"> </span><span class="k">range</span><span class="w"> </span><span class="nx">chunks</span><span class="w"> </span><span class="p">{</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Fprintf</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;data: %s\n\n&#34;</span><span class="p">,</span><span class="w"> </span><span class="nx">chunk</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nx">flusher</span><span class="p">.</span><span class="nf">Flush</span><span class="p">()</span><span class="w"> </span><span class="c1">// 少了这行，前面全白配</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="p">}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">fmt</span><span class="p">.</span><span class="nf">Fprint</span><span class="p">(</span><span class="nx">w</span><span class="p">,</span><span class="w"> </span><span class="s">&#34;data: [DONE]\n\n&#34;</span><span class="p">)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nx">flusher</span><span class="p">.</span><span class="nf">Flush</span><span class="p">()</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="p">}</span><span class="w">
</span></span></span></code></pre></div><p>注意 <code>w.Header().Set</code> 必须在第一次 <code>Write</code> 之前调用，否则不生效——这是另一个安静失败的经典来源。</p>
<h2 id="相关阅读">相关阅读</h2>
<ul>
<li><a href="/posts/new-api-binary-vs-docker/">new-api 部署选型：2G 小内存机器上裸二进制比 Docker 省多少</a> —— 反代前面那一层，部署形态怎么选</li>
<li><a href="/posts/api-gateway-real-ip-ratelimit/">API 中转站的限流为什么不能信 X-Forwarded-For</a> —— 本文配置里那行 <code>X-Real-IP</code> 为什么不能省</li>
</ul>
]]></content:encoded></item></channel></rss>