https://blog.ohtly.com/tags/gemma/ Recent content in Gemma on OHTLY Blog Hugo zh-cn Fri, 19 Jun 2026 20:14:37 +0800 https://blog.ohtly.com/posts/2026-06-19-%E9%85%8D%E7%BD%AEollama%E6%9C%AC%E5%9C%B0%E6%A8%A1%E5%9E%8B%E7%BF%BB%E8%AF%91%E6%96%87%E8%A8%80%E6%96%87/ Fri, 19 Jun 2026 20:14:37 +0800 https://blog.ohtly.com/posts/2026-06-19-%E9%85%8D%E7%BD%AEollama%E6%9C%AC%E5%9C%B0%E6%A8%A1%E5%9E%8B%E7%BF%BB%E8%AF%91%E6%96%87%E8%A8%80%E6%96%87/ <p>用《杜骗新书》&ldquo;假馬脫緞&quot;一文测试本地 LLM 的文言文翻译能力，记录上下文配置和多模型对比过程。</p> <h2 id="问题回答被截断"> 问题：回答被截断 <a class="heading-link" href="#%e9%97%ae%e9%a2%98%e5%9b%9e%e7%ad%94%e8%a2%ab%e6%88%aa%e6%96%ad"> <i class="fa-solid fa-link" aria-hidden="true" title="链接到标题"></i> <span class="sr-only">链接到标题</span> </a> </h2> <p>Ollama 默认 <code>num_ctx=2048</code>，客户端设到 4096 仍然不够。日志：</p> <pre tabindex="0"><code>slot release: stop processing: n_tokens = 4095, truncated = 1 </code></pre><p>prompt 占 ~1000 tokens，只剩 3000 给生成，翻译到一半就被截断。</p> <h2 id="解法服务端定制模型"> 解法：服务端定制模型 <a class="heading-link" href="#%e8%a7%a3%e6%b3%95%e6%9c%8d%e5%8a%a1%e7%ab%af%e5%ae%9a%e5%88%b6%e6%a8%a1%e5%9e%8b"> <i class="fa-solid fa-link" aria-hidden="true" title="链接到标题"></i> <span class="sr-only">链接到标题</span> </a> </h2> <p>客户端不支持按模型调参时，在 Ollama 服务端创建定制模型即可：</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>echo <span style="color:#e6db74">&#39;FROM gemma4:12b </span></span></span><span style="display:flex;"><span><span style="color:#e6db74">PARAMETER num_ctx 16384&#39;</span> &gt; gemma4-16k.Modelfile </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span>ollama create gemma4:12b-16k -f gemma4-16k.Modelfile </span></span></code></pre></div><h2 id="16gb-显存各模型上下文上限"> 16GB 显存各模型上下文上限 <a class="heading-link" href="#16gb-%e6%98%be%e5%ad%98%e5%90%84%e6%a8%a1%e5%9e%8b%e4%b8%8a%e4%b8%8b%e6%96%87%e4%b8%8a%e9%99%90"> <i class="fa-solid fa-link" aria-hidden="true" title="链接到标题"></i> <span class="sr-only">链接到标题</span> </a> </h2> <table> <thead> <tr> <th>模型</th> <th>模型大小</th> <th>每 token KV cache</th> <th>建议上下文</th> </tr> </thead> <tbody> <tr> <td>qwen3.5:9b</td> <td>6.6 GB</td> <td>~32 KB</td> <td>128K</td> </tr> <tr> <td>gpt-oss-abliterated:20b</td> <td>13 GB</td> <td>~34 KB</td> <td>64K</td> </tr> <tr> <td>gemma4:12b</td> <td>7.4 GB</td> <td>~448 KB</td> <td>16K</td> </tr> </tbody> </table> <p>Gemma4 的 KV 投影维度 512 per head，48 层中 40 层为 8 KV heads，导致 KV cache 开销远大于前两者。但滑动窗口层（8 层 1 KV head）用环形缓冲区节省了空间。</p>