Thinking markers leak into the reply
Check the instruct or chat template, tokenizer and reasoning mode before touching temperature. A marker such as <|think|> in the wrong format can appear as literal text.
English
Gemma 4 × SillyTavern settings
Fix the setting layer first
A bad Gemma 4 reply does not always need a new preset. Check the model ID, chat template, system role, context budget and samplers in order. This guide gives a testable baseline, not a magic recipe.
Start with the symptom
The output tells you which control to inspect
Separate a format error from a sampling choice. The fastest repair is usually the smallest change that explains the symptom.
The answer cuts off or is empty
Inspect the response token limit and the remaining context window. A long character card plus history can leave too little room for generation.
It loops or loses the character
Look for duplicated card text, an overfull context, or aggressive repetition controls. Test a clean prompt before changing several samplers together.
Model and memory
26B A4B and 31B are different trade-offs
Google describes 26B A4B as mixture-of-experts and 31B as dense. MoE activates fewer parameters per token, but its weights still need to be loaded. Quantization and context length change the practical memory cost.
| Model | Profile | Memory note | Fit |
|---|---|---|---|
| 26B A4B | Mixture-of-experts; 4B active per token | Google estimates about 14.4 GB for Q4_0 weights before context overhead | A useful local candidate when the backend supports its format |
| 31B | Dense model | Google estimates about 17.5 GB for Q4_0 weights before context overhead | Choose it only when memory and response speed are acceptable |
| Q4, Q5, Q6 | Quantized files, not new official model names | Lower precision can reduce memory, with a quality and speed trade-off | Follow the publisher model card and the backend format |
The figures are approximate weight-loading estimates. Add room for the runtime, KV cache and your selected context length.
Connection stack
Five layers sit between a card and a reply
Treat each layer as a separate checkpoint. If a symptom appears after one change, you know where to look.
Endpoint and model ID
Setting
Connector, server URL and exact checkpoint slug
Baseline / watch
Use the ID shown by your provider or backend
Wrong alias, stale model or unsupported quant format
Chat or instruct format
Setting
Chat Completions, Text Completion and template
Baseline / watch
Start with Chat Completions when the server supports it
Headers or thinking tags appear as plain text
System role
Setting
System message and its position
Baseline / watch
Keep it short, explicit and separate from the card
The backend ignores or duplicates the system message
Context and response
Setting
Context tokens and maximum response tokens
Baseline / watch
Reserve response space before loading long history
Cut-off replies, forgotten details or high memory use
Sampling
Setting
Temperature, Top P, Top K, Min P and repetition controls
Baseline / watch
Use a neutral starting point, then change one value
Loops, flat prose or unstable character voice
Sampler baseline
Use a baseline you can explain
Google and community guides often start near Temperature 1.0, Top P 0.95 and Top K 64. SillyTavern defines these controls separately. Treat the values as a first test, not a best preset.
First test
- Temperature 1.0
- Top P 0.95
- Top K 64
- Min P 0
- Repetition penalty 1
- DRY multiplier 0
SillyTavern documents different disable values for different controls. The backend can also expose a different range, so confirm the tooltip and server documentation.
1. Establish a clean run
Use one short prompt, the same card and the same context. Save the output so later changes have something to compare with.
2. Change temperature alone
If the reply is too flat, move temperature slightly up. If it is chaotic, move it slightly down. Keep Top P and Top K fixed for this test.
3. Tune the token pool
Change Top P or Top K, one at a time. Min P is disabled at 0 in SillyTavern and can be tested later at a low value when the backend supports it.
4. Add repetition control last
Check duplicated context first. Repetition penalty is disabled at 1 and DRY at multiplier 0. Strong values can punish ordinary words or make prose brittle.
Repair map
Match the fix to the evidence
Thinking is shown instead of parsed
Recheck the exact model template, tokenizer and reasoning mode. Try Chat Completions if available. Do not add a marker to a prompt until you know the backend expects it.
A long reply stops early
Raise response tokens only after checking context usage. Trim duplicated lore or card text first. A larger context window needs more memory for the KV cache.
Repetition starts after a few turns
Inspect repeated messages and the context boundary. Reset to repetition penalty 1 and DRY 0, then test one modest control rather than stacking penalties.
Different quant files feel different
Keep the backend, card and sampler fixed while comparing files. Record quant suffix, context length and generation speed. A Q4 file is not interchangeable evidence for a BF16 run.
A separate path
Skip local sampler work when the task is about a page
Tabbit is a Chromium-based AI browser for macOS and Windows. It does not run a local Gemma checkpoint or replace SillyTavern cards. It is useful when you need to read a live page, screenshot or file and ask beside it.
- 1
Install Tabbit
Download the desktop browser and open a new tab. The official site lists macOS 12+ and Windows 10+ support.
- 2
Use the current model picker
Pick a model that is actually listed in your installed Tabbit. The list can change. This page does not claim that Gemma 4 is available there.
- 3
Reference context with @
Keep a character wiki, prompt notes or documentation open. Use @ to reference a page, screenshot or local file, then ask for notes, a summary or a second draft.
Choose the workflow
Local SillyTavern or Tabbit?
The right route depends on the job. Keep precise model and card control local; use Tabbit when the page itself is the context.
| SillyTavern + local backend | Tabbit | |
|---|---|---|
| Checkpoint and quantization | You select and load it | Use a model shown in the picker |
| Template and sampler | Fine-grained controls | Managed by the selected model |
| Character cards and lorebooks | Core workflow | Keep the source page open |
| Page, screenshot and file context | Paste or configure an extension | @ the tab or file |
| Best first diagnostic | Model ID → template → context → sampler | Open, reference, ask |
Sampler baseline
SillyTavern documents different disable values for different controls. The backend can also expose a different range, so confirm the tooltip and server documentation.
FAQ
Gemma 4 settings, answered
What is a safe Gemma 4 SillyTavern baseline?+
Start with Temperature 1.0, Top P 0.95 and Top K 64, while leaving Min P at 0, repetition penalty at 1 and DRY multiplier at 0. These are starting values. Test one change at a time.
Should I use 26B A4B or 31B?+
They use different architectures. Google describes 26B A4B as MoE and 31B as dense. Check the model card, approximate weight memory, context budget and your backend before choosing.
Why do Gemma 4 thinking tokens appear as text?+
The model template, tokenizer, backend and reasoning mode may not agree. Confirm all four before adding markers such as <|think|>.
Do context and response tokens share memory?+
They share the request budget, and a larger total context needs more KV-cache memory. SillyTavern sends context tokens minus the response allowance.
Can Tabbit run my local Gemma 4 quant?+
This page makes no such claim. Install Tabbit and use only a model shown in its current picker. Tabbit is for page and file context, while SillyTavern remains the local card and sampler tool.
Make one change, then read the result
For local Gemma 4, keep the model card, template and context chain visible while you tune. For page-centered work, install Tabbit and choose a model from the current picker.
Available for macOS and Windows. Model availability can change.