Instructions to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF",
	filename="Qwen3.5-27B-GCTH_ST_V112-IQ3_XS-16gb_vram_optimized.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS
# Run inference directly in the terminal:
llama-cli -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS
# Run inference directly in the terminal:
llama-cli -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS
# Run inference directly in the terminal:
./llama-cli -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Use Docker

docker model run hf.co/Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

LM Studio
Jan
Ollama
How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with Ollama:
```
ollama run hf.co/Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS
```

Unsloth Studio

How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF to start chatting

How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Run Hermes

hermes

Docker Model Runner
How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with Docker Model Runner:
```
docker model run hf.co/Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS
```

Lemonade

How to use Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Gammaception/Qwen3.5-27B-Thireus-16gb-optimized-GGUF:IQ3_XS

Run and chat with the model

lemonade run user.Qwen3.5-27B-Thireus-16gb-optimized-GGUF-IQ3_XS

List all available models

lemonade list

Optimized llama.cpp GGUF quants of Qwen 3.5 27b made for 16gb VRAM cards.

Made using GGUF-Tool-suite! by Thireus, tuned by me :) (Gammaception)

Best 16 gb config for GC IQ3_M: (headless 64k ctx kv-cache@(q8_0,q8_0), -ub 256, q8_0 mmproj). Following official sampling parameters recommended, along with reasoning budget + message

EDIT 24/03/26: Updated GGUF to fix coding quality, new charts

EDIT 03/04/26: New variants with better quality ratios, update charts

	unsloth ud-iq3_xxs	bartowski iq3_xs	GC-Thireus iq3_xs	mungert iq3_xs	bartowski iq3_m	GC-Thireus iq3_m	unsloth ud-q3_k_xl	bartowski iq4_xs
Size (GiB)	10.716	11.357	11.800	12.078	12.240	12.465	13.447	14.130
PPL mean	7.349445	7.275691	7.003592	7.146062	7.166074	6.886104	7.095870	6.932149
KLD mean	0.080486	0.066744	0.044320	0.053966	0.057891	0.032533	0.034783	0.019950
Same top p	88.546	89.837	91.560	90.625	90.782	92.926	92.656	94.610