Automating Customer Ticket Sentiment and Category Tagging with Small LLMs (Llama 3.2 3B Prompt & Python Script)
Routing inbound customer support tickets by hand is expensive, slow, and error-prone. In high-volume SaaS environments, critical billing escalations and severe bug reports often languish in general queues behind basic password reset inquiries. While proprietary LLM APIs like GPT-4 can classify tickets with high accuracy, sending thousands of customer support messages to third-party cloud APIs each day accumulates steep monthly costs and exposes private user conversations.
With modern small language models like Llama 3.2 3B, you can run high-throughput ticket classification and sentiment analysis locally on your own hardware or inexpensive cloud VPS instances. By combining a strict few-shot system prompt with structured JSON output enforcement, a 3-billion-parameter model can match frontier LLM categorization accuracy at virtually zero marginal cost.
Why Small Local LLMs Excel at Ticket Triage
Frontier reasoning models (o1, Claude 3.5 Sonnet) are built for complex software engineering and deep multi-step logic. Categorizing a customer support message into predefined buckets and gauging sentiment is a pattern matching task where frontier models represent massive overkill.
| Architecture Dimension | Cloud API (GPT-4o / Claude 3.5) | Local Small LLM (Llama 3.2 3B via Ollama/vLLM) |
|---|---|---|
| Cost per 100k Tickets | $150 - $400 USD | $0 (Local) or $15/mo VPS fixed |
| Latency per Ticket | 800ms - 2,200ms (network overhead) | 45ms - 120ms (vLLM local batch) |
| Data Privacy & GDPR | Data leaves boundary to third party | 100% private on-premises execution |
| Rate Limits & Quotas | Strict tier quotas and HTTP 429s | Zero rate limits; limited only by compute |
The Production Few-Shot System Prompt
Small language models require unambiguous instructions and few-shot calibration to prevent output drift. This prompt specifies strict enum categories, a numeric sentiment scale, and strict JSON formatting:
You are an automated Customer Support Operations Engine.
Analyze the customer ticket and classify it into structured JSON.
Allowed Categories:
- BILLING: Invoices, credit cards, refund requests, subscription cancellations.
- TECHNICAL_BUG: System errors, broken buttons, 500 errors, crash reports.
- FEATURE_REQUEST: Asking for new integrations, UI tweaks, or capability expansions.
- ONBOARDING_HELP: Account setup, invite links, basic configuration walk-throughs.
- SPAM: Unsolicited sales, gibberish, automated marketing pitches.
Allowed Sentiments:
- URGENT_FRUSTRATED: Angry customer, threatening cancellation or legal action.
- NEUTRAL: Informational questions, clear problem statements without emotion.
- POSITIVE: Praise, compliments, grateful inquiries.
Allowed Priority Levels:
- P0: Total service outage, critical billing error, immediate churn risk.
- P1: Significant feature blocker affecting daily work.
- P2: Minor inconvenience, cosmetic issue, general inquiry.
Output Format:
Return ONLY valid JSON matching this schema:
{
"category": "CATEGORY_ENUM",
"sentiment": "SENTIMENT_ENUM",
"priority": "P0|P1|P2",
"escalate_to_human": true|false,
"summary": "One sentence summary under 15 words."
}
### Example 1:
Ticket: "Your checkout button is completely broken on Chrome! I tried 5 times and lost my customer! Fix this now or refund my entire annual plan!"
JSON:
{"category": "BILLING", "sentiment": "URGENT_FRUSTRATED", "priority": "P0", "escalate_to_human": true, "summary": "Checkout button broken on Chrome; customer demanding refund."}
### Example 2:
Ticket: "Hi, could you let me know if your API supports Webhook signatures with HMAC SHA256? Thanks!"
JSON:
{"category": "ONBOARDING_HELP", "sentiment": "NEUTRAL", "priority": "P2", "escalate_to_human": false, "summary": "Inquiring about API webhook HMAC SHA256 signature support."}
Step-by-Step Python Automation Script (triage.py)
We will use Ollama's local HTTP API with structured JSON output enforcement. First, make sure you have Ollama installed and pull the model:
# Install Ollama and pull Llama 3.2 3B
ollama pull llama3.2:3b
# Install Python requirements
pip install httpx pydantic
Create the Python script triage.py:
import json
import httpx
from pydantic import BaseModel, Field
from typing import Literal
# Define strict response validation schema using Pydantic
class TicketTriageResult(BaseModel):
category: Literal["BILLING", "TECHNICAL_BUG", "FEATURE_REQUEST", "ONBOARDING_HELP", "SPAM"]
sentiment: Literal["URGENT_FRUSTRATED", "NEUTRAL", "POSITIVE"]
priority: Literal["P0", "P1", "P2"]
escalate_to_human: bool
summary: str = Field(max_length=150)
SYSTEM_PROMPT = '''You are an automated Customer Support Operations Engine.
Analyze the customer ticket and classify it into structured JSON.
Allowed Categories: BILLING, TECHNICAL_BUG, FEATURE_REQUEST, ONBOARDING_HELP, SPAM.
Allowed Sentiments: URGENT_FRUSTRATED, NEUTRAL, POSITIVE.
Allowed Priority Levels: P0, P1, P2.
Return ONLY valid JSON matching the required schema.'''
def classify_ticket(ticket_text: str) -> TicketTriageResult:
url = "http://localhost:11434/api/chat"
payload = {
"model": "llama3.2:3b",
"messages": [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"Ticket: "{ticket_text}""}
],
"format": "json",
"stream": False,
"options": {
"temperature": 0.0,
"num_predict": 128
}
}
with httpx.Client(timeout=10.0) as client:
response = client.post(url, json=payload)
response.raise_for_status()
raw_output = response.json()["message"]["content"]
# Parse and validate with Pydantic
data = json.loads(raw_output)
return TicketTriageResult(**data)
if __name__ == "__main__":
sample_tickets = [
"I was double billed $99 on November 15th! Please reverse this charge immediately.",
"Would love to see an integration with Linear in your Q1 roadmap!",
"Can't log in, getting 500 internal server error on /auth/callback"
]
for ticket in sample_tickets:
result = classify_ticket(ticket)
print(f"
Ticket: {ticket}")
print(f"Classification: Category={result.category} | Priority={result.priority} | Escalate={result.escalate_to_human}")
print(f"Summary: {result.summary}")
Production Integration: Listening to Zendesk / Linear Webhooks
To run this classification engine autonomously as tickets arrive from your support portal, connect your Python triage script to an inbound webhook handler. Here is how to handle Zendesk or Freshdesk ticket creation webhooks using FastAPI:
from fastapi import FastAPI, BackgroundTasks, Request
app = FastAPI()
async def process_and_route_ticket(ticket_id: str, subject: str, description: str):
full_text = f"{subject}
{description}"
classification = classify_ticket(full_text)
# Route to specialized Slack channel or assign support agent
if classification.priority == "P0" or classification.escalate_to_human:
print(f"[ALERT] High-priority escalation for Ticket #{ticket_id}: {classification.summary}")
# Post to #urgent-support Slack channel via incoming webhook
else:
print(f"[AUTO-ROUTED] Ticket #{ticket_id} routed to queue {classification.category}")
@app.post("/webhooks/zendesk")
async def zendesk_webhook(request: Request, bg_tasks: BackgroundTasks):
data = await request.json()
bg_tasks.add_task(
process_and_route_ticket,
ticket_id=data.get("id"),
subject=data.get("subject"),
description=data.get("description")
)
return {"status": "enqueued"}
Production Hardening: High-Throughput Batch Processing
When running this in production against thousands of webhooks per hour, keep these performance optimizations in mind:
- Keep Temperature at 0.0: Deterministic classification requires zero creativity. Setting temperature to 0 eliminates hallucinatory tag variations.
- Cap Generation Length: A complete classification JSON string is roughly 60 tokens long. Setting
num_predict: 128prevents the model from rambling if unexpected input arrives. - vLLM for Multi-GPU Scaling: While Ollama is ideal for workstations and single-host setups, deploying
vLLMwith continuous batching achieves upwards of 1,200 ticket classifications per second on a single NVIDIA A10G GPU.
Frequently Asked Questions (FAQ)
Can Llama 3.2 3B handle multi-lingual customer tickets?
Yes. Llama 3.2 was pretrained on a massive multilingual corpus and comfortably handles English, Spanish, German, French, Portuguese, and Vietnamese. You can add a "detected_language": "ISO_CODE" field to the JSON schema to route non-English tickets directly to native-speaking support agents.
What happens if the model outputs malformed JSON?
Setting "format": "json" in Ollama activates constrained grammar decoding at the C++ llama.cpp layer, mathematically preventing the engine from outputting tokens that violate JSON syntax. Pydantic validation handles any schema validation errors.
Is 3B parameters large enough, or should I use 8B?
For classification and sentiment tagging, 3B parameters achieves >96% parity with 8B and 70B models when guided by few-shot examples. The 8B model is only required if you also ask the model to generate full natural language response drafts to customers.
How much RAM does Llama 3.2 3B require?
Quantized to 4-bit (Q4_K_M), Llama 3.2 3B consumes less than 2.2GB of RAM. It runs effortlessly on consumer laptops, Apple Silicon M-series chips, or an inexpensive $10/month cloud VPS.
Conclusion
Automating customer support ticket triage does not require expensive cloud API contracts or exposing private customer communications. By combining local lightweight models like Llama 3.2 3B with strict few-shot JSON prompts, engineering teams can deploy lightning-fast, zero-cost support routing that protects customer data and eliminates ticket backlogs.
Join the conversation