README.md

---
library_name: transformers
pipeline_tag: text-classification
license: mit
tags:
  - prompting
  - zero-shot
  - few-shot
  - football
  - sentiment
  - adaptive-retrieval
model_name: Football Sentiment Prompting (0/1/5-shot)
language:
  - en
datasets:
  - james-kramer/football_news
inference: false
---

# Method Card — Football Sentiment Prompting (0/1/5-shot)

## TL;DR
We compare zero-shot, adaptive one-shot, and adaptive 5-shot prompting for binary sentiment on football news. 
Same train/val/test as fine-tuning; we report metrics/CMs and discuss quality/latency/cost.

## Data
- Dataset: `james-kramer/football_news` (Hugging Face)
- Task: Binary sentiment (0=negative, 1=positive)
- Splits: Stratified 80/10/10
- Cleaning: strip text; drop empty/NA

## Models / APIs
- **LLM used:** gpt-4o-mini (OpenAI API, September 2025 snapshot)  
- **Similarity backend:** sklearn TF-IDF + cosine similarity 

## Prompting Strategy
- Zero-shot: instruction + schema (return 0 or 1 only).
- Adaptive one-shot: retrieve most similar train example and include it as exemplar.
- Adaptive 5-shot: retrieve top-5 similar exemplars.

## Prompt Templates
**Zero-shot**
You are a concise sentiment classifier.
Decide if the following football-related sentence is positive or negative.
Only answer with a single word: "positive" or "negative".

Sentence: "text", 
Answer:

**Adaptive One-shot**
You are a concise sentiment classifier for football news.
Decide if each sentence is positive or negative. Only answer with one word.

Example: [],
Sentence: "ex_text",
Label: "ex_label",

Now classify the target sentence.
Sentence: "text", 
Answer:

**Adaptive K-shot (e.g., K=5)**
You are a concise sentiment classifier for football news.
Decide if the sentence is positive or negative. Only answer with one word.
examples: [], 
Sentence: "text", 
Answer:


## Evaluation Protocol
- Metrics: accuracy, precision, recall, F1; confusion matrix
- Latency: avg wall-clock per example
- Seed: 42
- Reproducibility: prompts/selection/eval code in this repo

## Results (Val/Test)
- Val:
  - Zero-shot: acc 0.8, f1 0.75, cm [[5, 0], [2, 3]], ~0.416s/ex
  - One-shot: acc 0.5, f1 0.2857142857, cm [[4, 1], [4, 1]], ~0.304s/ex
  - 5-shot:   acc 0.8, f1 0.75, cm [[5, 0], [2, 3]], ~0.451s/ex
- Test:
  - Zero-shot: acc 0.7, f1 0.7272727273, cm [[3, 2], [1, 4]], ~0.282s/ex
  - One-shot: acc 0.7, f1 0.7272727273, cm [[3, 2], [1, 4]], ~0.354s/ex
  - 5-shot:   acc 0.7, f1 0.5714285714, cm [[5, 0], [3, 2]], ~0.449s/ex

## Tradeoffs
- Quality: zero-shot ≈ 5-shot ≥ one-shot on this dataset.
- Latency: increases with K (see Results section; ~0.28s/ex for zero-shot → ~0.45s/ex for 5-shot).  
- Cost: scales roughly linearly with prompt length (token count). For this dataset (~20 examples), 5-shot prompts were ~3× the token usage of zero-shot.

## Limits & Risks
- No leakage: retrieve exemplars from **train** only.
- Bias: sports phrasing may sway sentiment; small data → instability.

## Reproducibility
- Code: `prompts/`, `selection.py`, `evaluate_prompting.py`
- Seed: 42
- Python ≥ 3.10

## Usage Disclosure
This card and pipeline were organized with GenAI assistance; experiments and results were implemented and verified by the author.