nopp23/codedlang-coded-language-detector

CodedLang coded-language detector (hfl/chinese-roberta-wwm-ext fine-tune)

Binary classifier: does this Chinese Google-Maps-style review contain coded/evasive language (homophone substitution, orthographic tricks, cross-lingual phonetic spelling, cipher, etc.)?

Fine-tuned on 6208 reviews from the CodedLang dataset (Wan, Li & Huang, ACL 2026, RuyuanWan/CodedLang, CC BY 4.0).

Held-out test metrics (this checkpoint, 763 reviews never seen during training)

• precision: 0.92
• recall: 0.853
• f1: 0.885
• decision threshold: 0.05

This is a single train/dev/test split, not the headline number. The thesis's main result (pooled out-of-fold F1 0.894 across a 5-fold cross-validation, exceeding the paper's best few-shot LLM baseline of 0.78) comes from a different study with 5 separately-trained models, documented in codedlang2026-finetune.md. This repo's checkpoint is a single deployable model trained the same way, released for reproducibility and demoing, not for citing as the research result.

Limitations

• Community-labeled dataset (not expert-annotated); community-level knowledge, not conversation-specific codes.
• English/Chinese Google Maps reviews, not spoken dialogue or Thai text.
• Hyperparameters mostly copied from the 5-fold study, except batch size (4, not 16) and max sequence length (256, not 512) — reduced to fit this machine's GPU memory. 92% of reviews are under 256 characters, so truncation loss should be small, but this was not independently re-tuned for the new split.
• Single seed; variance across seeds not measured.

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("nopp23/codedlang-coded-language-detector")
model = AutoModelForSequenceClassification.from_pretrained("nopp23/codedlang-coded-language-detector")
inputs = tok("your review text here", return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    prob = torch.softmax(model(**inputs).logits, -1)[0, 1].item()
print("coded-language probability:", prob)