Choosing strategies

EnsembleDetector() selects the default English screening profile. It combines three English language heuristics with mojibake, keyboard adjacency, and control-character checks. Meaningful Hindi may be flagged; this is expected behavior.

The english_extended profile adds localized anomalies, repetition, and pattern matching. Measure its false positives before using it for automatic rejection. Use corruption for encoding/control artifacts without English plausibility scoring. spoofing is a Unicode script/confusable heuristic, not a full phishing or Unicode security implementation. Profile membership is listed in Detection Strategies.

What the preprint tells us about strategy choice

The preprint in Benchmark and preprint runs two profiles at their defaults on two English datasets: Survey Junk, which holds only junk survey answers, and Tweets vs. Random Strings. It supports three choices.

  • Survey and form free text: use english_fusion. It captured 89.5% of the survey junk and flagged 6.8% of tweets. Neither baseline matched it on both datasets at once. Add a minimum-length rule if one- and two-character answers count as junk.

  • Social-media text: do not use the default profile as shipped. The default english profile flagged 33.2% of tweets. Its mojibake and control_characters members target encoding damage, and about a quarter of the tweets carry mojibake that the dataset labels clean. Keep them when encoding damage counts as junk for you. Removing them is not enough for tweets, though: its markov_chain member alone flags 17.8% of tweets, and without the two encoding checks the profile still flags 18.2%.

  • With a labelled sample, learn a weight per strategy. The preprint’s tuning study learned a weight for each of the 32 strategies and a threshold under weighted voting from a 20 percent labelled sample (about 2,700 rows), in about 17 seconds of Optuna search. On the held-out 80 percent it flagged 4.9% of tweets against 6.9% for english_fusion, at about the same survey-junk capture but lower random-string recall (97.6% against 99.1%), averaged over five samples. It gave the encoding-damage checks little or no weight. The weights fit these datasets, so learn your own; Benchmark and preprint has the full results.

from pygarble import EnsembleDetector

# Learned on the preprint's primary tuning sample (seed 20261011).
weights = {
    "local_anomaly": 0.05,
    "entropy_based": 1.0,
    "vowel_ratio": 0.05,
    "keyboard_pattern": 0.35,
    "markov_chain": 0.8,
    "ngram_frequency": 0.65,
    "word_lookup": 1.0,
    "symbol_ratio": 0.85,
    "repetition": 0.3,
    "hex_string": 0.85,
    "pronounceability": 0.65,
    "unicode_script": 0.45,
    "bigram_probability": 0.55,
    "letter_position": 0.25,
    "consonant_sequence": 0.5,
    "vowel_pattern": 0.65,
    "letter_frequency": 0.75,
    "rare_trigram": 0.7,
    "function_word_density": 0.45,
    "affix_detection": 0.15,
    "zipf_conformity": 0.25,
    "word_collocation": 0.05,
    "log_likelihood_ratio": 0.9,
    "word_anomaly": 1.0,
    "keyboard_adjacency": 0.6,
    "cross_parsing": 0.6,
    "primed_compression": 0.95,
    "ngram_rank": 0.05,
    "permutation_test": 0.25,
}
tuned = EnsembleDetector(
    strategies=list(weights),
    weights=list(weights.values()),
    voting="weighted",
    threshold=0.16,
)
assert tuned.predict("asdfghjkl qwpoeiru")
assert not tuned.predict("Thanks for the quick reply, see you on Monday!")

When wrongly flagging real text costs much more than missing junk, the preprint found a character Markov baseline with a far lower false-positive rate. Otherwise, measure a threshold on labelled rows of your own data with Calibrating the threshold. The datasets are English and short; results on long documents and other languages are not measured.

Lexical and statistical strategies

  • word_lookup folds diacritics and checks Latin-letter words of at least two letters against the embedded dictionary. Unknown title-cased words contribute half weight. The default score is the weighted unknown fraction; a score of at least 0.5 flags the input. No eligible tokens yields zero, which is not evidence of linguistic understanding. Word lookup is not a member of the english, english_extended or legacy profiles; it is a member of the opt-in english_fusion profile described below.

  • markov_chain measures English character-transition likelihood over novel tokens. Dictionary-supported and recognized structured tokens do not contribute novelty evidence. Unlikely transitions increase the score.

  • log_likelihood_ratio compares English bigram likelihood with a uniform reference. Its sigmoid maps that statistic to a heuristic score, not a calibrated probability.

  • word_anomaly scores suspicious eligible tokens individually and reports their fraction, limiting dilution by surrounding ordinary prose.

  • entropy_based combines character entropy and bigram improbability.

Scores are strategy-specific evidence. Named English ensembles use any voting across applicable strategies, except english_fusion, which uses fisher; an inapplicable strategy does not vote. When all strategies are inapplicable, analyze() reports insufficient_evidence and an unflagged decision. That differs from a positive assertion that the text is meaningful. See API Reference for other voting policies and score semantics, and Calibrating the threshold for threshold selection on your own labelled data.

English-reference window strategies

Four strategies compare text with frequent English instead of applying hand-written rules. They read only the lowercase ASCII words, so numbers, URLs, versions and allowlisted words are left out. cross_parsing is a member of the opt-in english_fusion profile; the other three are not members of any named profile.

  • cross_parsing counts how many pieces are needed to spell the text from a reference built from common English words. English reuses long pieces; invented or mashed text breaks into many short ones.

  • primed_compression compresses the text with the same English words as a preset dictionary. English compresses well against it; invented text does not. Compressed sizes come from the platform’s zlib, so scores can differ slightly between zlib builds (for example zlib-ng).

  • ngram_rank ranks the text’s one- to three-letter sequences by frequency and measures how far those ranks are from English.

  • permutation_test asks how much more English-like the letter order is than shuffles of the same letters. It detects text whose letter order carries no English structure, such as keyboard mash or random letters. Pronounceable invented words already have English-like letter order, so it often misses them at any length, not only in short text; it is the weakest of the four on such text.

Each statistic is measured against synthetic English of the same length, and a score of 0.5 sits beyond the 99th percentile of that English. They are English-reference methods, so meaningful text in other languages written in Latin letters can be flagged. In spot checks, some ordinary French, German and Spanish sentences scored above 0.5 on cross_parsing and primed_compression while others did not, and romanised Hindi and Indonesian crossed 0.5 more often. Text in other scripts has no ASCII letters and is not scored (it reports insufficient_evidence). They need at least 8 letters (min_length) and become more reliable as text gets longer; text beyond 127 characters is split into windows and the median window decides, so one odd passage in a long document does not flag it.

Fisher voting and the english_fusion profile

voting="fisher" treats each applicable member’s score as a test statistic. A table shipped with the package records what that strategy scores on synthetic English (word salads drawn by word frequency, with target lengths from 8 to 480 characters; a text stops before the word that would pass its target and always has at least one word, so some are shorter than 8 characters), and the member’s p-value is the smallest tail probability, from 0.5 down to 0.001, whose threshold the score strictly exceeds. Fisher’s method combines the p-values, and fisher_alpha (default 0.001) places the combined p-value that scores exactly 0.5. Fisher voting is opt-in and works with any profile or strategy list.

The english_fusion profile uses it over three views of the same question: unknown words (word_lookup), character transitions (log_likelihood_ratio) and reuse of English phrases (cross_parsing). A member’s p-value never goes below 0.001, so when several members apply, one member cannot flag a text on its own at the default fisher_alpha: one member at 0.001 combines with two quiet members (p-value 1) to about 0.032, and with one quiet member to about 0.0079. At least two members must exceed their null thresholds, for example two at 0.001 with the third quiet (about 0.00011), or one at 0.001 with one at 0.01 (about 0.00079). A single member flags on its own only when it is the only applicable member.

This is a heuristic, not a test with a guaranteed error rate. The members are correlated, which Fisher’s method does not account for, and synthetic word salads are not real English, so the real false-positive rate at a given fisher_alpha can be higher or lower. All three members are English-reference methods. The profile can flag meaningful text in other languages written in Latin letters: in spot checks some French, German and Spanish sentences were flagged and others were not, and an Indonesian sentence was flagged. It gives no signal on other scripts. word_lookup applies to any non-empty text and scores 0 when there are no Latin-letter words, while the other two members do not apply, so Russian or Chinese text reads as clean; that is not evidence that the text is meaningful. Measure the profile on your own data before relying on it.

from pygarble.gibberish import EnsembleDetector

fusion = EnsembleDetector(profile="english_fusion")
assert fusion.predict("Please send the signed contract by Friday.") is False
assert fusion.predict("asdkfj qwpoeiru zxmcnv lkjasdf poiuqwer") is True

strict = EnsembleDetector(profile="english_fusion", fisher_alpha=0.0001)

Control characters: new in 0.11.0

Strategy.CONTROL_CHARACTERS examines raw Unicode characters before English normalization. It detects unexpected control characters, U+FFFD replacement characters, lone surrogates, and combining-mark runs exceeding a configurable limit. Ordinary tabs, line breaks, accents, and emoji joiners are not themselves flagged. This strategy belongs to english, english_extended, corruption, and llm_output.

from pygarble.gibberish import GarbleDetector, Strategy

detector = GarbleDetector(
    Strategy.CONTROL_CHARACTERS,
    strategy_kwargs={"max_combining_run": 8},
)
assert detector.predict("hello\x00world") is True
assert detector.predict("hello\nworld") is False
assert detector.predict("hello\ufffdworld") is True

max_combining_run defaults to 8 and must be a positive integer. A detected artifact scores 0.9; otherwise the score is 0.0. A detector threshold above 0.9 therefore suppresses these detections. These are fixed heuristic scores.

Localized anomalies: new in 0.11.0

Strategy.LOCAL_ANOMALY finds severe unfamiliar tokens or clusters of suspicious tokens in otherwise readable English. It uses the bundled English dictionary, shared bigram likelihood, and bounded token windows. It belongs to english_extended and llm_output and can also be selected independently.

from pygarble.gibberish import GarbleDetector, Strategy

detector = GarbleDetector(
    Strategy.LOCAL_ANOMALY,
    strategy_kwargs={
        "min_word_length": 8,
        "word_log_prob_threshold": -5.5,
        "window_words": 4,
    },
)
text = "Please review qxzjkwpvm before delivery."
result = detector.analyze(text)
assert result.garbled is True
assert any(text[s.start:s.end] == "qxzjkwpvm" for s in result.spans)

The example shows the defaults. min_word_length is the minimum length for an individual severe-token span; windows can use suspicious tokens of at least four characters. word_log_prob_threshold must be finite and negative; a more negative value requires a lower likelihood. window_words must be an integer from 1 to 32. A window requires at least two suspicious tokens, so choosing 1 disables window detections while retaining individual-token checks.

Detected anomalies score 0.8. Dictionary words, allowlisted terms, and structured tokens excluded by shared preprocessing do not become suspicious candidates. This is not a grammaticality or semantic-coherence check, and rare English words can still produce false positives.

Keyboard paths and repetition

Keyboard adjacency supports QWERTY, AZERTY, and QWERTZ with digit-row neighbors. It checks straight rows and substantially covered adjacent-key paths. It does not detect every possible keyboard walk. Repetition detects character, word, and bounded phrase repetition; intentional repetition can also be flagged.

from pygarble.gibberish import GarbleDetector, Strategy

keyboard = GarbleDetector(
    Strategy.KEYBOARD_ADJACENCY, keyboard_layout="azerty"
)
assert keyboard.predict("azerty") is True

repetition = GarbleDetector(Strategy.REPETITION)
assert repetition.predict("hello " * 10) is True

LLM output: new in 0.11.0

The llm_output profile is a cheap deterministic pre-check for degenerate model output, not a hallucination detector. It combines repetition, control characters, mojibake and local anomaly, so it catches repetition loops, encoding damage from bad decoding, stray control characters and dense token salad. It deliberately leaves out the Markov and word-anomaly members, so code, identifiers, product names and technical prose stay quiet. Use it as a guard that retries or rejects a response before it reaches a user.

from pygarble.gibberish import EnsembleDetector

guard = EnsembleDetector(profile="llm_output")
assert guard.predict("and so on and so on and so on and so on") is True
assert guard.predict("The answer is Café au lait.") is True
assert guard.predict('def parse(row): return row.split(",")') is False

response = "Restart the service, then confirm the new log file appears."
if guard.predict(response):
    raise RuntimeError("degenerate response; retry the request")

Tune on your own data

All 32 strategies remain individually available. Check both valid English inputs and expected corruption, including domain terms, identifiers, and short strings. Scores are not calibrated probabilities, and no strategy guarantees zero false positives. See API Reference for voting and abstention, and Upgrading for changes from the previous default. The earlier conditional-trigram experiment is retained in Git history; no additional trigram model is shipped by 0.11.0.