Detection Strategies

Generated by python scripts/update_strategy_docs.py.

There are 28 strategies. Language checks target English; meaningful Hindi and other non-English text may be flagged. Encoding and spoofing specialists report their own evidence independently.

Scores are heuristics, not calibrated probabilities. See the README for profile selection, applicability, and evaluation limitations.

Profiles

english: MARKOV_CHAIN, LOG_LIKELIHOOD_RATIO, WORD_ANOMALY, MOJIBAKE, KEYBOARD_ADJACENCY, CONTROL_CHARACTERS
english_extended: MARKOV_CHAIN, LOG_LIKELIHOOD_RATIO, WORD_ANOMALY, MOJIBAKE, KEYBOARD_ADJACENCY, CONTROL_CHARACTERS, PATTERN_MATCHING, LOCAL_ANOMALY, REPETITION
legacy: MARKOV_CHAIN, LOG_LIKELIHOOD_RATIO, WORD_ANOMALY
corruption: MOJIBAKE, CONTROL_CHARACTERS
spoofing: UNICODE_SCRIPT

Strategy settings

CONTROL_CHARACTERS

Detect raw control/decoding artifacts without English normalization.

Settings: max_combining_run, max_string_length.

LOCAL_ANOMALY

Bounded token windows find severe corruption embedded in English prose.

Settings: max_string_length, min_word_length, window_words, word_log_prob_threshold.

PATTERN_MATCHING

English text heuristic.

Settings: max_string_length, override_defaults, patterns.

ENTROPY_BASED

English text heuristic.

Settings: max_string_length.

VOWEL_RATIO

English text heuristic.

Settings: consonant_cluster_len, max_string_length, max_vowel_ratio, min_vowel_ratio.

KEYBOARD_PATTERN

English text heuristic.

Settings: max_string_length.

MARKOV_CHAIN

Detect garbled text using a character-level Markov chain.

Settings: max_string_length, min_length, threshold_per_char.

NGRAM_FREQUENCY

Detect garbled text using character trigram frequency analysis.

Settings: common_ratio_threshold, max_string_length, min_length.

WORD_LOOKUP

Detect garbled text by checking words against a dictionary.

Settings: max_string_length, min_word_length, unknown_threshold.

SYMBOL_RATIO

Detect garbled text based on symbol/number density.

Settings: allow_spaces, count_digits, max_string_length, min_length, symbol_threshold.

REPETITION

Detect garbled text based on character/pattern repetition.

Settings: diversity_threshold, max_char_repeat, max_pattern_repeat, max_string_length.

HEX_STRING

Detect garbled text that appears to be hexadecimal data.

Settings: hex_ratio_threshold, max_string_length, min_hex_length.

MOJIBAKE

Detect garbled text caused by encoding corruption (mojibake).

Settings: check_replacement_char, max_string_length, pattern_threshold, ratio_threshold.

PRONOUNCEABILITY

Detect garbled text based on pronounceability.

Settings: forbidden_cluster_threshold, max_string_length, min_word_length, vowel_min_ratio.

UNICODE_SCRIPT

Detect garbled/suspicious text based on Unicode script mixing.

Settings: check_homoglyphs, homoglyph_threshold, max_scripts, max_string_length.

BIGRAM_PROBABILITY

Detects garbled text by identifying impossible character bigrams.

Settings: max_string_length, min_length, threshold.

LETTER_POSITION

Detects garbled text by identifying letters in impossible positions.

Settings: max_string_length, min_word_length, threshold.

CONSONANT_SEQUENCE

Detects garbled text by identifying impossibly long consonant runs.

Settings: max_consonants, max_string_length, min_length.

VOWEL_PATTERN

Detects garbled text by analyzing vowel patterns.

Settings: max_string_length, max_vowel_run, min_length.

LETTER_FREQUENCY

Detects garbled text by comparing letter frequencies to English norms.

Settings: deviation_threshold, max_string_length, min_length.

RARE_TRIGRAM

Detects garbled text by identifying impossible trigram sequences.

Settings: max_string_length, min_length, threshold.

FUNCTION_WORD_DENSITY

Detect garbled text by checking for common English function words.

Settings: max_string_length, min_ratio, min_word_length, min_words.

AFFIX_DETECTION

Detect garbled text by checking for recognizable English affixes.

Settings: max_string_length, min_affix_ratio, min_analyzable_words, min_stem_length, min_word_length.

ZIPF_CONFORMITY

Detect garbled text by checking word frequency distribution.

Settings: hapax_threshold, max_string_length, min_words, ttr_threshold.

WORD_COLLOCATION

Detect garbled text by checking for common English word bigrams.

Settings: max_string_length, min_words, zero_collocation_min_words.

LOG_LIKELIHOOD_RATIO

English-versus-uniform bigram likelihood ratio (a heuristic score).

Settings: llr_midpoint, llr_scale, max_string_length, min_bigrams.

WORD_ANOMALY

Per-word anomaly scoring: flag text by its fraction of garbled tokens.

Settings: anomaly_weight, max_string_length, min_word_length, word_log_prob_threshold.

KEYBOARD_ADJACENCY

Detect keyboard mashing via physical key-adjacency walks.

Settings: chain_threshold, keyboard_layout, max_string_length, min_word_length, row_run_threshold.