ngramshrink - Man Page

Shrink n-gram model from input model file.

Usage: ngramshrink [--options] [in.fst [out.fst]]

PROGRAM FLAGS:

--backoff_label: type = int64_t, default = 0

Backoff label --check_consistency: type = bool, default = false Check model consistency --context_pattern: type = std::string, default = "" Pattern of contexts to prune --count_pattern: type = std::string, default = "" Pattern of counts to prune --list_file: type = std::string, default = "" File with list of n-grams to prune --method: type = std::string, default = "seymore" One of: "context_prune", "count_prune", "relative_entropy", "seymore", "list_prune" --min_order_to_prune: type = int32_t, default = 2 Minimum n-gram order to prune --norm_eps: type = double, default = 0.001 Normalization check epsilon --retry_downcase: type = bool, default = false If a pruned symbol is not found in the FST, automatically tries the lower-cased variant of this symbol. Only useful in list_prune mode. --shrink_opt: type = int32_t, default = 0 Optimization level: Range 0 (fastest) to 2 (most accurate) --target_number_of_ngrams: type = int64_t, default = -1 Maximum number of ngrams to leave in model after pruning. Value less than zero means no target number, just use theta. --theta: type = double, default = 0 Pruning threshold theta --total_unigram_count: type = double, default = -1 Total unigram count

LIBRARY FLAGS:

Flags from: flags.cc

--help: type = bool, default = false

show usage information --helpshort: type = bool, default = false show brief usage information --tmpdir: type = std::string, default = "/tmp" temporary directory --v: type = int32_t, default = 0 verbosity level

Flags from: fst.cc

--fst_align: type = bool, default = false

Write FST data aligned where appropriate --fst_default_cache_gc: type = bool, default = true Enable garbage collection of cache --fst_default_cache_gc_limit: type = int64_t, default = 1048576 Cache byte size that triggers garbage collection --fst_read_mode: type = std::string, default = "read" Default file reading mode for mappable files --fst_verify_properties: type = bool, default = false Verify FST properties queried by TestProperties --save_relabel_ipairs: type = std::string, default = "" Save input relabel pairs to file --save_relabel_opairs: type = std::string, default = "" Save output relabel pairs to file

Flags from: ngram-output.cc

--end_symbol: type = std::string, default = "</s>"

Class label for sentence end --start_symbol: type = std::string, default = "<s>" Class label for sentence start

Flags from: symbol-table.cc

--fst_compat_symbols: type = bool, default = true

Require symbol tables to match when appropriate --fst_field_separator: type = std::string, default = "         " Set of characters used as a separator between printed fields

Flags from: util.cc

--fst_error_fatal: type = bool, default = true

FST errors are fatal; o.w. return objects flagged as bad: e.g., FSTs: kError property set, FST weights: not a Member() --ngram_error_fatal: type = bool, default = true NGram errors are fatal if true; otherwise returns objects flagged as bad: e.g., NGramModel::Error() is true

Flags from: weight.cc

--fst_weight_parentheses: type = std::string, default = ""

Characters enclosing the first weight of a printed composite weight (e.g., pair weight, tuple weight and derived classes) to ensure proper I/O of nested composite weights; must have size 0 (none) or 2 (open and close parenthesis) --fst_weight_separator: type = std::string, default = "," Character separator between printed composite weights; must be a single character