metrax.WER

class metrax.WER(total: Array, count: Array)

Bases: Average

Computes Word Error Rate (WER) for speech recognition or text generation tasks.

Word Error Rate measures the edit distance between reference texts and predictions, normalized by the length of the reference texts. It is calculated as:

\[WER = \frac{S + D + I}{N}\]

where:

• S is the number of substitutions

• D is the number of deletions

• I is the number of insertions

• N is the number of words in the reference

A lower WER indicates better performance, with 0 being perfect.

This implementation accepts both pre-tokenized inputs (lists of tokens) and untokenized strings. When strings are provided, they are tokenized by splitting on whitespace.

__init__(total: Array, count: Array) → None

Methods

__init__(total, count)
compute()	Computes final metrics from intermediate values.
compute_value()	Wraps compute() and returns a values.Value.
empty()	Returns an empty instance (i.e. .merge(Metric.empty()) is a no-op).
from_fun(fun)	Calls cls.from_model_output with the return value from fun.
from_model_output(predictions, references)	Updates the metric.
from_output(name)	Calls cls.from_model_output with model output named name.
merge(other)	Returns Metric that is the accumulation of self and other.
reduce()	Reduces the metric along it first axis by calling _reduce_merge().
replace(**updates)	Returns a new object replacing the specified fields with new values.

Attributes

total
count

classmethod from_model_output(predictions: list[str], references: list[str]) → WER

Updates the metric.

Parameters:

• prediction – Either a string or a list of tokens in the predicted sequence.

• reference – Either a string or a list of tokens in the reference sequence.

Returns:

New WER metric instance.

Raises:

ValueError – If inputs are not properly formatted or are empty.

__init__(total: Array, count: Array) → None

replace(**updates)

Returns a new object replacing the specified fields with new values.