metrax.RougeL

class metrax.RougeL(total_precision: Array, total_recall: Array, total_f1: Array, num_examples: Array)

Bases: RougeBase

Computes macro-averaged ROUGE-L recall, precision, and F1-score.

ROUGE-L measures the longest common subsequence (LCS) between a prediction and a reference. This metric calculates ROUGE-L precision, recall, and F1-score for each individual prediction compared against its single corresponding reference. These per-instance scores are then averaged.

How ROUGE-L scores are calculated for each individual prediction-reference pair:

For a single prediction P and reference R:

\[LCS(P, R) = \text{length of the Longest Common Subsequence}\]

\[\text{Recall}_{\text{LCS}} = \frac{LCS(P, R)}{|R|}\]

\[\text{Precision}_{\text{LCS}} = \frac{LCS(P, R)}{|P|}\]

\[\text{F1}_{\text{LCS}} = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}\]

Final Macro-Averaged Metrics:

\[\text{MacroAvgPrecision} = \frac{\text{total_precision}}{\text{num_examples}}\]

\[\text{MacroAvgRecall} = \frac{\text{total_recall}}{\text{num_examples}}\]

\[\text{MacroAvgF1} = \frac{\text{total_f1}}{\text{num_examples}}\]

total_precision

Accumulated sum of LCS precision scores from each instance.

Type:

jax.Array

total_recall

Accumulated sum of LCS recall scores from each instance.

Type:

jax.Array

total_f1

Accumulated sum of LCS F1 scores from each instance.

Type:

jax.Array

num_examples

The number of instances (prediction-reference pairs) processed.

Type:

jax.Array

__init__(total_precision: Array, total_recall: Array, total_f1: Array, num_examples: Array) → None

Methods

__init__(total_precision, total_recall, ...)
compute()	Computes macro-averaged recall, precision, and F1-score.
compute_value()	Wraps compute() and returns a values.Value.
empty(**kwargs)	Creates an empty Rouge metric.
from_fun(fun)	Calls cls.from_model_output with the return value from fun.
from_model_output(predictions, references, ...)	Computes sums of per-instance ROUGE scores for a batch.
from_output(name)	Calls cls.from_model_output with model output named name.
merge(other)	Merges this Rouge metric with another.
reduce()	Reduces the metric along it first axis by calling _reduce_merge().
replace(**updates)	Returns a new object replacing the specified fields with new values.

Attributes

total_precision
total_recall
total_f1
num_examples

classmethod empty(**kwargs) → RougeL

Creates an empty Rouge metric. Implemented by subclasses.

__init__(total_precision: Array, total_recall: Array, total_f1: Array, num_examples: Array) → None

replace(**updates)

Returns a new object replacing the specified fields with new values.