autorag.evaluation package

Subpackages

• autorag.evaluation.metric package
  • Submodules
  • autorag.evaluation.metric.deepeval_prompt module
    • FaithfulnessTemplate
      • FaithfulnessTemplate.generate_claims()
      • FaithfulnessTemplate.generate_truths()
      • FaithfulnessTemplate.generate_verdicts()
  • autorag.evaluation.metric.generation module
    • async_g_eval()
    • bert_score()
    • bleu()
    • deepeval_faithfulness()
    • g_eval()
    • huggingface_evaluate()
    • make_generator_instance()
    • meteor()
    • rouge()
    • sem_score()
  • autorag.evaluation.metric.retrieval module
    • retrieval_f1()
    • retrieval_map()
    • retrieval_mrr()
    • retrieval_ndcg()
    • retrieval_precision()
    • retrieval_recall()
  • autorag.evaluation.metric.retrieval_contents module
    • retrieval_token_f1()
    • retrieval_token_precision()
    • retrieval_token_recall()
    • single_token_f1()
  • autorag.evaluation.metric.util module
    • autorag_metric()
    • autorag_metric_loop()
    • calculate_cosine_similarity()
    • calculate_inner_product()
    • calculate_l2_distance()
  • Module contents

Submodules

autorag.evaluation.generation module

autorag.evaluation.generation.evaluate_generation(metric_inputs: List[MetricInput], metrics: List[str] | List[Dict])

autorag.evaluation.retrieval module

autorag.evaluation.retrieval.evaluate_retrieval(metric_inputs: List[MetricInput], metrics: List[str] | List[Dict])

autorag.evaluation.retrieval_contents module

autorag.evaluation.retrieval_contents.evaluate_retrieval_contents(metric_inputs: List[MetricInput], metrics: List[str])

autorag.evaluation.util module

autorag.evaluation.util.cast_embedding_model(key, value)

autorag.evaluation.util.cast_metrics(metrics: List[str] | List[Dict]) → Tuple[List[str], List[Dict[str, Any]]]

Turn metrics to list of metric names and parameter list.

Parameters:

metrics – List of string or dictionary.

Returns:

The list of metric names and dictionary list of metric parameters.

Module contents