Skip to content

Latest commit

 

History

History
146 lines (121 loc) · 6.07 KB

File metadata and controls

146 lines (121 loc) · 6.07 KB

Machine Learning Classical

Domain Task Common Metrics
ML Classification Accuracy, Precision, Recall, F1, ROC-AUC, PR-AUC, Log Loss, Balanced Accuracy, MCC, Cohen's Kappa
ML Multi-Class Classification Top-1 Accuracy, Top-K Accuracy, Macro F1, Micro F1, Weighted F1
ML Regression MAE, MSE, RMSE, RMSLE, R², Adjusted R², MAPE, SMAPE, Median AE
ML Clustering Silhouette Score, Davies-Bouldin, Calinski-Harabasz, Adjusted Rand Index, NMI
ML Time Series MAPE, SMAPE, MAE, RMSE, MASE, WAPE, Forecast Bias
ML Anomaly Detection Precision, Recall, FPR, TPR, F1, AUROC, Average Precision
ML Recommendation CTR, MAP, MRR, NDCG, Hit Rate, Recall@K, Precision@K, Coverage
ML Ranking NDCG, MRR, Precision@K, Recall@K, MAP
ML Survival Analysis Concordance Index (C-index), Brier Score

Deep Learning

Domain Task Common Metrics
DL Image Classification Top-1 Accuracy, Top-5 Accuracy, Precision, Recall, F1
DL Object Detection mAP, IoU, Precision, Recall, AP50, AP75
DL Semantic Segmentation Dice Score, IoU, Pixel Accuracy, Mean IoU
DL Instance Segmentation mAP, Mask IoU
DL OCR Character Error Rate (CER), Word Error Rate (WER)
DL Speech Recognition WER, CER
DL NLP Classification Accuracy, F1, BLEU, ROUGE
DL Embeddings Cosine Similarity, Retrieval Accuracy, Recall@K, MRR
DL Autoencoders Reconstruction Loss, MSE

Reinforcement Learning

Domain Task Common Metrics
RL General RL Average Reward, Cumulative Reward, Episode Return
RL Training Policy Loss, Value Loss, Entropy
RL Performance Success Rate, Win Rate, Task Completion Rate
RL Efficiency Sample Efficiency, Steps to Convergence
RL Stability Reward Variance, Training Stability
RL Exploration Entropy, Exploration Ratio
RL Environment Episode Length, Steps per Episode
RL Multi-Agent RL Cooperation Score, Competition Score
RL Robotics Collision Rate, Energy Consumption, Task Completion Time
RL Game AI ELO Rating, Win Percentage
RL Safety RL Constraint Violations, Safety Score

GenAI

Domain Task Common Metrics
GenAI LLM Relevance, Faithfulness, Hallucination Rate, Helpfulness, Toxicity, Coherence, Completeness
GenAI RAG Context Precision, Context Recall, Faithfulness, Answer Relevance, Retrieval Accuracy
GenAI Chatbots User Satisfaction, CSAT, Resolution Rate, Conversation Length
GenAI Text Generation BLEU, ROUGE, METEOR, BERTScore
GenAI Summarization ROUGE, BERTScore, Factual Consistency
GenAI Translation BLEU, COMET, TER
GenAI Diffusion Models FID, IS (Inception Score), CLIP Score, Human Preference
GenAI Image Generation FID, CLIP Score, Aesthetic Score

Serving

-Latency -P50 Latency -P95 Latency -P99 Latency -Throughput (RPS) -Concurrent Requests -Queue Length -Request Count -Success Rate -Error Rate -Timeout Rate -Availability -Uptime

Infra

-CPU Utilization -Memory Usage -GPU Utilization -GPU Memory Usage -Disk Usage -Disk I/O -Network I/O -Container/Pod Health -Instance Count -Autoscaling Events -Restart Count -Temperature (CPU/GPU)

Data Quality

-Missing Value Percentage -Null Count -Schema Violations -Feature Drift -Data Drift -Concept Drift -Outlier Percentage -Data Freshness -Distribution Shift

Cost

-Cost per Day -Compute Cost -Storage Cost -GPU Hours Consumed -Token Cost (for LLMs)

Reliability

-MTTR (Mean Time To Recovery) -MTBF (Mean Time Between Failures) -SLA Compliance -SLO Compliance -Incident Count -Alert Count

Deployment

-Model Version -Deployment Frequency -Rollback Count -Environment (Dev/Staging/Prod) -Deployment Age

Usage

-Active Users -Requests per User -Peak Traffic -Geographic Distribution -Traffic Distribution

Observability

-Log Count -Trace Count -Span Duration -Event Count

Security

-Authentication Failures -Rate Limit Violations -Unauthorized Access Attempts -API Key Usage