datarobot-model-explainability

Name: datarobot-model-explainability
Author: datarobot-oss

22stars19forks

How to install datarobot-model-explainability?

datarobot-oss/datarobot-agent-skills/datarobot-model-explainability

$npx -y skills add datarobot-oss/datarobot-agent-skills --skill datarobot-model-explainability

Installs into the current project.

Summary

Tools and guidance for model explainability, prediction explanations, feature impact analysis, SHAP values, SHAP distributions, anomaly assessment, and model diagnostics. Use when analyzing model explanations, feature impact, SHAP values, SHAP distributions, anomaly assessment, o

Files · 1

View on GitHub

SKILL.md

1	# DataRobot Model Explainability Skill
2
3	This skill covers SHAP insights, XEMP prediction explanations, anomaly explanations, and model diagnostics.
4
5	> SDK version: Use `datarobot>=3.6.0` for the full API set in this skill (`ShapDistributions`
6	> was added in 3.6; `ShapMatrix`, `ShapImpact`, and `ShapPreview` are available in
7	> `datarobot>=3.4.0`). Use `from datarobot.insights import ShapMatrix, ...` with
8	> `entity_id=model_id` — not legacy `datarobot.models.ShapMatrix` (`project_id` / `dataset_id`).
9	> `ShapMatrix`, `ShapImpact`, `ShapPreview`, and `ShapDistributions` are the canonical SHAP API.
10	> The older `dr.PredictionExplanations` (XEMP-based) remains available but is the secondary path.
11
12	---
13
14	## Quick Start
15
16	\| Goal \| API to use \| Prerequisites \|
17	\|------\|-----------\|---------------\|
18	\| SHAP values for all features, all rows \| `ShapMatrix.create(entity_id=model_id)` \| None - universal SHAP \|
19	\| Per-row top-feature explanations \| `ShapPreview.create(entity_id=model_id)` \| None \|
20	\| Aggregated feature importance via SHAP \| `ShapImpact.create(entity_id=model_id)` \| None \|
21	\| SHAP value distributions across features \| `ShapDistributions.create(entity_id=model_id)` \| None \|
22	\| SHAP for a filtered segment \| `dr.DataSlice.create(...)` + `ShapMatrix.create(..., data_slice_id=...)` \| Data slice definition \|
23	\| XEMP-based prediction explanations \| `dr.PredictionExplanations.create(...)` \| Feature Impact; PE initialization; dataset uploaded \|
24	\| Anomaly explanations (time series) \| `AnomalyAssessmentRecord.compute(project_id, model_id, ...)` \| Anomaly model \|
25	\| ROC / lift / confusion (insights) \| `RocCurve.create(...)` / `LiftChart.create(...)` / `ConfusionMatrix.create(...)` \| Validation data \|
26	\| ROC / lift / confusion (Model helpers) \| `model.get_roc_curve()` / `model.get_lift_chart()` / `model.get_confusion_chart()` \| Validation data \|
27
28	Universal SHAP is the preferred path - no dataset pre-upload or Feature Impact step required.
29
30	## When to use this skill
31
32	Use this skill when you need to explain leaderboard model behavior, compute SHAP insights, use
33	XEMP prediction explanations, analyze anomaly explanations, or retrieve model diagnostics.
34
35	## Key capabilities
36
37	### 1. SHAP insights
38
39	- Compute `ShapMatrix`, `ShapPreview`, `ShapImpact`, and `ShapDistributions`
40	- Filter insights with `dr.DataSlice`
41
42	### 2. XEMP and anomaly explanations
43
44	- Use XEMP `dr.PredictionExplanations` when specifically required
45	- Retrieve time series anomaly assessment records and explanations
46
47	### 3. Diagnostics
48
49	- Retrieve ROC, lift, and confusion insights
50	- Use Model helpers for ROC, lift, confusion, and feature effects
51
52	## Setup
53
54	```python
55	import os
56	import datarobot as dr
57	from datarobot.insights import ShapMatrix, ShapImpact, ShapPreview, ShapDistributions
58
59	dr.Client(
60	token=os.environ["DATAROBOT_API_TOKEN"],
61	endpoint=os.environ.get("DATAROBOT_ENDPOINT", "https://app.datarobot.com/api/v2"),
62	)
63	```
64
65	---
66
67	## Core API: `datarobot.insights`
68
69	```python
70	import pandas as pd
71	from datarobot.insights import ShapMatrix, ShapImpact, ShapPreview, ShapDistributions
72
73	model_id = "YOUR_MODEL_ID"
74
75	matrix = ShapMatrix.create(entity_id=model_id)
76	df = pd.DataFrame(matrix.matrix, columns=matrix.columns)
77
78	impact = ShapImpact.create(entity_id=model_id)
79	preview = ShapPreview.create(entity_id=model_id)
80	distributions = ShapDistributions.create(entity_id=model_id)
81	```
82
83	Use `ShapMatrix` for full row-by-feature SHAP values, `ShapPreview` for compact top-driver rows,
84	`ShapImpact` for aggregated SHAP importance, and `ShapDistributions` for per-feature SHAP
85	distributions. Use `source="externalTestSet"` plus `external_dataset_id` for external datasets.
86	See `references/shap_api_reference.md` for parameters, exports, and limitations.
87
88	---
89
90	## Secondary path: XEMP Prediction Explanations
91
92	Use `dr.PredictionExplanations` when XEMP explanations are specifically required (e.g., certain
93	regulatory contexts, or when SHAP is unavailable for the model type).
94
95	Prerequisites (all required before calling `.create()`):
96	1. Feature Impact must be computed: `model.request_feature_impact()` and wait
97	2. Prediction explanations initialized: `dr.PredictionExplanationsInitialization.create(...)`
98	3. Scoring dataset uploaded to the AI Catalog
99
100	```python
101	import datarobot as dr
102
103	model = dr.Model.get(project=project_id, model_id=model_id)
104	model.request_feature_impact().wait_for_completion()
105	dr.PredictionExplanationsInitialization.create(project_id=project_id, model_id=model_id)
106
107	dataset = dr.Dataset.upload("./data/scoring_data.csv")
108	pe_job = dr.PredictionExplanations.create(
109	project_id=project_id,
110	model_id=mode

People who liked this also viewed

6 picks

Type

datarobot-model-explainability

22stars19forks

View on GitHub

How to install datarobot-model-explainability?

datarobot-oss/datarobot-agent-skills/datarobot-model-explainability

$npx -y skills add datarobot-oss/datarobot-agent-skills --skill datarobot-model-explainability

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# DataRobot Model Explainability Skill
2
3	This skill covers SHAP insights, XEMP prediction explanations, anomaly explanations, and model diagnostics.
4
5	> SDK version: Use `datarobot>=3.6.0` for the full API set in this skill (`ShapDistributions`
6	> was added in 3.6; `ShapMatrix`, `ShapImpact`, and `ShapPreview` are available in
7	> `datarobot>=3.4.0`). Use `from datarobot.insights import ShapMatrix, ...` with
8	> `entity_id=model_id` — not legacy `datarobot.models.ShapMatrix` (`project_id` / `dataset_id`).
9	> `ShapMatrix`, `ShapImpact`, `ShapPreview`, and `ShapDistributions` are the canonical SHAP API.
10	> The older `dr.PredictionExplanations` (XEMP-based) remains available but is the secondary path.
11
12	---
13
14	## Quick Start
15
16	\| Goal \| API to use \| Prerequisites \|
17	\|------\|-----------\|---------------\|
18	\| SHAP values for all features, all rows \| `ShapMatrix.create(entity_id=model_id)` \| None - universal SHAP \|
19	\| Per-row top-feature explanations \| `ShapPreview.create(entity_id=model_id)` \| None \|
20	\| Aggregated feature importance via SHAP \| `ShapImpact.create(entity_id=model_id)` \| None \|
21	\| SHAP value distributions across features \| `ShapDistributions.create(entity_id=model_id)` \| None \|
22	\| SHAP for a filtered segment \| `dr.DataSlice.create(...)` + `ShapMatrix.create(..., data_slice_id=...)` \| Data slice definition \|
23	\| XEMP-based prediction explanations \| `dr.PredictionExplanations.create(...)` \| Feature Impact; PE initialization; dataset uploaded \|
24	\| Anomaly explanations (time series) \| `AnomalyAssessmentRecord.compute(project_id, model_id, ...)` \| Anomaly model \|
25	\| ROC / lift / confusion (insights) \| `RocCurve.create(...)` / `LiftChart.create(...)` / `ConfusionMatrix.create(...)` \| Validation data \|
26	\| ROC / lift / confusion (Model helpers) \| `model.get_roc_curve()` / `model.get_lift_chart()` / `model.get_confusion_chart()` \| Validation data \|
27
28	Universal SHAP is the preferred path - no dataset pre-upload or Feature Impact step required.
29
30	## When to use this skill
31
32	Use this skill when you need to explain leaderboard model behavior, compute SHAP insights, use
33	XEMP prediction explanations, analyze anomaly explanations, or retrieve model diagnostics.
34
35	## Key capabilities
36
37	### 1. SHAP insights
38
39	- Compute `ShapMatrix`, `ShapPreview`, `ShapImpact`, and `ShapDistributions`
40	- Filter insights with `dr.DataSlice`
41
42	### 2. XEMP and anomaly explanations
43
44	- Use XEMP `dr.PredictionExplanations` when specifically required
45	- Retrieve time series anomaly assessment records and explanations
46
47	### 3. Diagnostics
48
49	- Retrieve ROC, lift, and confusion insights
50	- Use Model helpers for ROC, lift, confusion, and feature effects
51
52	## Setup
53
54	```python
55	import os
56	import datarobot as dr
57	from datarobot.insights import ShapMatrix, ShapImpact, ShapPreview, ShapDistributions
58
59	dr.Client(
60	token=os.environ["DATAROBOT_API_TOKEN"],
61	endpoint=os.environ.get("DATAROBOT_ENDPOINT", "https://app.datarobot.com/api/v2"),
62	)
63	```
64
65	---
66
67	## Core API: `datarobot.insights`
68
69	```python
70	import pandas as pd
71	from datarobot.insights import ShapMatrix, ShapImpact, ShapPreview, ShapDistributions
72
73	model_id = "YOUR_MODEL_ID"
74
75	matrix = ShapMatrix.create(entity_id=model_id)
76	df = pd.DataFrame(matrix.matrix, columns=matrix.columns)
77
78	impact = ShapImpact.create(entity_id=model_id)
79	preview = ShapPreview.create(entity_id=model_id)
80	distributions = ShapDistributions.create(entity_id=model_id)
81	```
82
83	Use `ShapMatrix` for full row-by-feature SHAP values, `ShapPreview` for compact top-driver rows,
84	`ShapImpact` for aggregated SHAP importance, and `ShapDistributions` for per-feature SHAP
85	distributions. Use `source="externalTestSet"` plus `external_dataset_id` for external datasets.
86	See `references/shap_api_reference.md` for parameters, exports, and limitations.
87
88	---
89
90	## Secondary path: XEMP Prediction Explanations
91
92	Use `dr.PredictionExplanations` when XEMP explanations are specifically required (e.g., certain
93	regulatory contexts, or when SHAP is unavailable for the model type).
94
95	Prerequisites (all required before calling `.create()`):
96	1. Feature Impact must be computed: `model.request_feature_impact()` and wait
97	2. Prediction explanations initialized: `dr.PredictionExplanationsInitialization.create(...)`
98	3. Scoring dataset uploaded to the AI Catalog
99
100	```python
101	import datarobot as dr
102
103	model = dr.Model.get(project=project_id, model_id=model_id)
104	model.request_feature_impact().wait_for_completion()
105	dr.PredictionExplanationsInitialization.create(project_id=project_id, model_id=model_id)
106
107	dataset = dr.Dataset.upload("./data/scoring_data.csv")
108	pe_job = dr.PredictionExplanations.create(
109	project_id=project_id,
110	model_id=mode

People who liked this also viewed

6 picks

Type