Policy

See Behavior policy for which action is modeled and how to model both medication columns.

class ehr2rl.BehaviorPolicy(n_action_bins=5, random_state=42)

Estimate the probability of each observed action given the state.

Only the first action column is modeled. Integer actions with at most n_action_bins distinct values are used as classes directly; other actions are split into at most n_action_bins quantile bins.

Parameters:

• n_action_bins (int) – Maximum number of action classes.

• random_state (int) – Seed for the classifier.

fit(dataset)

Fit the policy on every timestep of a dataset.

Trains a standardized multinomial logistic regression, or a constant classifier when only one action class is observed.

Parameters:

dataset (EHRDataset) – Non-empty dataset.

Returns:

This policy, for chaining.

Return type:

BehaviorPolicy

Raises:

ValueError – If the dataset is empty.

predict_proba(states)

Return the probability of each action class for each state.

Parameters:

states (ndarray) – Shape (N, D).

Returns:

Shape (N, n_classes). Columns follow model.classes_.

Return type:

ndarray

Raises:

ValueError – If the policy has not been fit.

propensity_scores(dataset)

Return the probability of the action actually taken at each timestep.

An action class not seen during fit gets the first class’s probability.

Parameters:

dataset (EHRDataset) – Dataset to score, with the same action encoding used for fitting.

Returns:

Shape (N,), where N is the total number of timesteps.

Return type:

ndarray

Raises:

ValueError – If the policy has not been fit or the dataset is empty.