Compare plateaus and stability across rule forms

Runs the plateau analysis, and optionally the bootstrap stability analysis, across several exhaustive searches at once and stacks the results into long tables ready for reporting. Use it to put the hierarchical and non-hierarchical rules, or different endorsement thresholds, side by side.

Usage

compare_rule_forms(
  fits,
  data = NULL,
  delta = 1,
  n_boot = 0,
  seed = 123,
  show_progress = TRUE
)

Arguments

fits

A named list of score_all_combinations results. The names label the rule forms in the output, e.g. list("4/6 Hierarchical" = fit_a, "3/6 Non-hierarchical" = fit_b).

data

The data frame the searches were run on. Required only when n_boot > 0.

delta

Numeric. Plateau width in percentage points of balanced accuracy (default 1). A vector is allowed, e.g. delta = c(1, 5); the bootstrap, which is far more expensive, uses the first value only.

n_boot

Integer. Bootstrap replicates per rule form. The default 0 skips the bootstrap entirely, so the plateau comparison stays cheap.

seed

Integer. Random seed shared by every rule form's bootstrap.

show_progress

Logical. If TRUE (default), report progress.

Value

An object of class ptsdiag_rule_forms: a list of long data frames, each carrying a rule_form column.

• plateau: the summary of compute_plateau for every rule form and delta.

• selection: symptom_selection for every rule form and delta, one row per item.

• stability, per_set: the summary and per_set tables of bootstrap_stability, present only when n_boot > 0.

The stability weighted columns of selection are filled only for the first delta, since that is the width the bootstrap ran at; rows for any further widths carry NA there rather than weights describing a different plateau.

Details

Each rule form must be scored separately, because their candidate spaces differ: a six-item hierarchical search evaluates 13,685 combinations against the non-hierarchical search's 38,760. Every proportion in the output is divided by the denominator belonging to its own rule form, which is exactly the comparison a single pooled table would get wrong.

See also

compute_plateau, bootstrap_stability, symptom_selection.

Examples

# \donttest{
ptsd_data <- rename_ptsd_columns(simulated_ptsd[1:120, ],
                                 id_col = c("patient_id", "age", "sex"))
fits <- list(
  "3/4 Non-hierarchical" = score_all_combinations(
    ptsd_data, n_symptoms = 4, n_required = 3, show_progress = FALSE
  ),
  "2/4 Non-hierarchical" = score_all_combinations(
    ptsd_data, n_symptoms = 4, n_required = 2, show_progress = FALSE
  )
)

comparison <- compare_rule_forms(fits, delta = c(1, 5))
comparison$plateau
#>              rule_form delta   ba_best ba_threshold plateau_size n_candidates
#> 1 3/4 Non-hierarchical     1 1.0000000    0.9909910            8         4845
#> 2 3/4 Non-hierarchical     5 1.0000000    0.9684685           39         4845
#> 3 2/4 Non-hierarchical     1 0.8888889    0.8888889            1         4845
#> 4 2/4 Non-hierarchical     5 0.8888889    0.8888889            1         4845
#>   plateau_proportion
#> 1       0.0016511868
#> 2       0.0080495356
#> 3       0.0002063983
#> 4       0.0002063983
# }