Balanced accuracy is the mean of two proportions estimated on disjoint groups, so sensitivity and specificity are independent. Two ways of using that are offered, and the difference matters at the boundary.
Usage
ba_ci(tp, fn, fp, tn, conf_level = 0.95, method = c("wilson", "exact", "wald"))Arguments
- tp, fn, fp, tn
Counts of true positives, false negatives, false positives and true negatives. Vectorised and recycled together.
- conf_level
Confidence level.
- method
"wilson"(default) or"exact"to square-and-add the corresponding component intervals, or"wald"for the analytic normal-approximation variance.
Details
method = "wilson" (default) and method = "exact" combine the component
intervals by Newcombe's square-and-add: with component intervals
\((l_1, u_1)\) and \((l_2, u_2)\) around \(p_1\) and \(p_2\),
$$\mathrm{lo} = \mathrm{BA} - \tfrac{1}{2}\sqrt{(p_1-l_1)^2 + (p_2-l_2)^2}$$ $$\mathrm{hi} = \mathrm{BA} + \tfrac{1}{2}\sqrt{(u_1-p_1)^2 + (u_2-p_2)^2}$$
method = "wald" uses the analytic normal-approximation form
\(\mathrm{Var}(\mathrm{BA}) = \tfrac{1}{4}(\mathrm{Var}(\mathrm{sens}) +
\mathrm{Var}(\mathrm{spec}))\).
The Wald form is degenerate at a boundary. \(\mathrm{Var}(p) =
p(1-p)/n\) is zero when a proportion is 0 or 1, so a component at the boundary
contributes no uncertainty at all. A rule with perfect sensitivity on six
positive cases and perfect specificity on 177 negatives returns
1.00 (1.00, 1.00) — an interval of zero width from 183 observations. The
square-and-add methods do not have this failure, because Wilson and
Clopper-Pearson intervals are not degenerate at the boundary: the same table
gives 1.00 (0.77, 1.00) under "exact". ba_ci() warns if "wald" is
used on a boundary proportion.
Whichever method is used, the interval is marginal. It is not a valid basis for comparing two symptom combinations, because such a comparison is paired on the same individuals and the two estimates are strongly positively correlated. Any table printing it alongside several combinations should say so.
References
Newcombe RG (1998). Interval estimation for the difference between independent proportions: comparison of eleven methods. Statistics in Medicine, 17(8), 873-890.
Examples
ba_ci(tp = 80, fn = 20, fp = 10, tn = 90)
#> est lo hi
#> [1,] 0.85 0.7920758 0.8901385
# At the boundary the analytic form collapses and the others do not.
ba_ci(tp = 6, fn = 0, fp = 0, tn = 177, method = "exact")
#> est lo hi
#> [1,] 1 0.7701395 1
suppressWarnings(ba_ci(tp = 6, fn = 0, fp = 0, tn = 177, method = "wald"))
#> est lo hi
#> [1,] 1 1 1