Benchmark: Comparing the Monte Carlo Method with Nonparametric Bootstrapping

Ivan Jacob Agaloos Pesigan

2024-04-14

We compare the Monte Carlo (MC) method with nonparametric bootstrapping (NB) for standardized regression coefficients. In this example, we use the data set and the model used in betaMC: Example Using the BetaMC Function.

library(betaMC)
library(boot)
library(microbenchmark)

The BetaMC() function is used to generate MC confidence intervals. The BetaNB() function is used to generate NB confidence intervals.

BetaNB <- function(formula, data, B) {
  statistic <- function(formula, data, indices) {
    return(
      coef(lm(formula = formula, data = as.data.frame(scale(data[indices, ]))))[-1]
    )
  }
  return(boot.ci(boot(data = data, statistic = statistic, formula = formula, R = B)))
}

Data and Model

df <- betaMC::nas1982

Benchmark

Arguments

Variables	Values	Notes
R	5000	Number of Monte Carlo replications.
B	5000	Number of bootstrap samples.

benchmark <- microbenchmark(
  MC = {
    formula <- "QUALITY ~ NARTIC + PCTGRT + PCTSUPP"
    object <- lm(formula = formula, data = df)
    mc <- MC(object = object, R = R, type = "mvn")
    BetaMC(object = mc)
  },
  NB = {
    formula <- "QUALITY ~ NARTIC + PCTGRT + PCTSUPP"
    object <- lm(formula = formula, data = df)
    BetaNB(formula = formula, data = df, B = B)
  },
  times = 10
)

Summary of Benchmark Results

summary(benchmark, unit = "ms")
#>   expr       min        lq      mean    median        uq       max neval
#> 1   MC  262.4154  328.6991  335.3215  343.1367  356.0945  369.8192    10
#> 2   NB 3915.8488 4155.9073 4557.1998 4598.3571 4873.7456 5454.9808    10

Summary of Benchmark Results Relative to the Faster Method

summary(benchmark, unit = "relative")
#>   expr      min      lq     mean   median       uq     max neval
#> 1   MC  1.00000  1.0000  1.00000  1.00000  1.00000  1.0000    10
#> 2   NB 14.92233 12.6435 13.59054 13.40095 13.68666 14.7504    10

Plot