Comparing Performance in R Using Microbenchmark

This post is a very simple display of how to use microbenchmark in R. Other sites might have longer and more detailed posts, but this post is primarily to 'spread the word' about this useful function, and show how to plot it. An alternative version of this post exists in Microsoft's Azure Notebooks, as Performance Testing Results with Microbenchmark

Load Libraries

Memoise as part of the code to test, microbenchmark to show usage, and ggplot2 to plot the result.

 library(memoise)  
 library(microbenchmark)  
 library(ggplot2)  

Create Functions

Generate several functions with varied performance times, a base function plus functions that leverage vectorization and memoisation.

 # base function  
 monte_carlo = function(N) {  
   hits = 0  
   for (i in seq_len(N)) {  
     u1 = runif(1)  
     u2 = runif(1)  
     if (u1 ^ 2 > u2)  
       hits = hits + 1  
     }  
   return(hits / N)  
 }  
   
 # memoise test function   
 monte_carlo_memo <- memoise(monte_carlo)  
   
 # vectorize function  
 monte_carlo_vec <- function(N) mean(runif(N) ^ 2 > runif(N))  
   
 # memoise vectorized function  
 monte_carlo_vec_memo <- memoise(monte_carlo_vec)  

Run Tests

Using microbenchmark, loop 100 times through each function.

 n <- 999999  
 comparison <- microbenchmark(times = 100  
                , memoised = system.time(monte_carlo_memo(n))  
                , vectorised = system.time(monte_carlo_vec(n))  
                , both = system.time(monte_carlo_vec_memo(n))) 
 comparison

Display Results:

Display the results of the microbenchmark run.

 Unit: milliseconds  
       expr      min        lq     mean   median       uq        max  neval  
   memoised    45.88     51.55   109.45    57.93    63.04    5027.95    100  
 vectorised   176.18    226.01   235.94   236.47   248.78     332.85    100  
       both    46.23     50.67    59.60    54.74    62.86     224.50    100  

Plot

Using autoplot, display the results.

 autoplot(comparison)