05 Simulation Experiments

Running a groups of simulations: the `experiment` function

Once a simulation is properly initialized and runs correctly with the spades function, it is likely time to do things like run replicates. There are two functions, experiment and experiment2 to do this. These both give opportunities to create simulation experiments, such as through varying parameters and modules. experiment is for simpler “fully factorial” cases, and experiment2 can take any simList class objects for the simulation experiment. With experiment, for example, you can pass alternative values to parameters in modules and the experiment will build a fully factorial experiment, with replication and run them. This function (as with POM and splitRaster) is parallel-aware and can either proceed with a cluster object (cl argument) or a cluster running in the background using the raster::beginCluster.

tmpdir <- file.path(tempdir(), "experiment")

library("SpaDES.core")

## Loading required package: quickPlot

## Loading required package: reproducible

## 
## Attaching package: 'SpaDES.core'

## The following objects are masked from 'package:stats':
## 
##     end, start

## The following object is masked from 'package:utils':
## 
##     citation

library("SpaDES.experiment")

## 
## Attaching package: 'SpaDES.experiment'

## The following objects are masked from 'package:SpaDES.core':
## 
##     POM, experiment, experiment2, simInitAndExperiment

# Copy Example 5 here:
mySim <- simInit(
  times = list(start = 0.0, end = 2.0, timeunit = "year"),
  params = list(.globals = list(stackName = "landscape", burnStats = "nPixelsBurned")),
  modules = list("randomLandscapes", "fireSpread", "caribouMovement"),
  paths = list(modulePath = system.file("sampleModules", package = "SpaDES.core"),
               outputPath = tmpdir),
  # Save final state of landscape and caribou
  outputs = data.frame(objectName = c("landscape", "caribou"), stringsAsFactors = FALSE)
)

## Setting:
##   options(
##     spades.outputPath = '/tmp/RtmpsoZfad/experiment'
##     spades.modulePath = '/github/workspace/pkglib/SpaDES.core/sampleModules'
##   )

## Paths set to:
##   options(
##     rasterTmpDir = '/tmp/RtmpsoZfad/SpaDES/scratch/raster'
##     reproducible.cachePath = '/tmp/RtmpsoZfad/reproducible/cache'
##     spades.inputPath = '/tmp/RtmpsoZfad/SpaDES/inputs'
##     spades.outputPath = '/tmp/RtmpsoZfad/experiment'
##     spades.modulePath = '/github/workspace/pkglib/SpaDES.core/sampleModules'
##     spades.scratchPath = '/tmp/RtmpsoZfad/SpaDES/scratch'
##   )
##   terra::terraOptions(tempdir = '/tmp/RtmpsoZfad/SpaDES/scratch/terra'

## Using GITHUB_PAT to access files on GitHub

## For better security, user should use the newer way to store git credentials.
## Using a GITHUB_PAT environment variable will continue to work, but see: https://usethis.r-lib.org/articles/git-credentials.html

## No packages to install/update

## Loading required package: RColorBrewer

## Loading required package: grid

## Loading required package: sf

## Linking to GEOS 3.12.1, GDAL 3.8.4, PROJ 9.4.0; sf_use_s2() is TRUE

## Loading required package: terra

## terra 1.8.5

## 
## Attaching package: 'terra'

## The following object is masked from 'package:grid':
## 
##     depth

## The following object is masked from 'package:SpaDES.core':
## 
##     time<-

## The following object is masked from 'package:SpaDES.tools':
## 
##     wrap

## Dec25 06:17:13 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.

## Dec25 06:17:13 simInit randomLandscapes: module code: landscape is declared in metadata outputObjects, but is not assigned in the module

## Dec25 06:17:13 simInit randomLandscapes: module code: Init: local variable 'inMemory' assigned but may not be used

## Dec25 06:17:13 simInit randomLandscapes: outputObjects: Par$stackName is assigned to sim inside Init, but is not declared in metadata outputObjects

## Dec25 06:17:13 simInit randomLandscapes: inputObjects: Par$stackName is used from sim inside Init, but is not declared in metadata inputObjects

## Dec25 06:17:13 simInit randomLandscapes: inputObjects: Par$stackName is used from sim inside doEvent.randomLandscapes, but is not declared in metadata inputObjects

## Dec25 06:17:13 simInit fireSpread: module code: landscape is declared in metadata outputObjects, but is not assigned in the module

## Dec25 06:17:13 simInit fireSpread: module code: landscape, nPixelsBurned are declared in metadata inputObjects, but no default(s) are provided in .inputObjects

## Dec25 06:17:13 simInit fireSpread: module code: landscape is declared in metadata inputObjects, but is not used in the module

## Dec25 06:17:13 simInit fireSpread: outputObjects: Par$stackName is assigned to sim inside Burn, but is not declared in metadata outputObjects

## Dec25 06:17:13 simInit fireSpread: outputObjects: Par$stackName is assigned to sim inside Init, but is not declared in metadata outputObjects

## Dec25 06:17:13 simInit fireSpread: outputObjects: Par$stackName is assigned to sim inside doEvent.fireSpread, but is not declared in metadata outputObjects

## Dec25 06:17:13 simInit fireSpread: inputObjects: Par$stackName is used from sim inside Burn, but is not declared in metadata inputObjects

## Dec25 06:17:13 simInit fireSpread: inputObjects: Par$stackName is used from sim inside Init, but is not declared in metadata inputObjects

## Dec25 06:17:13 simInit fireSpread: inputObjects: Par$stackName is used from sim inside Stats, but is not declared in metadata inputObjects

## Dec25 06:17:13 simInit fireSpread: inputObjects: Par$stackName, DEM, Fires are used from sim inside doEvent.fireSpread, but are not declared in metadata inputObjects

## Dec25 06:17:13 simInit caribouMovement: module code: landscape, caribou are declared in metadata inputObjects, but no default(s) are provided in .inputObjects

## Dec25 06:17:13 simInit caribouMovement: module code: landscape is declared in metadata inputObjects, but is not used in the module

## Dec25 06:17:13 simInit caribouMovement: inputObjects: Par$stackName is used from sim inside Init, but is not declared in metadata inputObjects

## Dec25 06:17:13 simInit caribouMovement: inputObjects: Par$stackName is used from sim inside Move, but is not declared in metadata inputObjects

## Dec25 06:17:13 simInit The following .globals were used:

## Dec25 06:17:13 simInit Key: <global, module>
## Dec25 06:17:13 simInit               module    global
## Dec25 06:17:13 simInit               <char>    <char>
## Dec25 06:17:13 simInit  1:       fireSpread burnStats
## Dec25 06:17:13 simInit  2:  caribouMovement stackName
## Dec25 06:17:13 simInit  3:       fireSpread stackName
## Dec25 06:17:13 simInit  4: randomLandscapes stackName

## Elapsed time for simInit: 3.758849 secs

sims <- experiment(mySim, replicates = 2, .plotInitialTime = NA) # no plotting

## No packages to install/update

## Dec25 06:17:13 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.

## Dec25 06:17:13 chckpn:init total elpsd: 3.8 secs | 0 checkpoint init 0

## Dec25 06:17:13 save  :init total elpsd: 3.8 secs | 0 save init 0

## Dec25 06:17:13 prgrss:init total elpsd: 3.8 secs | 0 progress init 0

## Dec25 06:17:13 load  :init total elpsd: 3.8 secs | 0 load init 0

## Dec25 06:17:13 rndmLn:init total elpsd: 3.8 secs | 0 randomLandscapes init 1

## Dec25 06:17:14 rndmLn:init New objects created:

## Dec25 06:17:14 rndmLn:init <char>
## Dec25 06:17:14 rndmLn:init  1:  landscape

## Dec25 06:17:14 frSprd:init total elpsd: 4.2 secs | 0 fireSpread init 1

## Dec25 06:17:14 frSprd:init fireSpread

## Dec25 06:17:14 frSprd:init New objects created:

## Dec25 06:17:14 frSprd:init <char>
## Dec25 06:17:14 frSprd:init  1: nPixelsBurned

## Dec25 06:17:14 crbMvm:init total elpsd: 4.3 secs | 0 caribouMovement init 1

## Dec25 06:17:14 crbMvm:init New objects created:

## Dec25 06:17:14 crbMvm:init <char>
## Dec25 06:17:14 crbMvm:init  1:    caribou

## Dec25 06:17:14 frSprd:burn total elpsd: 4.3 secs | 1 fireSpread burn 5

## Dec25 06:17:14 crbMvm:move total elpsd: 4.4 secs | 1 caribouMovement move 5

## Dec25 06:17:14 crbMvm:move New objects created:

## Dec25 06:17:14 crbMvm:move <char>
## Dec25 06:17:14 crbMvm:move  1: habitatQuality

## Dec25 06:17:14 frSprd:stats total elpsd: 4.4 secs | 1 fireSpread stats 5

## Dec25 06:17:14 frSprd:stats fireSpread

## Dec25 06:17:14 crbMvm:move total elpsd: 4.4 secs | 1.083 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.5 secs | 1.167 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.5 secs | 1.25 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.5 secs | 1.333 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.5 secs | 1.417 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.5 secs | 1.5 caribouMovement move 5

## Dec25 06:17:14 crbMvm:move total elpsd: 4.5 secs | 1.583 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.6 secs | 1.667 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.6 secs | 1.75 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.6 secs | 1.833 caribouMovement move

## Dec25 06:17:14 crbMvm:move total elpsd: 4.6 secs | 1.917 caribouMovement move

## Dec25 06:17:14 frSprd:burn total elpsd: 4.6 secs | 2 fireSpread burn 5

## Dec25 06:17:14 crbMvm:move total elpsd: 4.7 secs | 2 caribouMovement move 5

## Dec25 06:17:14 frSprd:stats total elpsd: 4.7 secs | 2 fireSpread stats 5

## Dec25 06:17:14 frSprd:stats fireSpread

## Dec25 06:17:14 save  :spades total elpsd: 4.7 secs | 2 save spades 10

## simList saved in
## SpaDES.core:::savedSimEnv()$.sim
## It will be deleted at next spades() call.

## No packages to install/update

## Dec25 06:17:16 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.

## Dec25 06:17:16 chckpn:init total elpsd: 3.8 secs | 0 checkpoint init 0

## Dec25 06:17:16 save  :init total elpsd: 3.8 secs | 0 save init 0

## Dec25 06:17:16 prgrss:init total elpsd: 3.8 secs | 0 progress init 0

## Dec25 06:17:16 load  :init total elpsd: 3.8 secs | 0 load init 0

## Dec25 06:17:16 rndmLn:init total elpsd: 3.8 secs | 0 randomLandscapes init 1

## Dec25 06:17:16 rndmLn:init New objects created:

## Dec25 06:17:16 rndmLn:init <char>
## Dec25 06:17:16 rndmLn:init  1:  landscape

## Dec25 06:17:16 frSprd:init total elpsd: 3.9 secs | 0 fireSpread init 1

## Dec25 06:17:16 frSprd:init fireSpread

## Dec25 06:17:16 frSprd:init New objects created:

## Dec25 06:17:16 frSprd:init <char>
## Dec25 06:17:16 frSprd:init  1: nPixelsBurned

## Dec25 06:17:16 crbMvm:init total elpsd: 3.9 secs | 0 caribouMovement init 1

## Dec25 06:17:16 crbMvm:init New objects created:

## Dec25 06:17:16 crbMvm:init <char>
## Dec25 06:17:16 crbMvm:init  1:    caribou

## Dec25 06:17:16 frSprd:burn total elpsd: 3.9 secs | 1 fireSpread burn 5

## Dec25 06:17:16 crbMvm:move total elpsd: 3.9 secs | 1 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move New objects created:

## Dec25 06:17:16 crbMvm:move <char>
## Dec25 06:17:16 crbMvm:move  1: habitatQuality

## Dec25 06:17:16 frSprd:stats total elpsd: 3.9 secs | 1 fireSpread stats 5

## Dec25 06:17:16 frSprd:stats fireSpread

## Dec25 06:17:16 crbMvm:move total elpsd: 4 secs | 1.083 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move total elpsd: 4 secs | 1.167 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move total elpsd: 4 secs | 1.25 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move total elpsd: 4 secs | 1.333 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move total elpsd: 4 secs | 1.417 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move total elpsd: 4 secs | 1.5 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move total elpsd: 4 secs | 1.583 caribouMovement move 5

## Dec25 06:17:16 crbMvm:move total elpsd: 4.1 secs | 1.667 caribouMovement move

## Dec25 06:17:16 crbMvm:move total elpsd: 4.1 secs | 1.75 caribouMovement move

## Dec25 06:17:16 crbMvm:move total elpsd: 4.1 secs | 1.833 caribouMovement move

## Dec25 06:17:16 crbMvm:move total elpsd: 4.1 secs | 1.917 caribouMovement move

## Dec25 06:17:16 frSprd:burn total elpsd: 4.1 secs | 2 fireSpread burn 5

## Dec25 06:17:17 crbMvm:move total elpsd: 4.2 secs | 2 caribouMovement move 5

## Dec25 06:17:17 frSprd:stats total elpsd: 4.2 secs | 2 fireSpread stats 5

## Dec25 06:17:17 frSprd:stats fireSpread

## Dec25 06:17:17 save  :spades total elpsd: 4.2 secs | 2 save spades 10

## simList saved in
## SpaDES.core:::savedSimEnv()$.sim
## It will be deleted at next spades() call.

## shutting down parallel nodes

attr(sims, "experiment")$expDesign # shows 2 replicates of same experiment

##   modules expLevel replicate
## 1       1        1         1
## 2       1        1         2

Caching at the experiment level

As with spades, an experiment can be cached.

At the `experiment` level

This functionality can be achieved within an experiment call. This can be done 2 ways, either: “internally” through the cache argument, which will cache each spades call; or, “externally” which will cache the entire experiment. If there are lots of spades calls, then the former will be slow as the simList will be digested once per spades call.

Using cache argument

system.time({
  sims1 <- experiment(mySim, replicates = 2, cache = TRUE, .plotInitialTime = NA)
})

## No packages to install/update

## Dec25 06:17:19 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.

## Dec25 06:17:19 chckpn:init total elpsd: 3.8 secs | 0 checkpoint init 0

## Dec25 06:17:19 save  :init total elpsd: 3.8 secs | 0 save init 0

## Dec25 06:17:19 prgrss:init total elpsd: 3.8 secs | 0 progress init 0

## Dec25 06:17:19 load  :init total elpsd: 3.8 secs | 0 load init 0

## Dec25 06:17:19 rndmLn:init total elpsd: 3.8 secs | 0 randomLandscapes init 1

## Dec25 06:17:19 rndmLn:init New objects created:

## Dec25 06:17:19 rndmLn:init <char>
## Dec25 06:17:19 rndmLn:init  1:  landscape

## Dec25 06:17:19 frSprd:init total elpsd: 3.9 secs | 0 fireSpread init 1

## Dec25 06:17:19 frSprd:init fireSpread

## Dec25 06:17:19 frSprd:init New objects created:

## Dec25 06:17:19 frSprd:init <char>
## Dec25 06:17:19 frSprd:init  1: nPixelsBurned

## Dec25 06:17:19 crbMvm:init total elpsd: 3.9 secs | 0 caribouMovement init 1

## Dec25 06:17:19 crbMvm:init New objects created:

## Dec25 06:17:19 crbMvm:init <char>
## Dec25 06:17:19 crbMvm:init  1:    caribou

## Dec25 06:17:19 frSprd:burn total elpsd: 3.9 secs | 1 fireSpread burn 5

## Dec25 06:17:19 crbMvm:move total elpsd: 3.9 secs | 1 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move New objects created:

## Dec25 06:17:19 crbMvm:move <char>
## Dec25 06:17:19 crbMvm:move  1: habitatQuality

## Dec25 06:17:19 frSprd:stats total elpsd: 3.9 secs | 1 fireSpread stats 5

## Dec25 06:17:19 frSprd:stats fireSpread

## Dec25 06:17:19 crbMvm:move total elpsd: 4 secs | 1.083 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4 secs | 1.167 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4 secs | 1.25 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4 secs | 1.333 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4 secs | 1.417 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4 secs | 1.5 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4 secs | 1.583 caribouMovement move 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4.1 secs | 1.667 caribouMovement move

## Dec25 06:17:19 crbMvm:move total elpsd: 4.1 secs | 1.75 caribouMovement move

## Dec25 06:17:19 crbMvm:move total elpsd: 4.1 secs | 1.833 caribouMovement move

## Dec25 06:17:19 crbMvm:move total elpsd: 4.1 secs | 1.917 caribouMovement move

## Dec25 06:17:19 frSprd:burn total elpsd: 4.1 secs | 2 fireSpread burn 5

## Dec25 06:17:19 crbMvm:move total elpsd: 4.2 secs | 2 caribouMovement move 5

## Dec25 06:17:19 frSprd:stats total elpsd: 4.2 secs | 2 fireSpread stats 5

## Dec25 06:17:19 frSprd:stats fireSpread

## Dec25 06:17:19 save  :spades total elpsd: 4.2 secs | 2 save spades 10

## simList saved in
## SpaDES.core:::savedSimEnv()$.sim
## It will be deleted at next spades() call.

## Saved! Cache file: 556765d7880e4160.rds; fn: spades

## Object to retrieve (fn: spades, 556765d7880e4160.rds) ...

## Loaded! Cached result from previous spades call

## from  module

## shutting down parallel nodes

##    user  system elapsed 
##   3.220   0.059   3.286

# internal -- second time faster
system.time({
  sims2 <- experiment(mySim, replicates = 2, cache = TRUE, .plotInitialTime = NA)
})

## Object to retrieve (fn: spades, 556765d7880e4160.rds) ...

## Loaded! Cached result from previous spades call

## from  module

## Object to retrieve (fn: spades, 556765d7880e4160.rds) ...

## Loaded! Cached result from previous spades call

## from  module

## shutting down parallel nodes

##    user  system elapsed 
##   0.137   0.007   0.145

all.equal(sims1, sims2)

##  [1] "Component 1: Names: 5 string mismatches"                                                                                                                 
##  [2] "Component 1: Length mismatch: comparison on first 6 components"                                                                                          
##  [3] "Component 1: Component 2: Modes: numeric, NULL"                                                                                                          
##  [4] "Component 1: Component 2: Lengths: 4, 0"                                                                                                                 
##  [5] "Component 1: Component 2: target is numeric, current is NULL"                                                                                            
##  [6] "Component 1: Component 3: target is NULL, current is PackedSpatVector"                                                                                   
##  [7] "Component 1: Component 4: Attributes: < Names: 2 string mismatches >"                                                                                    
##  [8] "Component 1: Component 4: Attributes: < Length mismatch: comparison on first 4 components >"                                                             
##  [9] "Component 1: Component 4: Attributes: < Component 1: Names: 1 string mismatch >"                                                                         
## [10] "Component 1: Component 4: Attributes: < Component 1: Attributes: < names for target but not for current > >"                                             
## [11] "Component 1: Component 4: Attributes: < Component 1: Attributes: < Length mismatch: comparison on first 0 components > >"                                
## [12] "Component 1: Component 4: Attributes: < Component 1: Length mismatch: comparison on first 1 components >"                                                
## [13] "Component 1: Component 4: Attributes: < Component 1: Component 1: Modes: character, list >"                                                              
## [14] "Component 1: Component 4: Attributes: < Component 1: Component 1: Lengths: 81, 1 >"                                                                      
## [15] "Component 1: Component 4: Attributes: < Component 1: Component 1: target is character, current is list >"                                                
## [16] "Component 1: Component 4: Attributes: < Component 2: 1 string mismatch >"                                                                                
## [17] "Component 1: Component 4: Attributes: < Component 3: Modes: numeric, character >"                                                                        
## [18] "Component 1: Component 4: Attributes: < Component 3: Lengths: 162, 1 >"                                                                                  
## [19] "Component 1: Component 4: Attributes: < Component 3: Attributes: < Modes: list, NULL > >"                                                                
## [20] "Component 1: Component 4: Attributes: < Component 3: Attributes: < Lengths: 2, 0 > >"                                                                    
## [21] "Component 1: Component 4: Attributes: < Component 3: Attributes: < names for target but not for current > >"                                             
## [22] "Component 1: Component 4: Attributes: < Component 3: Attributes: < current is not list-like > >"                                                         
## [23] "Component 1: Component 4: Attributes: < Component 3: target is matrix, current is character >"                                                           
## [24] "Component 1: Component 4: Attributes: < Component 4: Modes: character, numeric >"                                                                        
## [25] "Component 1: Component 4: Attributes: < Component 4: Lengths: 1, 10000 >"                                                                                
## [26] "Component 1: Component 4: Attributes: < Component 4: Attributes: < target is NULL, current is list > >"                                                  
## [27] "Component 1: Component 4: Attributes: < Component 4: target is character, current is matrix >"                                                           
## [28] "Component 1: Component 5: Attributes: < Component 1: Component 1: Length mismatch: comparison on first 1 components >"                                   
## [29] "Component 1: Component 5: Attributes: < Component 1: Component 1: Component 1: Attributes: < Component \"row.names\": Numeric: lengths (8, 9) differ > >"
## [30] "Component 1: Component 5: Attributes: < Component 1: Component 1: Component 1: Component 1: Numeric: lengths (8, 9) differ >"                            
## [31] "Component 1: Component 5: Attributes: < Component 1: Component 1: Component 1: Component 2: Numeric: lengths (8, 9) differ >"                            
## [32] "Component 1: Component 5: Attributes: < Component 1: Component 1: Component 1: Component 3: Numeric: lengths (8, 9) differ >"                            
## [33] "Component 1: Component 5: Attributes: < Component 1: Component 1: Component 1: Component 4: Numeric: lengths (8, 9) differ >"                            
## [34] "Component 1: Component 5: Attributes: < Component 1: Component 1: Component 1: Component 5: Numeric: lengths (8, 9) differ >"                            
## [35] "Component 1: Component 5: Attributes: < Component 3: 1 string mismatch >"                                                                                
## [36] "Component 1: Component 5: Attributes: < Component 4: Attributes: < Component 1: Mean relative difference: 4 > >"                                         
## [37] "Component 1: Component 5: Attributes: < Component 4: Attributes: < Component 2: Component 2: 1 string mismatch > >"                                      
## [38] "Component 1: Component 5: Attributes: < Component 4: Numeric: lengths (10000, 50000) differ >"                                                           
## [39] "Component 1: Component 6: Modes: S4, numeric"                                                                                                            
## [40] "Component 1: Component 6: Lengths: 1, 2"                                                                                                                 
## [41] "Component 1: Component 6: Attributes: < Modes: list, NULL >"                                                                                             
## [42] "Component 1: Component 6: Attributes: < Lengths: 4, 0 >"                                                                                                 
## [43] "Component 1: Component 6: Attributes: < names for target but not for current >"                                                                          
## [44] "Component 1: Component 6: Attributes: < current is not list-like >"

Wrapping `experiment` with `Cache`

Here, the simList (and other arguments to experiment) is hashed once, and if it is found to be the same as previous, then the returned list of simList objects is recovered. This means that even a very large experiment, with many replicates and combinations of parameters and modules can be recovered very quickly. Here we show that you can output objects to disk, so the list of simList objects doesn’t get too big. Then, when we recover it in the Cached version, all the files are still there, the list of simList objects is small, so very fast to recover.

# External
outputs(mySim) <- data.frame(objectName = "landscape")
system.time({
  sims3 <- Cache(experiment, mySim, replicates = 3, .plotInitialTime = NA,
                 clearSimEnv = TRUE)
})

## No packages to install/update

## Dec25 06:17:23 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.

## Dec25 06:17:23 chckpn:init total elpsd: 3.8 secs | 0 checkpoint init 0

## Dec25 06:17:23 save  :init total elpsd: 3.8 secs | 0 save init 0

## Dec25 06:17:23 prgrss:init total elpsd: 3.8 secs | 0 progress init 0

## Dec25 06:17:23 load  :init total elpsd: 3.8 secs | 0 load init 0

## Dec25 06:17:23 rndmLn:init total elpsd: 3.8 secs | 0 randomLandscapes init 1

## Dec25 06:17:23 rndmLn:init New objects created:

## Dec25 06:17:23 rndmLn:init <char>
## Dec25 06:17:23 rndmLn:init  1:  landscape

## Dec25 06:17:23 frSprd:init total elpsd: 3.9 secs | 0 fireSpread init 1

## Dec25 06:17:23 frSprd:init fireSpread

## Dec25 06:17:23 frSprd:init New objects created:

## Dec25 06:17:23 frSprd:init <char>
## Dec25 06:17:23 frSprd:init  1: nPixelsBurned

## Dec25 06:17:23 crbMvm:init total elpsd: 3.9 secs | 0 caribouMovement init 1

## Dec25 06:17:23 crbMvm:init New objects created:

## Dec25 06:17:23 crbMvm:init <char>
## Dec25 06:17:23 crbMvm:init  1:    caribou

## Dec25 06:17:23 frSprd:burn total elpsd: 3.9 secs | 1 fireSpread burn 5

## Dec25 06:17:23 crbMvm:move total elpsd: 3.9 secs | 1 caribouMovement move 5

## Dec25 06:17:23 crbMvm:move New objects created:

## Dec25 06:17:23 crbMvm:move <char>
## Dec25 06:17:23 crbMvm:move  1: habitatQuality

## Dec25 06:17:23 frSprd:stats total elpsd: 3.9 secs | 1 fireSpread stats 5

## Dec25 06:17:23 frSprd:stats fireSpread

## Dec25 06:17:23 crbMvm:move total elpsd: 3.9 secs | 1.083 caribouMovement move

## Dec25 06:17:23 crbMvm:move total elpsd: 4 secs | 1.167 caribouMovement move 5

## Dec25 06:17:23 crbMvm:move total elpsd: 4 secs | 1.25 caribouMovement move 5

## Dec25 06:17:23 crbMvm:move total elpsd: 4 secs | 1.333 caribouMovement move 5

## Dec25 06:17:23 crbMvm:move total elpsd: 4 secs | 1.417 caribouMovement move 5

## Dec25 06:17:23 crbMvm:move total elpsd: 4 secs | 1.5 caribouMovement move 5

## Dec25 06:17:23 crbMvm:move total elpsd: 4.1 secs | 1.583 caribouMovement move

## Dec25 06:17:23 crbMvm:move total elpsd: 4.1 secs | 1.667 caribouMovement move

## Dec25 06:17:24 crbMvm:move total elpsd: 4.1 secs | 1.75 caribouMovement move

## Dec25 06:17:24 crbMvm:move total elpsd: 4.1 secs | 1.833 caribouMovement move

## Dec25 06:17:24 crbMvm:move total elpsd: 4.1 secs | 1.917 caribouMovement move

## Dec25 06:17:24 frSprd:burn total elpsd: 4.1 secs | 2 fireSpread burn 5

## Dec25 06:17:24 crbMvm:move total elpsd: 4.2 secs | 2 caribouMovement move 5

## Dec25 06:17:24 frSprd:stats total elpsd: 4.2 secs | 2 fireSpread stats 5

## Dec25 06:17:24 frSprd:stats fireSpread

## Dec25 06:17:24 save  :spades total elpsd: 4.2 secs | 2 save spades 10

## simList saved in
## SpaDES.core:::savedSimEnv()$.sim
## It will be deleted at next spades() call.

## No packages to install/update

## Dec25 06:17:25 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.

## Dec25 06:17:25 chckpn:init total elpsd: 3.8 secs | 0 checkpoint init 0

## Dec25 06:17:25 save  :init total elpsd: 3.8 secs | 0 save init 0

## Dec25 06:17:25 prgrss:init total elpsd: 3.8 secs | 0 progress init 0

## Dec25 06:17:25 load  :init total elpsd: 3.8 secs | 0 load init 0

## Dec25 06:17:25 rndmLn:init total elpsd: 3.8 secs | 0 randomLandscapes init 1

## Dec25 06:17:25 rndmLn:init New objects created:

## Dec25 06:17:25 rndmLn:init <char>
## Dec25 06:17:25 rndmLn:init  1:  landscape

## Dec25 06:17:25 frSprd:init total elpsd: 3.9 secs | 0 fireSpread init 1

## Dec25 06:17:25 frSprd:init fireSpread

## Dec25 06:17:25 frSprd:init New objects created:

## Dec25 06:17:25 frSprd:init <char>
## Dec25 06:17:25 frSprd:init  1: nPixelsBurned

## Dec25 06:17:25 crbMvm:init total elpsd: 3.9 secs | 0 caribouMovement init 1

## Dec25 06:17:25 crbMvm:init New objects created:

## Dec25 06:17:25 crbMvm:init <char>
## Dec25 06:17:25 crbMvm:init  1:    caribou

## Dec25 06:17:25 frSprd:burn total elpsd: 3.9 secs | 1 fireSpread burn 5

## Dec25 06:17:25 crbMvm:move total elpsd: 3.9 secs | 1 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move New objects created:

## Dec25 06:17:25 crbMvm:move <char>
## Dec25 06:17:25 crbMvm:move  1: habitatQuality

## Dec25 06:17:25 frSprd:stats total elpsd: 3.9 secs | 1 fireSpread stats 5

## Dec25 06:17:25 frSprd:stats fireSpread

## Dec25 06:17:25 crbMvm:move total elpsd: 4 secs | 1.083 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4 secs | 1.167 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4 secs | 1.25 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4 secs | 1.333 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4 secs | 1.417 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4 secs | 1.5 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4 secs | 1.583 caribouMovement move 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4.1 secs | 1.667 caribouMovement move

## Dec25 06:17:25 crbMvm:move total elpsd: 4.1 secs | 1.75 caribouMovement move

## Dec25 06:17:25 crbMvm:move total elpsd: 4.1 secs | 1.833 caribouMovement move

## Dec25 06:17:25 crbMvm:move total elpsd: 4.1 secs | 1.917 caribouMovement move

## Dec25 06:17:25 frSprd:burn total elpsd: 4.1 secs | 2 fireSpread burn 5

## Dec25 06:17:25 crbMvm:move total elpsd: 4.2 secs | 2 caribouMovement move 5

## Dec25 06:17:25 frSprd:stats total elpsd: 4.2 secs | 2 fireSpread stats 5

## Dec25 06:17:25 frSprd:stats fireSpread

## Dec25 06:17:25 save  :spades total elpsd: 4.2 secs | 2 save spades 10

## simList saved in
## SpaDES.core:::savedSimEnv()$.sim
## It will be deleted at next spades() call.

## No packages to install/update

## Dec25 06:17:26 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.

## Dec25 06:17:26 chckpn:init total elpsd: 3.8 secs | 0 checkpoint init 0

## Dec25 06:17:26 save  :init total elpsd: 3.8 secs | 0 save init 0

## Dec25 06:17:26 prgrss:init total elpsd: 3.8 secs | 0 progress init 0

## Dec25 06:17:26 load  :init total elpsd: 3.8 secs | 0 load init 0

## Dec25 06:17:26 rndmLn:init total elpsd: 3.8 secs | 0 randomLandscapes init 1

## Dec25 06:17:26 rndmLn:init New objects created:

## Dec25 06:17:26 rndmLn:init <char>
## Dec25 06:17:26 rndmLn:init  1:  landscape

## Dec25 06:17:26 frSprd:init total elpsd: 3.9 secs | 0 fireSpread init 1

## Dec25 06:17:26 frSprd:init fireSpread

## Dec25 06:17:26 frSprd:init New objects created:

## Dec25 06:17:26 frSprd:init <char>
## Dec25 06:17:26 frSprd:init  1: nPixelsBurned

## Dec25 06:17:26 crbMvm:init total elpsd: 3.9 secs | 0 caribouMovement init 1

## Dec25 06:17:26 crbMvm:init New objects created:

## Dec25 06:17:26 crbMvm:init <char>
## Dec25 06:17:26 crbMvm:init  1:    caribou

## Dec25 06:17:26 frSprd:burn total elpsd: 3.9 secs | 1 fireSpread burn 5

## Dec25 06:17:26 crbMvm:move total elpsd: 3.9 secs | 1 caribouMovement move 5

## Dec25 06:17:26 crbMvm:move New objects created:

## Dec25 06:17:26 crbMvm:move <char>
## Dec25 06:17:26 crbMvm:move  1: habitatQuality

## Dec25 06:17:26 frSprd:stats total elpsd: 3.9 secs | 1 fireSpread stats 5

## Dec25 06:17:26 frSprd:stats fireSpread

## Dec25 06:17:26 crbMvm:move total elpsd: 4 secs | 1.083 caribouMovement move 5

## Dec25 06:17:26 crbMvm:move total elpsd: 4 secs | 1.167 caribouMovement move 5

## Dec25 06:17:26 crbMvm:move total elpsd: 4 secs | 1.25 caribouMovement move 5

## Dec25 06:17:26 crbMvm:move total elpsd: 4 secs | 1.333 caribouMovement move 5

## Dec25 06:17:26 crbMvm:move total elpsd: 4 secs | 1.417 caribouMovement move 5

## Dec25 06:17:26 crbMvm:move total elpsd: 4 secs | 1.5 caribouMovement move 5

## Dec25 06:17:26 crbMvm:move total elpsd: 4.1 secs | 1.583 caribouMovement move

## Dec25 06:17:26 crbMvm:move total elpsd: 4.1 secs | 1.667 caribouMovement move

## Dec25 06:17:26 crbMvm:move total elpsd: 4.1 secs | 1.75 caribouMovement move

## Dec25 06:17:26 crbMvm:move total elpsd: 4.1 secs | 1.833 caribouMovement move

## Dec25 06:17:26 crbMvm:move total elpsd: 4.1 secs | 1.917 caribouMovement move

## Dec25 06:17:26 frSprd:burn total elpsd: 4.1 secs | 2 fireSpread burn 5

## Dec25 06:17:26 crbMvm:move total elpsd: 4.2 secs | 2 caribouMovement move 5

## Dec25 06:17:26 frSprd:stats total elpsd: 4.2 secs | 2 fireSpread stats 5

## Dec25 06:17:26 frSprd:stats fireSpread

## Dec25 06:17:26 save  :spades total elpsd: 4.2 secs | 2 save spades 10

## simList saved in
## SpaDES.core:::savedSimEnv()$.sim
## It will be deleted at next spades() call.

## shutting down parallel nodes

## Saved! Cache file: aa25cfcd4ea03398.rds; fn: experiment

##    user  system elapsed 
##   4.448   0.073   4.527

The second time is way faster. We see the output files in the same location.

system.time({
  sims4 <- Cache(experiment, mySim, replicates = 3, .plotInitialTime = NA,
                 clearSimEnv = TRUE)
})

## Object to retrieve (fn: experiment, aa25cfcd4ea03398.rds) ...

## Loaded! Cached result from previous experiment call

##    user  system elapsed 
##   0.062   0.004   0.066

# test they are all equal
lapply(1:2, function(x) all.equal(sims3[[x]], sims4[[x]]))

## [[1]]
## [1] TRUE
## 
## [[2]]
## [1] TRUE

dir(outputPath(mySim), recursive = TRUE)

## [1] "experiment.RData"         "rep1/caribou_year2.rds"  
## [3] "rep1/landscape_year2.rds" "rep2/caribou_year2.rds"  
## [5] "rep2/landscape_year2.rds" "rep3/landscape_year2.rds"

Notice that speed up can be enormous; in this case ~100 times faster.

Nested Caching

This is a continuation of the Nested Caching section in the iv-caching vignetted in the SpaDES.core package. - Imagine we have large model, with many modules, with replication and alternative module collections (e.g., alternative fire models) - To run this would have a nested structure with the following functions:

simInit --> many .inputObjects calls

experiment --> many spades calls --> many module calls --> many event calls --> many function calls

Lets say we start to introduce caching to this structure. We start from the “inner” most functions that we could imaging Caching would be useful. Lets say there are some GIS operations, like raster::projectRaster, which operates on an input shapefile. We can Cache the projectRaster call to make this much faster, since it will always be the same result for a given input raster.

If we look back at our structure above, we see that we still have LOTS of places that are not Cached. That means that the experiment call will still spawn many spades calls, which will still spawn many module calls, and many event calls, just to get to the one Cache(projectRaster) call which is Cached. This function will likely be called hundreds of times (because experiment runs the spades call 100 times due to replication). This is good, but Cache does take some time. So, even if Cache(projectRaster) takes only 0.02 seconds, calling it hundreds of times means maybe 4 seconds. If we are doing this for many functions, then this will be too slow.

We can start putting Cache all up the sequence of calls. Unfortunately, the way we use Cache at each of these levels is a bit different, so we need a slightly different approach for each.

Cache the `experiment` call

Cache(experiment)

This will assess the simList (the objects, times, modules, etc.) and if they are all the same, it will return the final list of simLists that came from the first experiment call. NOTE: because this can be large, it is likely that you want clearSimEnv = TRUE, and have all objects that are needed after the experiment call saved to disk. Any stochasticity/randomness inside modules will be frozen. This is likely ok if the objective is to show results in a web app (via shiny or otherwise) or another visualization about the experiment outputs, e.g., comparing treatments, once sufficient stochasticity has been achieved.

mySimListOut <- Cache(experiment, mySim, clearSimEnv = TRUE)

Cache the `spades` calls inside `experiment`

experiment(cache = TRUE)

This will cache each of the spades calls inside the experiment call. That means that there are as many cache events as there are replicates and experimental treatments, which, again could be a lot. Like caching the experiment call, stochasticity/randomness will be frozen. Note, one good use of this is when you are making iterative, incremental replication, e.g.,

mySimOut <- experiment(mySim, replicates = 5, cache = TRUE)

You decide after waiting 10 minutes for it to finish, that you need more replication. Rather than start from zero replicates, you can just pick up where you left off:

mySimOut <- experiment(mySim, replicates = 10, cache = TRUE)

This will only add 5 more replicates.

true

05 Simulation Experiments

Running a groups of simulations: the experiment function

Caching at the experiment level

At the experiment level