Package 'SpaDES.project'

Description

Quickly setup 'SpaDES' project directories and add modules using templates.

Author(s)

Maintainer: Eliot J B McIntire [email protected] (ORCID)

Other contributors:

• Alex M Chubaty [email protected] (ORCID) [contributor]

• Ian Eddy [email protected] (ORCID) [contributor]

• Ceres Barros [email protected] [contributor]

See Also

Useful links:

• https://spades-project.predictiveecology.org/

• https://github.com/PredictiveEcology/SpaDES.project

• Report bugs at https://github.com/PredictiveEcology/SpaDES.project/issues

Description

For a given name, this will return the default library for packages.

Usage

.libPathDefault(name)

Arguments

Value

A path where the packages will be installed.

Description

1. remove project library directory created using setupProject();

2. remove project paths created using setupProject;

3. restore original library paths.

Usage

.teardownProject(prjPaths, origLibPaths)

Arguments

Value

NULL. Invoked for its side effects.

Note

not intended to be called by users

Description

Searches from current working directory for and Rstudio project file or git repository, falling back on using the current working directory.

Usage

findProjectPath()

findProjectName()

Value

findProjectPath returns an absolute path; findProjectName returns the basename of the path.

Description

This can be used within e.g., the options or params arguments for setupProject to get a ready-made file for a project.

Usage

getGithubFile(
  gitRepoFile,
  overwrite = FALSE,
  destDir = ".",
  verbose = getOption("Require.verbose")
)

Arguments

See Also

getModule

Examples

filename <- getGithubFile("PredictiveEcology/LandWeb@development/01b-options.R",
                          destDir = Require::tempdir2())

Description

Simple function to download a SpaDES module as GitHub repository

Usage

getModule(
  modules,
  modulePath,
  overwrite = FALSE,
  verbose = getOption("Require.verbose", 1L)
)

Arguments

See Also

getGithubFile

Description

When exploring existing modules, these tools help identify and navigate modules and their interdependencies.

Usage

listModules(
  keywords,
  accounts,
  includeForks = FALSE,
  includeArchived = FALSE,
  excludeStale = TRUE,
  omit = c("fireSense_dataPrepFitRas"),
  purge = FALSE,
  returnList = FALSE,
  verbose = getOption("Require.verbose", 1L)
)

moduleDependencies(
  modules,
  modulePath = getOption("reproducible.modulePath", ".")
)

moduleDependenciesToGraph(md)

PlotModuleGraph(graph)

Arguments

Value

listModules returns a character vector of paste0(account, "/", Repository) for all SpaDES modules in the given repositories with the accounts and keywords provided.

See Also

metadataInModules() helps to see different metadata elements in a folder of modules.

Examples

listModules(accounts = "PredictiveEcology", "none")

Description

Make DESCRIPTION file(s) from SpaDES module metadata

Usage

makeDESCRIPTIONproject(
  modules,
  modulePath,
  projectPath = ".",
  singleDESCRIPTION = TRUE,
  package = "Project",
  title = "Project",
  description = "Project",
  version = "1.0.0",
  authors = Sys.info()["user"],
  write = TRUE,
  verbose = getOption("Require.verbose")
)

makeDESCRIPTION(
  modules,
  modulePath,
  projectPath = ".",
  singleDESCRIPTION = FALSE,
  package,
  title,
  date,
  description,
  version,
  authors,
  write = TRUE,
  verbose,
  metadataList,
  ...
)

Arguments

Description

Parses module code, looking for the metadataItem (default = "reqdPkgs") element in the defineModule function.

Usage

packagesInModules(modules, modulePath = getOption("spades.modulePath"))

metadataInModules(
  modules,
  metadataItem = "reqdPkgs",
  modulePath = getOption("spades.modulePath"),
  needUnlist,
  verbose = getOption("Require.verbose", 1L)
)

Arguments

Value

A character vector of sorted, unique packages that are identified in all named modules, or if modules is omitted, then all modules in modulePath.

Description

pkgload::load_all does not automatically deal with dependency chains: the user must manually load the dependency chain in order with separate calls to pkgload::load_all. Also, it does not use caching. This function allows nested caching for a sequence of packages that depend on one another. For example, if a user has 3 packages that have dependency chain: A is a dependency of B which is a dependency of C. If a change happens in C, then pkgload::load_all will only be called on C. If a change happens in A, then pkgload::load_all will be called on A, then B, then C.

Usage

pkgload2(
  depsPaths = file.path("~/GitHub", c("reproducible", "SpaDES.core", "LandR")),
  envir = parent.frame()
)

Arguments

Value

This is called for its side effects, which are 2: pkgload::load_all on the packages that need it, and an object, .prevDigs that is assigned to envir.

Description

This function will create a sub-folder of the lib.loc directory that is based on the R version and the platform, as per the standard R package directory naming convention

Usage

setProjPkgDir(lib.loc = "packages", verbose = getOption("Require.verbose", 1L))

Arguments

Description

These functions will allow more user control, though in most circumstances, it should be unnecessary to call them directly.

Usage

setupPaths(
  name,
  paths,
  inProject,
  standAlone = TRUE,
  libPaths = NULL,
  updateRprofile = getOption("SpaDES.project.updateRprofile", TRUE),
  Restart = getOption("SpaDES.project.Restart", FALSE),
  overwrite = FALSE,
  envir = parent.frame(),
  useGit = getOption("SpaDES.project.useGit", FALSE),
  verbose = getOption("Require.verbose", 1L),
  dots,
  defaultDots,
  ...
)

setupFunctions(
  functions,
  name,
  sideEffects,
  paths,
  overwrite = FALSE,
  envir = parent.frame(),
  verbose = getOption("Require.verbose", 1L),
  dots,
  defaultDots,
  ...
)

setupSideEffects(
  name,
  sideEffects,
  paths,
  times,
  overwrite = FALSE,
  envir = parent.frame(),
  verbose = getOption("Require.verbose", 1L),
  dots,
  defaultDots,
  ...
)

setupOptions(
  name,
  options,
  paths,
  times,
  overwrite = FALSE,
  envir = parent.frame(),
  verbose = getOption("Require.verbose", 1L),
  dots,
  defaultDots,
  useGit = getOption("SpaDES.project.useGit", FALSE),
  updateRprofile = getOption("SpaDES.project.updateRprofile", TRUE),
  ...
)

setupModules(
  name,
  paths,
  modules,
  inProject,
  useGit = getOption("SpaDES.project.useGit", FALSE),
  overwrite = FALSE,
  envir = parent.frame(),
  gitUserName,
  verbose = getOption("Require.verbose", 1L),
  dots,
  defaultDots,
  updateRprofile = getOption("SpaDES.project.updateRprofile", TRUE),
  ...
)

setupPackages(
  packages,
  modulePackages = list(),
  require = list(),
  paths,
  libPaths,
  setLinuxBinaryRepo = TRUE,
  standAlone,
  envir = parent.frame(),
  verbose = getOption("Require.verbose"),
  dots,
  defaultDots,
  ...
)

setupParams(
  name,
  params,
  paths,
  modules,
  times,
  options,
  overwrite = FALSE,
  envir = parent.frame(),
  verbose = getOption("Require.verbose", 1L),
  dots,
  defaultDots,
  ...
)

setupGitIgnore(
  paths,
  gitignore = getOption("SpaDES.project.gitignore", TRUE),
  verbose
)

setupStudyArea(
  studyArea,
  paths,
  envir,
  verbose = getOption("Require.verbose", 1L)
)

setupFiles(
  files,
  paths,
  envir = parent.frame(),
  verbose = getOption("Require.verbose", 1L)
)

Arguments

Details

setPaths will fill in any paths that are not explicitly supplied by the user as a named list. These paths that can be set are: projectPath, packagePath, cachePath, inputPath, modulePath, outputPath, rasterPath, scratchPath, terraPath. These are grouped thematically into three groups of paths: projectPath and packagePath affect the project, regardless of whether a user uses SpaDES modules. cachePath, inputPath, outputPath and modulePath are all used by SpaDES within module contexts. scratchPath, rasterPath and terraPath are all "temporary" or "scratch" directories.

setupFunctions will source the functions supplied, with a parent environment being the internal temporary environment of the setupProject, i.e., they will have access to all the objects in the call.

Most arguments in the family of ⁠setup*⁠ functions are run sequentially, even within the argument. Since most arguments take lists, the user can set values at a first value of a list, then use it in calculation of the 2nd value and so on. See examples. This "sequential" evaluation occurs in the ..., setupSideEffects, setupOptions, setupParams (this does not work for setupPaths) can handle sequentially specified values, meaning a user can first create a list of default options, then a list of user-desired options that may or may not replace individual values. This can create hierarchies, based on order.

setupOptions can handle sequentially specified values, meaning a user can first create a list of default options, then a list of user-desired options that may or may not replace individual values. Thus final values will be based on the order that they are provided.

setupModules will download all modules do not yet exist locally. The current test for "exists locally" is simply whether the directory exists. If a user wants to update the module, overwrite = TRUE must be set, or else the user can remove the folder manually.

setupPackages will read the modules' metadata reqdPkgs element. It will combine these with any packages passed manually by the user to packages, and pass all these packages to Require::Install(...).

setupGitIgnore will add.

setupStudyArea only uses inputPath within its paths argument, which will be passed to path argument of gadm. User can pass any named list element that matches the columns in the sf object, including e.g., NAME_1 and, if level = 2, is specified, then NAME_2.

setupStudyArea(list(NAME_1 = "Alberta", "NAME_2" = "Division No. 17", level = 2))

setupFiles is a convenience function intended for interactive use to verify the files being parsed. This is similar to parse, but each element must be a named list or a named object, such as a function. It uses the same specification for https://github.com files as setupProject, i.e., using @ for branch.

setupFiles("PredictiveEcology/PredictiveEcology.org@main/tutos/castorExample/params.R")

Value

setupPaths returns a list of paths that are created. projectPath will be assumed to be the base of other non-temporary and non-R-library paths. This means that all paths that are directly used by simInit are assumed to be relative to the projectPath. If a user chooses to specify absolute paths, then they will be returned as is. It is also called for its side effect which is to call setPaths, with each of these paths as an argument. See table for details. If a user supplies extra paths not useable by SpaDES.core::simInit, these will added as an attribute ("extraPaths") to the paths element in the returned object. These will still exist directly in the returned list if a user uses setupPaths directly, but these will not be returned with setupProject because setupProject is intended to be used with SpaDES.core::simInit. In addition, three paths will be added to this same attribute automatically: projectPath, packagePath, and .prevLibPaths which is the previous value for .libPaths() before changing to packagePath.

setupFunctions returns NULL. All functions will be placed in envir.

setupSideEffects is run for its side effects (e.g., web authentication, custom package options that cannot use base::options), with deliberately nothing returned to user. This, like other parts of this function, attempts to prevent unwanted outcomes that occur when a user uses e.g., source without being very careful about what and where the objects are sourced to.

setupOptions is run for its side effects, namely, changes to the options(). The list of modified options will be added as an attribute (attr(out, "projectOptions")), e.g., so they can be "unset" by user later.

setupModules is run for its side effects, i.e., downloads modules and puts them into the paths[["modulePath"]]. It will return a named list, where the names are the full module names and the list elemen.ts are the R packages that the module depends on (reqsPkgs)

setupPackages is run for its side effects, i.e., installing packages to paths[["packagePath"]].

setupParams prepares a named list of named lists, suitable to be passed to the params argument of simInit.

setupGitIgnore is run for its side effects, i.e., adding either paths$packagePath and/or paths$modulePath to the .gitignore file. It will check whether packagePath is located inside the paths$projectPath and will add this folder to the .gitignore if TRUE. If the project is a git repository with git submodules, then it will add nothing else. If the project is a git repository without git submodules, then the paths$modulePath will be added to the .gitignore file. It is assumed that these modules are used in a ⁠read only⁠ manner.

setupStudyArea will return an sf class object coming from geodata::gadm, with subregion specification as described in the studyArea argument.fsu

setupFiles a named list with each element that was parsed.

Paths

Examples

## simplest case; just creates folders
out <- setupProject(
  paths = list(projectPath = ".") #
)
# specifying functions argument, with a local file and a definition here
tf <- tempfile(fileext = ".R")
fnDefs <- c("fn <- function(x) x\n",
            "fn2 <- function(x) x\n",
            "fn3 <- function(x) terra::rast(x)")
cat(text = fnDefs, file = tf)
funHere <- function(y) y
out <- setupProject(functions = list(a = function(x) return(x),
                                     tf,
                                     funHere = funHere), # have to name it
                    # now use the functions when creating objects
                    drr = 1,
                    b = a(drr),
                    q = funHere(22),
                    ddd = fn3(terra::ext(0,b,0,b)))

Description

setupProject calls a sequence of functions in this order: setupOptions (first time), setupPaths, setupRestart, setupFunctions, setupModules, setupPackages, setupSideEffects, setupOptions (second time), setupParams, and setupGitIgnore.

This sequence will create folder structures, install missing packages from those listed in either the packages, require arguments or in the modules reqdPkgs fields, load packages (only those in the require argument), set options, download or confirm the existence of modules. It will also return elements that can be passed directly to simInit or simInitAndSpades, specifically, modules, params, paths, times, and any named elements passed to .... This function will also , if desired, change the .Rprofile file for this project so that every time the project is opened, it has a specific .libPaths().

There are a number of convenience elements described in the section below. See Details. Because of this sequence, users can take advantage of settings (i.e., objects) that happen (are created) before others. For example, users can set paths then use the paths list to set options that will can update/change paths, or set times and use the times list for certain entries in params.

Usage

setupProject(
  name,
  paths,
  modules,
  packages,
  times,
  options,
  params,
  sideEffects,
  functions,
  config,
  require = NULL,
  studyArea = NULL,
  Restart = getOption("SpaDES.project.Restart"),
  useGit = getOption("SpaDES.project.useGit"),
  setLinuxBinaryRepo = getOption("SpaDES.project.setLinuxBinaryRepo"),
  standAlone = getOption("SpaDES.project.standAlone"),
  libPaths = NULL,
  updateRprofile = getOption("SpaDES.project.updateRprofile"),
  overwrite = getOption("SpaDES.project.overwrite"),
  verbose = getOption("Require.verbose", 1L),
  defaultDots,
  envir = parent.frame(),
  dots,
  ...
)

Arguments

Value

setupProject will return a named list with elements modules, paths, params, and times. The goal of this list is to contain list elements that can be passed directly to simInit.

It will also append all elements passed by the user in the .... This list can be passed directly to SpaDES.core::simInit() or SpaDES.core::simInitAndSpades() using a do.call(). See example.

NOTE: both projectPath and packagePath will be omitted in the paths list as they are used to set current directory (found with getwd()) and .libPaths()[1], but are not accepted by simInit. setupPaths will still return these two paths as its outputs are not expected to be passed directly to simInit (unlike setupProject outputs).

Faster runtime after project is set up

There are a number of checks that occur during setupProject. These take time, particularly after an R restart (there is some caching in RAM that occurs, but this will only speed things up if there is no restart of R). To get the "fastest", these options or settings will speed things up, at the expense of not being completely re-runnable. You can add one or more of these to the arguments. These will only be useful after a project is set up, i.e., setupProject and SpaDES.core::simInit has/have been run at least once to completion (so packages are installed).

options = c(
  reproducible.useMemoise = TRUE,               # For caching, use memory objects
  Require.cloneFrom = Sys.getenv("R_LIBS_USER"),# Use personal library as possible source of packages
  spades.useRequire = FALSE,                    # Won't install packages/update versions
  spades.moduleCodeChecks = FALSE,              # moduleCodeChecks checks for metadata mismatches
  reproducible.inputPaths = "~/allData"),       # For sharing data files across projects
packages = NULL,                                # Prevents any packages installs with setupProject
useGit = FALSE                                  # Prevents checks using git

These will be set early in setupProject, so will affect the running of setupProject. If the user manually sets one of these in addition to setting these, the user options will override these. The remining causes of setupProject being "slow" will be loading the required packages.

These options/arguments can now be set all at once (with caution as these changes will affect how your script will be run) with options(SpaDES.project.fast = TRUE) or in the options argument.

Objective

The overarching objectives for these functions are:

1. To prepare what is needed for simInit.

2. To help a user eliminate virtually all assignments to the .GlobalEnv, as these create and encourage spaghetti code that becomes unreproducible as the project increases in complexity.

3. Be very simple for beginners, but powerful enough to expand to almost any needs of arbitrarily complex projects, using the same structure

4. Deal with the complexities of R package installation and loading when working with modules that may have been created by many users

5. Create a common SpaDES project structure, allowing easy transition from one project to another, regardless of complexity.

Convenience elements

Sequential evaluation

Throughout these functions, efforts have been made to implement sequential evaluation, within files and within lists. This means that a user can use the values from an upstream element in the list. For example, the following where projectPath is part of the list that will be assigned to the paths argument and it is then used in the subsequent list element is valid:

setupPaths(paths = list(projectPath = "here",
                        modulePath = file.path(paths[["projectPath"]], "modules")))

Because of such sequential evaluation, paths, options, and params files can be sequential lists that have impose a hierarchy specified by the order. For example, a user can first create a list of default options, then several lists of user-desired options behind an ⁠if (user("emcintir"))⁠ block that add new or override existing elements, followed by machine specific values, such as paths.

setupOptions(
  maxMemory <- 5e+9 # if (grepl("LandWeb", runName)) 5e+12 else 5e+9

  # Example -- Use any arbitrary object that can be passed in the `...` of `setupOptions`
  #  or `setupProject`
  if (.mode == "development") {
     list(test = 2)
  }
  if (machine("A127")) {
    list(test = 3)
  }
)

Values and/or files

The arguments, paths, options, and params, can all understand lists of named values, character vectors, or a mixture by using a list where named elements are values and unnamed elements are character strings/vectors. Any unnamed character string/vector will be treated as a file path. If that file path has an @ symbol, it will be assumed to be a file that exists on a GitHub repository in ⁠https://github.com⁠. So a user can pass values, or pointers to remote and/or local paths that themselves have values.

The following will set an option as declared, plus read the local file (with relative path), plus download and read the cloud-hosted file.

setupProject(
   options = list(reproducible.useTerra = TRUE,
                  "inst/options.R",
                  "PredictiveEcology/SpaDES.project@transition/inst/options.R")
                 )
   )

This approach allows for an organic growth of complexity, e.g., a user begins with only named lists of values, but then as the number of values increases, it may be helpful to put some in an external file.

NOTE: if the GitHub repository is private the user must configure their GitHub token by setting the GITHUB_PAT environment variable – unfortunately, the usethis approach to setting the token will not work at this moment.

Specifying paths, options, params

If paths, options, and/or params are a character string or character vector (or part of an unnamed list element) the string(s) will be interpreted as files to parse. These files should contain R code that specifies named lists, where the names are one or more paths, options, or are module names, each with a named list of parameters for that named module. This last named list for params follows the convention used for the params argument in simInit(..., params = ).

These files can use paths, times, plus any previous list in the sequence of params or options specified. Any functions that are used must be available, e.g., prefixed Require::normPath if the package has not been loaded (as recommended).

If passing a file to options, it should not set options() explicitly; only create named lists. This enables options checking/validating to occur within setupOptions and setupParams. A simplest case would be a file with this: opts <- list(reproducible.destinationPath = "~/destPath").

All named lists will be parsed into their own environment, and then will be sequentially evaluated (i.e., subsequent lists will have access to previous lists), with each named elements setting or replacing the previously named element of the same name, creating a single list. This final list will be assigned to, e.g., options() inside setupOptions.

Because each list is parsed separately, they to not need to be assigned objects; if they are, the object name can be any name, even if similar to another object's name used to built the same argument's (i.e. paths, params, options) final list. Hence, in an file to passed to options, instead of incrementing the list as:

a <- list(optA = 1)
b <- append(a, list(optB = 2))
c <- append(b, list(optC = 2.5))
d <- append(c, list(optD = 3))

one can do:

a <- list(optA = 1)
a <- list(optB = 2)
c <- list(optC = 2.5)
list(optD = 3)

NOTE: only atomics (i.e., character, numeric, etc.), named lists, or either of these that are protected by 1 level of "if" are parsed. This will not work, therefore, for other side-effect elements, like authenticating with a cloud service.

Several helper functions exist within SpaDES.project that may be useful, such as user(...), machine(...)

Can hard code arguments that may be missing

To allow for batch submission, a user can specify code argument = value even if value is missing. This type of specification will not work in normal parsing of arguments, but it is designed to work here. In the next example, .mode = .mode can be specified, but if R cannot find .mode for the right hand side, it will just skip with no error. Thus a user can source a script with the following line from batch script where .mode is specified. When running this line without that batch script specification, then this will assign no value to .mode. We include .nodes which shows an example of passing a value that does exist. The non-existent .mode will be returned in the out, but as an unevaluated, captured list element.

.nodes <- 2
out <- setupProject(.mode = .mode,
                    .nodes = .nodes,
                    options = "inst/options.R"
                    )

See Also

setupPaths(), setupOptions(), setupPackages(), setupModules(), setupGitIgnore(). Also, helpful functions such as user(), machine(), node()

vignette("i-getting-started", package = "SpaDES.project")

Examples

## For more examples:
vignette("i-getting-started", package = "SpaDES.project")

library(SpaDES.project)



 ## simplest case; just creates folders
out <- setupProject(
  paths = list(projectPath = ".") #
)

Description

These demonstrate default values for some options that can be set in SpaDES.project. To see defaults, run spadesProjectOptions(). See Details below.

Usage

spadesProjectOptions()

Details

Below are options that can be set with options("spades.xxx" = newValue), where xxx is one of the values below, and newValue is a new value to give the option. Sometimes these options can be placed in the user's .Rprofile file so they persist between sessions.

The following options are used, using the prefix: spades

Value

named list of the default options currently available.

Description

A set of lightweight helpers that are often not strictly necessary, but they make code easier to read.

Usage

user(username = NULL)

machine(machinename = NULL)

node(machinename = NULL)

Arguments

Details

node is an alias for machine

Value

if username is non-NULL, returns a logical indicating whether the current user matches the supplied username. Otherwise returns a character string with the value of the current user.

machine returns a logical indicating whether the current machine name Sys.info()[["nodename"]] is matched by machinename.