Fit and validate Neural Networks based on Ensembles of Small of Models approach
Source:R/esm_net.R
esm_net.RdThis function constructs Neural Networks using the Ensembles of Small Models (ESM) approach (Breiner et al., 2015, 2018).
Arguments
- data
data.frame. Database with the response (0,1) and predictors values.
- response
character. Column name with species absence-presence data (0,1)
- predictors
character. Vector with the column names of quantitative predictor variables (i.e. continuous variables). This function only can construct models with continuous variables and does not allow categorical variables. Usage predictors = c("aet", "cwd", "tmin").
- partition
character. Column name with training and validation partition groups.
- thr
character. Threshold used to get binary suitability values (i.e. 0,1). It is useful for threshold-dependent performance metrics. It is possible to use more than one threshold type. It is necessary to provide a vector for this argument. The following threshold criteria are available:
equal_sens_spec: Threshold at which the sensitivity and specificity are equal.
max_sens_spec: Threshold at which the sum of the sensitivity and specificity is the highest (aka threshold that maximizes the TSS).
max_jaccard: The threshold at which the Jaccard index is the highest.
max_sorensen: The threshold at which the Sorensen index is highest.
max_fpb: The threshold at which FPB (F-measure on presence-background data) is highest.
sensitivity: Threshold based on a specified sensitivity value. Usage thr = c('sensitivity', sens='0.6') or thr = c('sensitivity'). 'sens' refers to sensitivity value. If a sensitivity values is not specified, the default used is 0.9
If the user wants to include more than one threshold type, it is necessary concatenate threshold types, e.g., thr=c('max_sens_spec', 'max_jaccard'), or thr=c('max_sens_spec', 'sensitivity', sens='0.8'), or thr=c('max_sens_spec', 'sensitivity'). Function will use all thresholds if no threshold is specified
- size
numeric. Number of units in the hidden layer. Can be zero if there are skip-layer units. Default 2.
- decay
numeric. Parameter for weight decay. Default 0.
Value
A list object with:
esm_model: A list with "nnet" class object from nnet package for each bivariate model. This object can be used for predicting ensemble of small model with
sdm_predictfunction.predictors: A tibble with variables use for modeling.
performance: Performance metric (see
sdm_eval). Those threshold dependent metric are calculated based on the threshold specified in thr argument.performance_part: Performance metric for each replica and partition (see
sdm_eval).
Details
This method consists of creating bivariate models with all the pair-wise combinations of predictors and perform an ensemble based on the average of suitability weighted by Somers' D metric (D = 2 x (AUC -0.5)). ESM is recommended for modeling species with few occurrences. This function does not allow categorical variables because the use of these types of variables could be problematic when using with few occurrences. Further detail see Breiner et al. (2015, 2018).
References
Breiner, F. T., Guisan, A., Bergamini, A., & Nobis, M. P. (2015). Overcoming limitations of modelling rare species by using ensembles of small models. Methods in Ecology and Evolution, 6(10), 1210-218. https://doi.org/10.1111/2041-210X.12403
Breiner, F. T., Nobis, M. P., Bergamini, A., & Guisan, A. (2018). Optimizing ensembles of small models for predicting the distribution of species with few occurrences. Methods in Ecology and Evolution, 9(4), 802-808. https://doi.org/10.1111/2041-210X.12957
Examples
# \donttest{
data("abies")
require(dplyr)
# Using k-fold partition method
set.seed(10)
abies2 <- abies %>%
na.omit() %>%
group_by(pr_ab) %>%
dplyr::slice_sample(n = 10) %>%
group_by()
abies2 <- part_random(
data = abies2,
pr_ab = "pr_ab",
method = c(method = "rep_kfold", folds = 3, replicates = 5)
)
abies2
#> # A tibble: 20 × 18
#> id pr_ab x y aet cwd tmin ppt_djf ppt_jja pH awc
#> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 12040 0 -308909. 384248. 573. 332. 4.84 521. 48.8 5.63 0.108
#> 2 10361 0 -254286. 417158. 260. 469. 2.93 151. 15.1 6.20 0.0950
#> 3 9402 0 -286979. 386206. 587. 376. 6.45 333. 15.7 5.5 0.160
#> 4 9815 0 -291849. 445595. 443. 455. 4.39 332. 19.1 6 0.0700
#> 5 10524 0 -256658. 184438. 355. 568. 5.87 303. 10.6 5.20 0.0800
#> 6 8860 0 121343. -164170. 354. 733. 3.97 182. 9.83 0 0
#> 7 6431 0 107903. -122968. 461. 578. 4.87 161. 7.66 5.90 0.0900
#> 8 11730 0 -333903. 431238. 561. 364. 6.73 387. 25.2 5.80 0.130
#> 9 808 0 -150163. 357180. 339. 564. 2.64 220. 15.3 6.40 0.100
#> 10 11054 0 -293663. 340981. 477. 396. 3.89 332. 26.4 4.60 0.0634
#> 11 2960 1 -49273. 181752. 512. 275. 0.920 319. 17.3 5.92 0.0900
#> 12 3065 1 126907. -198892. 322. 544. 0.700 203. 10.6 5.60 0.110
#> 13 5527 1 116751. -181089. 261. 537. 0.363 178. 7.43 0 0
#> 14 4035 1 -31777. 115940. 394. 440. 2.07 298. 11.2 6.01 0.0769
#> 15 4081 1 -5158. 90159. 301. 502. 0.703 203. 14.6 6.11 0.0633
#> 16 3087 1 102151. -143976. 299. 425. -2.08 205. 13.4 3.88 0.110
#> 17 3495 1 -19586. 89803. 438. 419. 2.13 189. 15.2 6.19 0.0959
#> 18 4441 1 49405. -60502. 362. 582. 2.42 218. 7.84 5.64 0.0786
#> 19 301 1 -132516. 270845. 367. 196. -2.56 422. 26.3 6.70 0.0300
#> 20 3162 1 59905. -53634. 319. 626. 1.99 212. 4.50 4.51 0.0396
#> # ℹ 7 more variables: depth <dbl>, landform <fct>, .part1 <int>, .part2 <int>,
#> # .part3 <int>, .part4 <int>, .part5 <int>
# Without threshold specification and with kfold
esm_net_t1 <- esm_net(
data = abies2,
response = "pr_ab",
predictors = c("aet", "cwd", "tmin", "ppt_djf", "ppt_jja", "pH", "awc", "depth"),
partition = ".part",
thr = NULL
)
#>
|
| | 0%
|
|== | 4%
|
|===== | 7%
|
|======== | 11%
|
|========== | 14%
|
|============ | 18%
|
|=============== | 21%
|
|================== | 25%
|
|==================== | 29%
|
|====================== | 32%
|
|========================= | 36%
|
|============================ | 39%
|
|============================== | 43%
|
|================================ | 46%
|
|=================================== | 50%
|
|====================================== | 54%
|
|======================================== | 57%
|
|========================================== | 61%
|
|============================================= | 64%
|
|================================================ | 68%
|
|================================================== | 71%
|
|==================================================== | 75%
|
|======================================================= | 79%
|
|========================================================== | 82%
|
|============================================================ | 86%
|
|============================================================== | 89%
|
|================================================================= | 93%
|
|==================================================================== | 96%
|
|======================================================================| 100%
esm_net_t1$esm_model # bivariate model
#> $`0.855555555555556`
#> a 2-2-1 network with 9 weights
#> inputs: aet tmin
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.0629629629629629`
#> a 2-2-1 network with 9 weights
#> inputs: aet ppt_jja
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.175`
#> a 2-2-1 network with 9 weights
#> inputs: aet pH
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.190740740740741`
#> a 2-2-1 network with 9 weights
#> inputs: aet awc
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.322222222222222`
#> a 2-2-1 network with 9 weights
#> inputs: aet depth
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.610185185185185`
#> a 2-2-1 network with 9 weights
#> inputs: cwd tmin
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.0703703703703704`
#> a 2-2-1 network with 9 weights
#> inputs: cwd ppt_jja
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.0722222222222222`
#> a 2-2-1 network with 9 weights
#> inputs: cwd pH
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.00694444444444442`
#> a 2-2-1 network with 9 weights
#> inputs: cwd awc
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.374537037037037`
#> a 2-2-1 network with 9 weights
#> inputs: cwd depth
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.468981481481481`
#> a 2-2-1 network with 9 weights
#> inputs: tmin ppt_djf
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.791203703703704`
#> a 2-2-1 network with 9 weights
#> inputs: tmin ppt_jja
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.946296296296296`
#> a 2-2-1 network with 9 weights
#> inputs: tmin pH
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`1`
#> a 2-2-1 network with 9 weights
#> inputs: tmin awc
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.363425925925926`
#> a 2-2-1 network with 9 weights
#> inputs: tmin depth
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.0967592592592592`
#> a 2-2-1 network with 9 weights
#> inputs: ppt_djf ppt_jja
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.0157407407407408`
#> a 2-2-1 network with 9 weights
#> inputs: ppt_djf awc
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.519444444444444`
#> a 2-2-1 network with 9 weights
#> inputs: ppt_djf depth
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.0300925925925926`
#> a 2-2-1 network with 9 weights
#> inputs: ppt_jja pH
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.369907407407407`
#> a 2-2-1 network with 9 weights
#> inputs: ppt_jja depth
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.23287037037037`
#> a 2-2-1 network with 9 weights
#> inputs: pH depth
#> output(s): pr_ab
#> options were - entropy fitting
#>
#> $`0.209722222222222`
#> a 2-2-1 network with 9 weights
#> inputs: awc depth
#> output(s): pr_ab
#> options were - entropy fitting
#>
esm_net_t1$predictors
#> # A tibble: 1 × 8
#> c1 c2 c3 c4 c5 c6 c7 c8
#> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 aet cwd tmin ppt_djf ppt_jja pH awc depth
esm_net_t1$performance
#> # A tibble: 7 × 33
#> model threshold thr_value n_presences n_absences TPR_mean TPR_sd TNR_mean
#> <chr> <chr> <dbl> <int> <int> <dbl> <dbl> <dbl>
#> 1 esm_net equal_sens_… 0.536 10 10 1 0 1
#> 2 esm_net lpt 0.496 10 10 1 0 1
#> 3 esm_net max_fpb 0.574 10 10 1 0 1
#> 4 esm_net max_jaccard 0.574 10 10 1 0 1
#> 5 esm_net max_sens_sp… 0.574 10 10 1 0 1
#> 6 esm_net max_sorensen 0.574 10 10 1 0 1
#> 7 esm_net sensitivity 0.593 10 10 1 0 1
#> # ℹ 25 more variables: TNR_sd <dbl>, W_TPR_TNR_mean <dbl>, W_TPR_TNR_sd <dbl>,
#> # SORENSEN_mean <dbl>, SORENSEN_sd <dbl>, JACCARD_mean <dbl>,
#> # JACCARD_sd <dbl>, FPB_mean <dbl>, FPB_sd <dbl>, OR_mean <dbl>, OR_sd <dbl>,
#> # TSS_mean <dbl>, TSS_sd <dbl>, KAPPA_mean <dbl>, KAPPA_sd <dbl>,
#> # MCC_mean <dbl>, MCC_sd <dbl>, AUC_mean <dbl>, AUC_sd <dbl>,
#> # BOYCE_mean <dbl>, BOYCE_sd <dbl>, CRPS_mean <dbl>, CRPS_sd <dbl>,
#> # IMAE_mean <dbl>, IMAE_sd <dbl>
esm_net_t1$performance_part
#> # A tibble: 105 × 21
#> model replicates part threshold thr_value n_presences n_absences TPR TNR
#> <chr> <chr> <chr> <chr> <dbl> <int> <int> <dbl> <dbl>
#> 1 esm_… .part1 1 max_sore… 0.496 4 4 1 1
#> 2 esm_… .part1 1 max_jacc… 0.496 4 4 1 1
#> 3 esm_… .part1 1 max_fpb 0.496 4 4 1 1
#> 4 esm_… .part1 1 max_sens… 0.496 4 4 1 1
#> 5 esm_… .part1 1 equal_se… 0.496 4 4 1 1
#> 6 esm_… .part1 1 lpt 0.496 4 4 1 1
#> 7 esm_… .part1 1 sensitiv… 0.496 4 4 1 1
#> 8 esm_… .part1 2 max_sore… 0.681 3 3 1 1
#> 9 esm_… .part1 2 max_jacc… 0.681 3 3 1 1
#> 10 esm_… .part1 2 max_fpb 0.681 3 3 1 1
#> # ℹ 95 more rows
#> # ℹ 12 more variables: W_TPR_TNR <dbl>, SORENSEN <dbl>, JACCARD <dbl>,
#> # FPB <dbl>, OR <dbl>, TSS <dbl>, KAPPA <dbl>, MCC <dbl>, AUC <dbl>,
#> # BOYCE <dbl>, CRPS <dbl>, IMAE <dbl>
# }