Contributing to FIMS
The advantage of having a modular modeling system like FIMS is that its development can also be broken down and tackled in component parts. By utilizing github’s collaborative structure, while one developer can focus on improving recruitment estimates in FIMS, others can tackle, for example, age composition likelihoods or diagnostic tools.
In this way, scientists at different centers can focus on adding those features that are critically necessary to make the assessments in their regions compatible with the FIMS platform. Our small team chipped off a little piece of age-based stock assessment models that was important for our two regions despite being on complete opposite sides of the country, at the Southwest Fisheries Science Center and the Northeast Fisheries Science Center.
The default selectivity option in FIMS is logistic or double logistic, but the double normal or age-specific selectivity are used in stock synthesis (SS) and the Woods Hole Assessment Model (WHAM) respectively. We set out to add these options and remove one more barrier to replicating regional assessments in FIMS.
Modifying the FIMS Workflow
Though daunting at first, we realized we could add these selectivity options by copying the existing scaffolding from previous selectivity options or even look to features like the recruitment or abundance specification to figure out indexing, calculations, and how variables are passed around the FIMS framework. Implementing new cpp architecture was the most tricky, since neither of us had a strong background in it.
Before any modifications were made, we branched the github repository into a development branch with an informative name- dev-selectivity-at-age and made all changes there.
The crux of how selectivity is specified and used in FIMS came down to modifying or adding a few bits of code, described below:
src/fims_modules.hpp This code lists all the FIMS modules. We added “SelectivityatAge” as a module, with corresponding variable names
inst/include/common/fims_math.hpp This code contains all of the “fims math” or other formulas and math functions not found in base R functionality. We added new functions to help conversion of Type&x to size_t for indexing purposes in age_specific.hpp
inst/include/interface/rcpp/rcpp_interface.hpp This is the RCPP interface “to declare things”. We copied the code used for the other selectivity options (i.e., “LogisticSelectivityInterface” and “DoubleLogisticSelectivityInterface”) and used it to declare AgeSpecificSelectivityInterface
inst/include/interface/rcpp/rcpp_objects/rcpp_selectivity.hpp This code is an RCPP interface to declare objects, specifically different types of selectivity. There was already code for “LogisticSelectivityInterface” and “DoubleLogisticSelectivityInterface” so it was trivial to copy this architecture wholesale for “AgeSpecificSelectivityInterface”
inst/include/population_dynamics/selectivity/selectivity.hpp This code explicitly defines any downstream “functors” (see below) and makes sure toexplicitly include them for use in FIMS. We added a single line “#include”functors/age_specific.hpp”
inst/include/population_dynamics/selectivity/functors/age_specific.hpp This was a brand new bit of code we created, again mimicking the functors for other selectivity options like logistic and double logistic. Here’s where the actual “math” all gets applied, the correct indexing figured out for the age-specific values, and the inverse logit applied to the parameters.
R/FIMS-package.R This code lists all objects that are “imported” and “exported” in FIMS and we modified to include ‘@export AgeSpecificSelectivity’ and ran devtools::document() to add function to exported functions
Default Parameters
All of the above made it explicitly possible to use an age-based selectivity to fit assessment models in FIMS. But for accessibility and to make sure the model does run (rather than simply ‘can’ run), it was critical to specify default parameters when this selectivity is chosen by a user.
R/create_default_parameters.R This bit of code contains several functions necessary for broadly and specifically specifying default parameters in FIMS. The nested functions are as follows:
- create_default_parameters() -> create_default_fleet() -> create_default_selectivity() -> create_default_AgeSpecific
And at the center of these matryoshka dolls is a function that uses the template for setting default parameters to specify the name, type, label, ages, that these are fixed effects and estimated on the logistic scale (i.e., constrained between 0 and 1), using a qlogis() function.
create_default_AgeSpecific <- function(
module_name = NA_character_,
data
) {
default <- create_default_parameters_template(n_parameters = get_n_ages(data)) |>
dplyr::mutate(
module_name = !!module_name,
module_type = "AgeSpecific",
label = "logit_sel_at_age",
age = get_ages(data),
value = qlogis(1/(1+(exp(-1*(get_ages(data)-2))))),
estimation_type = "fixed_effects"
)
}Vignettes
Once the functionality and default parameters were set, it was time to add a vignette that demonstrates the selectivity-at-age function working in action. We saved a copy of the existing vignettes/fims-demo.Rmd as fims-demo-age-spec-sel.Rmd and changed the updated configurations section to:
updated_configurations <- default_configurations_unnested |>
dplyr::rows_update(
y = tibble::tibble(
module_name = c("Selectivity"),
fleet_name = c("survey1"),
module_type = c("AgeSpecific")
),
by = c("module_name", "fleet_name")
)by changing the module_type from the previous selectivity option. Further down in the code we changed the specified starting parameters as well:
dplyr::rows_update(
tibble::tibble(
fleet_name = "survey1",
label = "logit_sel_at_age",
age = seq(get_n_ages(data_4_model)),
value = c(0, 0, 2.999999, 4.999993, 6.999946,
8.999956, 11.000085, 12.982599, 15.019483, 18.420681,
18.420681, 18.420681),
estimation_type = c(rep("fixed_effects", 2), rep("constant", 10))
),
by = c("fleet_name", "label", "age")
)This meant only the first two ages were estimated as fixed effects and the rest of the age-specific selectivity was fixed at or near 1. From previous experience it is often best to fix one or several selectivity-at-age values close to 1, otherwise the selectivity and fishing mortality parameters can become confounded.
Tests
Once the actual functionality was in place, default parameters are set, and a vignette is made, it was time to guarantee compatibility across all other areas of FIMS both now and in the future by writing in some checks and tests. There are two main categories of tests in FIMS, the gtests and testthat. The former focuses on cpp code and the latter on R code. In both, we added similar tests, 1) that the input and output get passed around effectively, 2) that the inverse-logit and logit transformations handled values correctly, and 3) that FIMS could handle “edge-cases” or those extreme values where things run up against their bounds.
tests/gtest/CMAkeLists.txt. We just needed to modify one thing- adding test names for AgeSpecificSelectivity to ensure these tests are called during compilation
tests/gtest/test_AgeSpecificSelectivity_Evaluate.cpp Tests performance of evaluate calls with multiple ages and multiple time steps, to show inverse-logit transformations and indexing perform as expected. It was easy to set using the built in FIMS:::use_gtest_template()
tests/testthat/test-rcpp-selectivity.R - This code is generic to all selectivity options, unlike the gtest above. But we added sections specific to the AgeSpecificSelectivity option to check that the estimation type and evaluate() works correctly:
# Create selectivity2
selectivity2 <- methods::new(AgeSpecificSelectivity)
selectivity2$logit_sel_at_age$resize(1)
selectivity2$logit_sel_at_age[1]$value <- 1
selectivity2$logit_sel_at_age[1]$estimation_type$set("random_effects")
selectivity2$ages$resize(1)
selectivity2$ages$set(0, 1)
#' @description Test that `get_id()` for `AgeSpecificSelectivity` works when a second object is created.
expect_equal(selectivity2$get_id(), 2)
#' @description Test that the `logit_sel_at_age` value is set to 1.
expect_equal(selectivity2$logit_sel_at_age[1]$value, 1.0)
#' @description Test that the `logit_sel_at_age` estimation type is set to "random_effects".
expect_equal(selectivity2$logit_sel_at_age[1]$estimation_type$get(), "random_effects")
#' @description Test that `evaluate()` works for `AgeSpecificSelectivity` with "random_effects".
expect_equal(
selectivity2$evaluate(1),
# Line below equals 0.2716494
1.0 / (1.0 + exp(-1.0)), # inverse logit equation
tolerance = 0.0000001
)Essentially we specified values, ran the evaluation, and used a workhorse function in FIMS testing - expect_equal() - to see if we got back the result we expected.
Roadblocks/Lessons Learned
🚧 Roadblock #1: Flexible indexing and age-specification options
Unlike for other selectivity options which include a few parameters that establish a function between age and selectivity, the selectivity-at-age option uses a flexible number of parameters, specified by the user. At a maximum there can be one parameter for every age and at a minimum there can be no parameters (all values fixed at their input). There was no precedent for this setting and the other selectivity options wouldn’t help us resolve it. Ultimately, we asked for individual sessions with FIMS leadership to explain the issue and ask for solutions. They recommended including n_ages, the number of ages, and min_age the minimum age when declaring the AgeSpecificSelectivity class.
🚧 Roadblock #2: Naming conflicts with other selectivity options
Only when we got to building out the vignette to demo the age-specific selectivity option did we notice a problem with our naming convention that would conflict with how FIMS builds out names and uses features. We’d been calling the age-specific option “SelectivityatAge” but when the create_default_parameters() function is run, this name gets knitted together with “Selectivity” creating the monstrous “SelectivityatAgeSelectivity” which FIMS can’t understand downstream.
Our early fix was ugly. We manually put in an if statement to catch when the incorrect name shows up and manually changes it back to something comprehensible. But once we showed the issue and our band-aid solution to the FIMS code club, a weekly meeting from 1:00-5:00 EST or 10:00-2:00 PST that is reserved for debugging and discussion, we were about to crowd source a permanent solution - going back through the FIMS structure and renamed everything that was “selectivity at age” to “age specific selectivity” to fix this overarching issue.
Takeaways
Don’t let contributing to FIMS intimidate you! Nobody understands all the nuanced ins and outs of FIMS except a few core developers and given the amount of features FIMS needs to meet its 25% by 2026 (25% of NOAA stocks to be capable of using FIMS to fit the model and generate management advice), more than that core group needs to be actively contributing. Take on small tasks at first - a new version of an existing feature, like these selectivity options. But maximize efficiency by identifying a feature that is important to several stocks. Starting small will help build confidence and familiarity with FIMS’s modular nature so you might be comfortable with contributing bigger changes down the line. Hope this post helps to identify some key starting spots and illustrates how modifications are really just a matter of taking, copying, and ajusting existing code. Please let us know if you have questions or we can offer advice, at alexander.jensen[at]noaa.gov or emily.liljestrand[at]noaa.gov.