make workflow configs parseable outside R and resolve input paths from env - #4082
make workflow configs parseable outside R and resolve input paths from env#4082divine7022 wants to merge 13 commits into
Conversation
| - Only crop codes present in the crosswalk resolve to an N rate envelope. Cycles whose | ||
| code does not resolve are dropped and reported at run time. | ||
|
|
||
| # Running outside the BU cluster |
There was a problem hiding this comment.
Edit this header to something more generic like "Config file".
|
|
||
| # paths in config.yml are relative to ccmmf_dir so the yaml stays plain data any | ||
| # parser can read. a value from the environment wins and is used as given, so a | ||
| # run can point anywhere without editing the file |
There was a problem hiding this comment.
I'd like to check in on the second assumption above -- from a design perspective, why would we prioritize values set from the environment over values within the file. Also, why would doing so somehow make the code more portable, since you just need other code to set all those environment variables to local values, which means that you now have two different files trying to set the same configs (config.yml and the script setting the env variables). This seems redundant, confusing, and error prone. There's also no reason a config file can't have the user define a root directory as an absolute path and then define other paths as relative paths based on that root (or even to detect whether the other path is absolute or relative)
There was a problem hiding this comment.
I had a related discussion with @hdpriest-ui yesterday - setting values in config is preferred over using environment variables for portability and reproducibility.
There was a problem hiding this comment.
But the use of environmental variables is widespread in the code so it isn't necessary or prudent to force refactoring working code in the short term.
Description
Motivation and Context
Review Time Estimate
Types of changes
Checklist: