pvnet_app

Internal OCF application to run PVNet models and (optionally) PVNet summation models for the UK in a live environment. This involves accessing weather data stored in AWS S3 buckets, processing and loading this data using either our ocf-data-sampler, pulling pre-trained models from HuggingFace and then producing solar PV power forecasts for the UK by feeding the processed weather data into the model.

The app supports multiple model versions being deployed to live environments and these can be ran with specific configurations which are set via environment variables.

Environment Variables

The app is configured at runtime with many environmental variables. See settings.py.

ML Models Used for PVNet

Model Name	Uses satellite	Uses UKV	Uses ECMWF	Uses cloudcasting	PVNet Hugging Face Link	PVNet Summation Hugging Face Link
pvnet_v2	yes	yes	yes	-	HF Link	Summation HF Link
pvnet_intra_allbells30	yes	yes	yes	-	HF Link	Summation HF Link
pvnet_ecmwf	-	-	yes	-	HF Link	Summation HF Link
pvnet_sat_only	yes	-	-	-	HF Link	Summation HF Link
pvnet_ukv_only	-	yes	-	-	HF Link	Summation HF Link
pvnet_day_ahead	-	yes	yes	-	HF Link	Summation HF Link
pvnet_cloud	yes	yes	yes	yes	HF Link	Summation HF Link

Validation Checks

We run a number of validation checks on the input data and on the forecasts that are produced.

Before feeding data into the model(s) we check whether the available data is compatible with what each model expects.

Satellite data

We check:

• Whether 5-minute and/or 15-minute satellite data is available.
• If more than 5% of the satellite data in the required area is NaN. If so, the satellite data is treated as completely missing.
• Whether there are any missing timestamps in the satellite data. Gaps of up to 15 minutes are linearly interpolated.
• After interpolation, the most recent missing timestamps are extended forward with NaNs up to the delay the model allows.
• Whether the exact timestamps the model expects are all available after infilling.

NWP data

We check:

• If the NWP data contains any NaNs — if so, that NWP source is treated as missing.
• Whether the exact timestamps the model expects from each NWP source are available.

ML batch checks

Just before the batch goes into the ML models, we check that:

• The NWP data for each provider is not entirely zeros. We raise an error if, for any NWP provider, all the NWP data is zero.

Forecast checks

After the ML models have run, we check the national forecast only:

• It does not exceed 100% of the national capacity. The forecast fails validation if any value is above this.
• It does not exceed an absolute ceiling of 20 GW. The forecast fails validation if any value is above this.
• It does not zig-zag (go up, then down, then up). Swings above 500* MW fail validation; swings above 250* MW produce a warning. *configurable - see setting.py
• Forecast values are positive when the sun is up (above the configured elevation).

Development

Running tests locally

To be able to run the tests locally it is recommended to use conda and uv. The easiest way to to create the required environment and run the pytest is to run make test. This will create a new environment in your working directory under .venv.

Running the app locally

It is possbile to run the app locally by setting the required environment variables listed in settings.py, these should point to the relevant data sources and the data platform instance for the environment you want to run the app in. You will need to make sure the data platform at DATA_PLATFORM_HOST:DATA_PLATFORM_PORT is reachable, as well as authenticating against any cloud providers where data may be stored (e.g if using AWS S3 then can do this via the AWS CLI command aws configure).

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Dubraska Solórzano 💻	James Fulton 💻	Megawattz 💻	Peter Dudfield 💻	DivyamAgg24 💻	Aryan Bhosale 💻	Felix 💻
Aditya Sawant 💻	Sukhil Patel 💻	Ali Rashid 💻	Mahmoud Abdulmawlaa 💻	Meghana Sancheti 💻	Dheeraj Mukirala 📖	utsav-pal 💻 📖
RAM KUMAR R 💻	Ripudaman Singh 💻

This project follows the all-contributors specification. Contributions of any kind welcome!