Just to start some discussion here... there are a number of places in the new website that use data from various APIs. The results are neat, but it creates a tension: how do we keep the data relatively fresh without overloading the APIs? With a normal server architecture, perhaps we would get the data once a day and cache it, but we're using GitHub Pages, so our options are more limited.
@adriandarian Has hooked up a GitHub Actions system that grabs the latest data and caches it at build time, which is a nice strategy. This means, however, that the data will get progressively out of date; sometimes we go many months between deploys!
It looks like there may be a way to get GitHub Actions to run on a schedule, but even that times out after a while (60 days was the info I saw but I have no verified that).
The code we have also has a fallback to get the data directly on the client side (if the GitHub Actions cache didn't work) and cache it in local storage. I suppose one option would be to switch over to that after the GitHub Actions cache is more than a month old or something.
None of the data we're using the APIs for needs to be super fresh, so it's okay if it's a little out of date.
Or maybe this is a tempest in a teapot... it's possible we don't get enough traffic to be a problem for any of these APIs, and we should just be hitting them normally without trying to do anything to mitigate the calls. We used to have Google Analytics, but it doesn't appear to be working anymore. The Google Search console says we're getting about 2k clicks from Google.
I guess I'm not sure what happens if we hit a rate limit. At the very least we should make sure our code doesn't break in that case.
Anyway, just wanted to have this conversation...
Just to start some discussion here... there are a number of places in the new website that use data from various APIs. The results are neat, but it creates a tension: how do we keep the data relatively fresh without overloading the APIs? With a normal server architecture, perhaps we would get the data once a day and cache it, but we're using GitHub Pages, so our options are more limited.
@adriandarian Has hooked up a GitHub Actions system that grabs the latest data and caches it at build time, which is a nice strategy. This means, however, that the data will get progressively out of date; sometimes we go many months between deploys!
It looks like there may be a way to get GitHub Actions to run on a schedule, but even that times out after a while (60 days was the info I saw but I have no verified that).
The code we have also has a fallback to get the data directly on the client side (if the GitHub Actions cache didn't work) and cache it in local storage. I suppose one option would be to switch over to that after the GitHub Actions cache is more than a month old or something.
None of the data we're using the APIs for needs to be super fresh, so it's okay if it's a little out of date.
Or maybe this is a tempest in a teapot... it's possible we don't get enough traffic to be a problem for any of these APIs, and we should just be hitting them normally without trying to do anything to mitigate the calls. We used to have Google Analytics, but it doesn't appear to be working anymore. The Google Search console says we're getting about 2k clicks from Google.
I guess I'm not sure what happens if we hit a rate limit. At the very least we should make sure our code doesn't break in that case.
Anyway, just wanted to have this conversation...