[cryocloud] Add opencode and claude configuration to prod.values.yaml - #8905
Conversation
Added opencode configuration with model details and options.
for more information, see https://pre-commit.ci
|
Merging this PR will trigger the following deployment actions. Support deploymentsNo support upgrades will be triggered Staging deployments
Production deployments
|
Added OPENCODE_CONFIG environment variable and updated claude-code-managed-settings with additional model configurations.
for more information, see https://pre-commit.ci
tsnow03
changed the title
Removed claude-code-managed-settings configuration in lieu of doing this manually for now. Less to deal with on short notice.
for more information, see https://pre-commit.ci
choldgraf
left a comment
choldgraf
left a comment
There was a problem hiding this comment.
This seems safe-ish to merge to me, though I'm hesitant to do so without knowing more about how this hub image works. I've taken the key that @tsnow03 sent me in a DM and encrypted it with sops at singleuser.extraEnv. I've also added some comments to document where the hub image is configured for this hub.
That said, I'm not going to merge this just yet unless @tsnow03 requests that I do so now before waiting for @agoose77 to wake up. I'm pretty sure this exposes the openai API key to all users - maybe that's OK but I'd want @tsnow03 or @agoose77 to say this was safe to try before merging solo
| name: NRP | ||
| options: | ||
| baseURL: https://ellm.nrp-nautilus.io/v1 | ||
| apiKey: '{env:OPENAI_API_KEY}' |
There was a problem hiding this comment.
I'm worried that putting the key to an env variable would make the API visible to all users of a hub (not on github, but in an authenticated user session), which feels like a security risk, but it seems like something we already do for other communities so I'm assuming it's fine - eg JupyterHealth has the same pattern:
https://github.com/search?q=repo%3A2i2c-org%2Finfrastructure+openai&type=code
@tsnow03 should probably know about this though! I think any user of the cryocloud hub will be able to view and potentially exfiltrate this key
There was a problem hiding this comment.
Yes, that is a risk. I'd like to find better ways to handle API keys, but it would likely require us to support something like using a Hub as an OIDC provider (which isn't yet supported).
There was a problem hiding this comment.
Yes, there is a general purpose application that I think will benefit the JupyterHub community that is an auth proxy that allows JupyterHub tokens to be used to access a REST API via a proxy which uses a credential to access the upstream API.
But in the absence of that, these shared credential approaches are okay in that you are sharing a single managed credential with all users. I talked to @tsnow03 about this a bit ago, and it's a question of trust and mitigation: to what extent do you trust the users of this hub with that credential (they can absolutely take this and use NRP LLM resources from their home machine), and mitigating that by rotating this key. An NRP LLM token is not the most precious resource because it is not billed and probably most cryocloud users would have access to it themselves if they bothered to go through the process of asking for an account, so there is little benefit to exfiltration in this particular case. If the resource protected by this token were either not likely to be available to most users of the hub and/or if there was a cost risk associated with abuse
Key rotation is not easily facilitated by storing the key in SOPS and requiring 2i2c folks to set the secret value, which Tasha has to retrieve from NRP, so this is likely to be infrequent, but can be done on a time scale Tasha feels is appropriate for mitigating that risk. For example, the hub this is copied from runs with a relatively small number of reasonably trusted users, but when we use it to host a workshop, for example, I rotate the keys immediately afterward.
| extraFiles: | ||
| # Opencode config for the NRP model provider. | ||
| # opencode itself is installed by the CryoCloud AI image, I think that's here: | ||
| # https://github.com/CryoInTheCloud/image-cryo-python-AI/blob/e80c8f17fbfcac53ddf1cd1c51f515915b31441c/appendix#L26-L29 |
There was a problem hiding this comment.
It's not totally clear to me how users are meant to use the image that has opencode etc - are you just going to ask people to use the dynamic image builder UI to past the link to the repo themselves? We should document where that repo is in the config - I took a stab at this but you should confirm that I'm right
There was a problem hiding this comment.
You would use the Custom image profile option, and paste the Docker image string.
|
Also, adding @minrk for review |
|
This can merge, but there are review comments that @tsnow03 should be aware of :) Thanks for getting to this @choldgraf! |
| name: NRP | ||
| options: | ||
| baseURL: https://ellm.nrp-nautilus.io/v1 | ||
| apiKey: '{env:OPENAI_API_KEY}' |
There was a problem hiding this comment.
Yes, there is a general purpose application that I think will benefit the JupyterHub community that is an auth proxy that allows JupyterHub tokens to be used to access a REST API via a proxy which uses a credential to access the upstream API.
But in the absence of that, these shared credential approaches are okay in that you are sharing a single managed credential with all users. I talked to @tsnow03 about this a bit ago, and it's a question of trust and mitigation: to what extent do you trust the users of this hub with that credential (they can absolutely take this and use NRP LLM resources from their home machine), and mitigating that by rotating this key. An NRP LLM token is not the most precious resource because it is not billed and probably most cryocloud users would have access to it themselves if they bothered to go through the process of asking for an account, so there is little benefit to exfiltration in this particular case. If the resource protected by this token were either not likely to be available to most users of the hub and/or if there was a cost risk associated with abuse
Key rotation is not easily facilitated by storing the key in SOPS and requiring 2i2c folks to set the secret value, which Tasha has to retrieve from NRP, so this is likely to be infrequent, but can be done on a time scale Tasha feels is appropriate for mitigating that risk. For example, the hub this is copied from runs with a relatively small number of reasonably trusted users, but when we use it to host a workshop, for example, I rotate the keys immediately afterward.
I was thinking the same thing - do you know if we're tracking that somewhere? If not then I can write up a little user story for it so we have an issue. I noticed this pattern in a bunch of places in the 2i2c hubs (e.g. accessing earthdata etc) and I suspect it'll be a useful thing to have a layer of security around |
|
Yes I approve! |
|
Thank you so much for all of your work on this @agoose77 @choldgraf @minrk!!! |
|
ok we have like 100 approvals so I'm just going to merge this 😅 |
4 participants
Added opencode and claude configuration with model details and options. I have no idea how to add the one secret in: OPENAI_API_KEY. I provided that key via email.
I copied these for opencode from @minrk, but I think our
.binder/postbuildtakes care of theinstall-opencode.sh:https://github.com/BIDS/hub-deploy/blob/2a0e060f930fe9ff7f2dc0e4a38ed9cb0dd791b4/hubs/demo/config.yaml#L257-L299
https://github.com/BIDS/hub-deploy/blob/2a0e060f930fe9ff7f2dc0e4a38ed9cb0dd791b4/hubs/demo/config.yaml#L229
https://github.com/BIDS/hub-deploy/blob/2a0e060f930fe9ff7f2dc0e4a38ed9cb0dd791b4/images/user/install-opencode.sh#L6
And used claude to wing its own config. Searching the internet, it looks like it should work. I based most of it on what the .json would be to be added manually by users if we don't have this.