-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
feat(templates): survey variables valiation on backend #4108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -48,6 +48,17 @@ func (c *TaskController) AddTask(w http.ResponseWriter, r *http.Request) { | |
| return | ||
| } | ||
|
|
||
| // By default a task may carry only the variables declared in the template | ||
| // survey, no matter who sends it: task variables override the template | ||
| // environment and are passed to the app with the highest precedence, so | ||
| // undeclared keys change settings the template author never exposed — see | ||
| // db.Task.ValidateSurveyVars. Templates whose tasks legitimately need | ||
| // arbitrary variables opt in with AllowAnyVarsInTask. | ||
| if err = taskObj.ValidateSurveyVars(tpl); err != nil { | ||
|
Comment on lines
+51
to
+57
|
||
| helpers.WriteError(w, err) | ||
| return | ||
| } | ||
|
|
||
|
Comment on lines
+51
to
+61
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Find other places that build/queue db.Task and check whether they call ValidateSurveyVars
rg -n --type=go 'AddTask\(' --glob '!*_test.go'
rg -n --type=go 'ValidateSurveyVars' --glob '!*_test.go'Repository: semaphoreui/semaphore Length of output: 159 🏁 Script executed: #!/bin/bash
set +e
printf 'tracked files:\n'
git ls-files | sed -n '1,120p'
printf '\nFind tasks.go:\n'
fd -a 'tasks\.go$' . 2>/dev/null || true
printf '\nSearch AddTask ValidateSurveyVars (no glob):\n'
rg -n 'AddTask\(|ValidateSurveyVars' -g '*.go' . || true
printf '\nSearch Schedule/Integration/Workflow task creation terms:\n'
rg -n 'schedule|Schedule|integration|Integration|workflow|Workflow|Environment:|Env:|TaskId|Task' -g '*.go' . | sed -n '1,240p'Repository: semaphoreui/semaphore Length of output: 25986 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '--- api/projects/tasks.go ---\n'
cat -n api/projects/tasks.go | sed -n '1,130p'
printf '\n--- services/tasks/TaskPool.go AddTask section ---\n'
cat -n services/tasks/TaskPool.go | sed -n '960,1055p'
printf '\n--- services/schedules/SchedulePool.go task creation section ---\n'
cat -n services/schedules/SchedulePool.go | sed -n '130,190p'
printf '\n--- api/integration.go task creation section ---\n'
cat -n api/integration.go | sed -n '320,380p'
printf '\n--- Pro workflow interfaces and call sites ---\n'
cat -n pro_interfaces/workflow_svc.go | sed -n '1,80p'
rg -n --type=go 'AddTask\(' services api pro* -g '*.go' || true
rg -n --type=go 'ValidateSurveyVars' services api db -g '*.go' || trueRepository: semaphoreui/semaphore Length of output: 250 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- api/projects/tasks.go ---'
cat -n api/projects/tasks.go | sed -n '1,130p'
printf '%s\n' ''
printf '%s\n' '--- services/tasks/TaskPool.go AddTask section ---'
cat -n services/tasks/TaskPool.go | sed -n '960,1055p'
printf '%s\n' ''
printf '%s\n' '--- services/schedules/SchedulePool.go task creation section ---'
cat -n services/schedules/SchedulePool.go | sed -n '130,190p'
printf '%s\n' ''
printf '%s\n' '--- api/integration.go task creation section ---'
cat -n api/integration.go | sed -n '320,380p'
printf '%s\n' ''
printf '%s\n' '--- Pro workflow interfaces and call sites ---'
cat -n pro_interfaces/workflow_svc.go | sed -n '1,80p'
rg -n --type=go 'AddTask\(' services api pro* -g '*.go' || true
rg -n --type=go 'ValidateSurveyVars' services api db -g '*.go' || trueRepository: semaphoreui/semaphore Length of output: 18511 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- db/Task.go relevant validations ---'
cat -n db/Task.go | sed -n '200,295p'
printf '%s\n' ''
printf '%s\n' '--- db/TaskParams and schedule task creation ---'
cat -n db/TaskParams.go | sed -n '1,220p'
cat -n db/Schedule.go | sed -n '1,140p'
printf '%s\n' ''
printf '%s\n' '--- api/integration.go GetTaskDefinition ---'
cat -n api/integration.go | sed -n '240,320p'
printf '%s\n' ''
printf '%s\n' '--- TaskPool validation remainder ---'
cat -n services/tasks/TaskPool.go | sed -n '1028,1080p'
printf '%s\n' ''
printf '%s\n' '--- validateSurveyVars implementation/test cases ---'
cat -n db/Task.go | sed -n '220,360p'
cat -n db/Task_test.go | sed -n '1,170p'Repository: semaphoreui/semaphore Length of output: 21739 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- db/Task.go ValidateNewTask ---'
rg -n -A80 -B10 'func \(task \*Task\) ValidateNewTask' db/Task.go
cat -n db/Task.go | sed -n '140,205p'
printf '%s\n' ''
printf '%s\n' '--- api integration task creation after GetTaskDefinition ---'
cat -n api/integration.go | sed -n '286,350p'
printf '%s\n' ''
printf '%s\n' '--- all AddTask call sites with enough context ---'
for f in api/projects/tasks.go services/schedules/SchedulePool.go api/integration.go; do
printf '%s\n' 'FILE: '"$f"
rg -n -C3 'AddTask\(' "$f" --glob '*.go' || true
doneRepository: semaphoreui/semaphore Length of output: 8366 Enforce survey-var validation at the task-pool entry point.
🤖 Prompt for AI Agents |
||
| newTask, err := taskPool(r).AddTask( | ||
| taskObj, | ||
| &user.ID, | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,7 @@ import ( | |
| "strings" | ||
| "time" | ||
|
|
||
| "github.com/semaphoreui/semaphore/pkg/common_errors" | ||
| "github.com/semaphoreui/semaphore/pkg/git" | ||
| "github.com/semaphoreui/semaphore/pkg/tz" | ||
|
|
||
|
|
@@ -197,6 +198,89 @@ func (task *Task) ValidateNewTask(template Template) error { | |
| return task.ExtractParams(params) | ||
| } | ||
|
|
||
| // parseTaskVars parses a task-supplied variable payload (Task.Environment or | ||
| // Task.Secret) into a map. Numbers are decoded as json.Number so integer survey | ||
| // variables can be checked without float rounding. | ||
| func parseTaskVars(payload string, field string) (map[string]any, error) { | ||
| res := make(map[string]any) | ||
|
|
||
| if payload == "" { | ||
| return res, nil | ||
| } | ||
|
|
||
| dec := json.NewDecoder(strings.NewReader(payload)) | ||
| dec.UseNumber() | ||
|
|
||
| if err := dec.Decode(&res); err != nil { | ||
| return nil, common_errors.NewValidationError("task " + field + " must be a JSON object") | ||
| } | ||
|
|
||
| return res, nil | ||
|
Comment on lines
+211
to
+218
|
||
| } | ||
|
Comment on lines
+201
to
+219
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win Reject trailing data in
🤖 Prompt for AI Agents |
||
|
|
||
| // ValidateSurveyVars ensures that the variables supplied with the task are | ||
| // declared in the template survey and match their declared type. | ||
| // | ||
| // Task variables are merged over the template environment and reach the app as | ||
| // extra variables (--extra-vars for Ansible, -var for Terraform apps, arguments | ||
| // for shell apps), where they take precedence over everything the template | ||
| // author configured. Undeclared keys therefore let the requester change | ||
| // settings the template never exposed — for Ansible that includes connection | ||
| // options such as ansible_ssh_common_args, whose ProxyCommand runs on the | ||
| // Semaphore server itself. Nobody may send them by default, which is also all | ||
| // the UI ever sends; a template opts in to arbitrary variables with | ||
| // AllowAnyVarsInTask. | ||
| func (task *Task) ValidateSurveyVars(template Template) error { | ||
| if template.AllowAnyVarsInTask { | ||
| return nil | ||
| } | ||
|
Comment on lines
+234
to
+236
|
||
|
|
||
| env, err := parseTaskVars(task.Environment, "environment") | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| secrets, err := parseTaskVars(task.Secret, "secret") | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| for name, value := range env { | ||
| v := template.GetSurveyVar(name) | ||
|
|
||
| if v == nil { | ||
| return undeclaredSurveyVarError(name) | ||
| } | ||
|
|
||
| // Secret variables belong to the secret payload, which is stored | ||
| // encrypted and masked in logs; accepting them here would persist the | ||
| // value in plaintext on the task row. | ||
| if v.Type == SurveyVarSecret { | ||
| return common_errors.NewValidationError( | ||
| "survey variable " + name + " must be sent in the task secret") | ||
| } | ||
|
|
||
| if err = v.ValidateValue(value); err != nil { | ||
| return err | ||
| } | ||
| } | ||
|
|
||
| for name := range secrets { | ||
| v := template.GetSurveyVar(name) | ||
|
|
||
| if v == nil || v.Type != SurveyVarSecret { | ||
| return undeclaredSurveyVarError(name) | ||
| } | ||
| } | ||
|
Comment on lines
+268
to
+274
|
||
|
|
||
| return nil | ||
| } | ||
|
|
||
| func undeclaredSurveyVarError(name string) error { | ||
| return common_errors.NewValidationError( | ||
| "variable " + name + " is not declared in the template survey") | ||
| } | ||
|
|
||
| func (task *TaskWithTpl) Fill(d Store) error { | ||
| if task.BuildTaskID != nil { | ||
| build, err := d.GetTask(task.ProjectID, *task.BuildTaskID) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| package db | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/assert" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func surveyTemplate() Template { | ||
| return Template{ | ||
| SurveyVars: []SurveyVar{ | ||
| {Name: "app_version", Type: SurveyVarStr}, | ||
| {Name: "replicas", Type: SurveyVarInt}, | ||
| {Name: "stage", Type: SurveyVarEnum, Values: []SurveyVarEnumValue{ | ||
| {Name: "Dev", Value: "dev"}, | ||
| {Name: "Prod", Value: "prod"}, | ||
| }}, | ||
| {Name: "notes", Type: SurveyVarText}, | ||
| {Name: "token", Type: SurveyVarSecret}, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| func TestTask_ValidateSurveyVars(t *testing.T) { | ||
| tests := []struct { | ||
| name string | ||
| environment string | ||
| secret string | ||
| errContains string | ||
| }{ | ||
| {name: "no variables"}, | ||
| {name: "declared variables", environment: `{"app_version":"1.2.3","replicas":3,"stage":"prod","notes":"hi"}`}, | ||
| {name: "declared secret", secret: `{"token":"s3cret"}`}, | ||
| {name: "integer as string", environment: `{"replicas":"3"}`}, | ||
| {name: "empty value of typed var", environment: `{"replicas":"","stage":""}`}, | ||
| {name: "null value of typed var", environment: `{"replicas":null,"stage":null}`}, | ||
|
|
||
| { | ||
| name: "undeclared variable", | ||
| environment: `{"ansible_ssh_common_args":"-o ProxyCommand=/bin/sh"}`, | ||
| errContains: "ansible_ssh_common_args is not declared", | ||
| }, | ||
| { | ||
| name: "undeclared variable next to a declared one", | ||
| environment: `{"app_version":"1.2.3","ansible_connection":"local"}`, | ||
| errContains: "ansible_connection is not declared", | ||
| }, | ||
| { | ||
| name: "undeclared secret", | ||
| secret: `{"ansible_ssh_common_args":"-o ProxyCommand=/bin/sh"}`, | ||
| errContains: "ansible_ssh_common_args is not declared", | ||
| }, | ||
| { | ||
| name: "declared non secret variable sent as secret", | ||
| secret: `{"app_version":"1.2.3"}`, | ||
| errContains: "app_version is not declared", | ||
| }, | ||
| { | ||
| name: "secret variable sent as environment", | ||
| environment: `{"token":"s3cret"}`, | ||
| errContains: "token must be sent in the task secret", | ||
| }, | ||
| { | ||
| name: "non integer value", | ||
| environment: `{"replicas":"3; rm -rf /"}`, | ||
| errContains: "replicas must be an integer", | ||
| }, | ||
| { | ||
| name: "fractional value of integer var", | ||
| environment: `{"replicas":1.5}`, | ||
| errContains: "replicas must be an integer", | ||
| }, | ||
| { | ||
| name: "value outside of enum", | ||
| environment: `{"stage":"staging"}`, | ||
| errContains: "stage has a value which is not allowed", | ||
| }, | ||
| { | ||
| name: "environment is not an object", | ||
| environment: `["app_version"]`, | ||
| errContains: "task environment must be a JSON object", | ||
| }, | ||
| { | ||
| name: "secret is not an object", | ||
| secret: `"token"`, | ||
| errContains: "task secret must be a JSON object", | ||
| }, | ||
| } | ||
|
|
||
| for _, tt := range tests { | ||
| t.Run(tt.name, func(t *testing.T) { | ||
| task := Task{Environment: tt.environment, Secret: tt.secret} | ||
|
|
||
| err := task.ValidateSurveyVars(surveyTemplate()) | ||
|
|
||
| if tt.errContains == "" { | ||
| assert.NoError(t, err) | ||
| return | ||
| } | ||
|
|
||
| require.Error(t, err) | ||
| assert.ErrorContains(t, err, tt.errContains) | ||
| }) | ||
| } | ||
| } | ||
|
|
||
| func TestTask_ValidateSurveyVars_TemplateWithoutSurvey(t *testing.T) { | ||
| task := Task{Environment: `{"anything":"value"}`} | ||
|
|
||
| err := task.ValidateSurveyVars(Template{}) | ||
|
|
||
| assert.ErrorContains(t, err, "anything is not declared") | ||
| } | ||
|
|
||
| func TestTask_ValidateSurveyVars_AllowAnyVarsInTask(t *testing.T) { | ||
| task := Task{ | ||
| Environment: `{"ansible_ssh_common_args":"-o ProxyCommand=/bin/sh","replicas":"many"}`, | ||
| Secret: `{"undeclared":"value"}`, | ||
| } | ||
|
|
||
| tpl := surveyTemplate() | ||
| tpl.AllowAnyVarsInTask = true | ||
|
|
||
| assert.NoError(t, task.ValidateSurveyVars(tpl)) | ||
|
|
||
| // The very same task is rejected while the template does not opt in. | ||
| tpl.AllowAnyVarsInTask = false | ||
| assert.Error(t, task.ValidateSurveyVars(tpl)) | ||
| } | ||
|
|
||
| func TestTemplate_GetSurveyVar(t *testing.T) { | ||
| tpl := surveyTemplate() | ||
|
|
||
| v := tpl.GetSurveyVar("stage") | ||
|
|
||
| require.NotNil(t, v) | ||
| assert.Equal(t, SurveyVarEnum, v.Type) | ||
| assert.Nil(t, tpl.GetSurveyVar("STAGE")) | ||
| assert.Nil(t, tpl.GetSurveyVar("unknown")) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When a task is created by an integration or schedule,
api/integration.go:366andservices/schedules/SchedulePool.go:164callTaskPool.AddTaskdirectly, bypassing this controller-only check. Consequently, a webhook extractor can still place request-controlled, undeclared or type-invalid values into the task environment despite the new default restriction, including the connection options this change is intended to block. Apply the validation in the shared task creation path before the secret is cleared.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@codex fix it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary
TaskPool.AddTaskpath, ensuring API, integration, and scheduled tasks all receive identical validation. Secret variables are validated before the secret payload is cleared. services/tasks/TaskPool.goL1023-L1047helpers.WriteErrorrather than converting them into internal-server errors. api/projects/tasks.goL51-L78a848e9e(fix(tasks): validate survey vars for every task source) and created the follow-up pull request metadata titled “fix(tasks): enforce survey validation across task sources.”Testing
git diff --checkgo test ./db ./services/tasks ./api/projectsgit status --short --branch(working tree clean)View task →