Spine correctness, hardening and poll_host decomposition

.github/actions/build-spine/action.yml

@@ -0,0 +1,60 @@
+name: Build Spine
+description: Install the build dependencies, then bootstrap and configure Spine.
+
+inputs:
+  extra-packages:
+    description: Additional apt packages, beyond the set every Spine build needs.
+    required: false
+    default: ''
+  configure-args:
+    description: Arguments passed through to ./configure.
+    required: false
+    default: ''
+  cflags:
+    description: CFLAGS for ./configure.
+    required: false
+    default: ''
+  ldflags:
+    description: LDFLAGS for ./configure.
+    required: false
+    default: ''
+
+runs:
+  using: composite
+  steps:
+    # mysql-server is deliberately absent: configure only looks for mysql.h and
+    # libmysqlclient, both of which come from libmysqlclient-dev, and no job
+    # starts or connects to a server.
+    - name: Install build dependencies
+      shell: bash
+      env:
+        EXTRA_PACKAGES: ${{ inputs.extra-packages }}
+      run: |
+        set -euo pipefail
+        sudo apt-get update
+        # shellcheck disable=SC2086
+        sudo apt-get install -y \
+          autoconf automake libtool build-essential help2man dos2unix \
+          libmysqlclient-dev libsnmp-dev libssl-dev libcmocka-dev \
+          $EXTRA_PACKAGES
+
+    - name: Bootstrap and configure
+      shell: bash
+      env:
+        CONFIGURE_ARGS: ${{ inputs.configure-args }}
+        IN_CFLAGS: ${{ inputs.cflags }}
+        IN_LDFLAGS: ${{ inputs.ldflags }}
+      run: |
+        set -euo pipefail
+        ./bootstrap
+        args=()
+        if [ -n "$CONFIGURE_ARGS" ]; then
+          read -ra args <<< "$CONFIGURE_ARGS"
+        fi
+        if [ -n "$IN_CFLAGS" ]; then
+          args+=("CFLAGS=$IN_CFLAGS")
+        fi
+        if [ -n "$IN_LDFLAGS" ]; then
+          args+=("LDFLAGS=$IN_LDFLAGS")
+        fi
+        ./configure ${args[@]+"${args[@]}"}

.github/workflows/ci.yml

@@ -24,82 +24,73 @@ jobs:
     env:
       CC: ${{ matrix.compiler }}
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - name: Install build dependencies
-        run: |
-          sudo apt-get update
-          sudo apt-get install -y \
-          mysql-server libmysqlclient-dev \
-          libsnmp-dev libssl-dev build-essential \
-          help2man autoconf automake libtool dos2unix libcmocka-dev
-
-      - name: Prepare for Spine Build
-        run: |
-          ./bootstrap
-          ./configure --enable-warnings
+      - uses: ./.github/actions/build-spine
+        with:
+          configure-args: --enable-warnings
 
       - name: Build Spine
         run: |
+          set -euo pipefail
           make -j"$(nproc)"
 
       - name: Run the unit tests
         run: |
           set -euo pipefail
           make check
 
-#  cppcheck:
-#    runs-on: ubuntu-latest
-#    steps:
-#      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
-#
-#      - name: Install cppcheck
-#        run: |
-#          sudo apt-get update 
-#          sudo apt-get install -y cppcheck build-essential
-#
-#      - name: Run cppcheck
-#        run: |
-#          cppcheck \
-#            --enable=all \
-#            --std=c11 \
-#            --error-exitcode=1 \
-#            --suppress=missingIncludeSystem \
-#            --suppress=unusedFunction \
-#            --suppress=checkersReport \
-#            --suppress=variableScope \
-#            --suppress=unreadVariable \
-#            --suppress=shadowVariable \
-#            --suppress=constVariablePointer \
-#            --suppress=redundantAssignment \
-#            --suppress=toomanyconfigs \
-#            *.c *.h
+      # test_ping_icmp needs a raw ICMP socket, which the runner user cannot
+      # open, so under `make check` above it skips and reports nothing. Running
+      # it with privilege is the only way those cases actually execute, and
+      # ping_icmp() is a SUID-root path, so a silent skip is the wrong default.
+      - name: Run the privileged ping tests
+        run: |
+          set -euo pipefail
+          sudo ./tests/unit/test_ping_icmp
+          echo "raw ICMP cases executed rather than skipped"
+
+  net-snmp-no-md5:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      # Simulate an MD5-disabled/FIPS net-snmp header. The explicit SHA-1
+      # fallback must configure and link without SNMP_DEFAULT_AUTH_PROTO.
+      - uses: ./.github/actions/build-spine
+        with:
+          configure-args: CPPFLAGS=-DNETSNMP_DISABLE_MD5
+
+      - name: Build without net-snmp MD5 auth
+        run: |
+          set -euo pipefail
+          make -j"$(nproc)"
+          make check
 
   # Coverage is reported, not gated.  The number is only meaningful once
   # something measures it; the threshold conversation comes after that.
   coverage:
     runs-on: ubuntu-latest
     timeout-minutes: 20
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - name: Install build dependencies
-        run: |
-          set -euo pipefail
-          sudo apt-get update
-          sudo apt-get install -y mysql-server libmysqlclient-dev libsnmp-dev \
-            libssl-dev build-essential help2man autoconf automake libtool \
-            dos2unix libcmocka-dev gcovr
+      - uses: ./.github/actions/build-spine
+        with:
+          extra-packages: gcovr
+          cflags: -g -O0 --coverage
+          ldflags: --coverage
 
       - name: Build instrumented
         run: |
           set -euo pipefail
-          ./bootstrap
-          ./configure CFLAGS="-g -O0 --coverage" LDFLAGS="--coverage"
           make -j"$(nproc)"
 
       - name: Run the unit tests
-        run: make check
+        run: |
+          set -euo pipefail
+          make check
 
       - name: Report coverage
         run: |
@@ -113,9 +104,9 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 20
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
         with:
           python-version: "3.x"
 
@@ -127,71 +118,66 @@ jobs:
       # informational so we have a baseline to chip away at.
       - name: Run flawfinder
         run: |
+          set -euo pipefail
           flawfinder \
             --minlevel=3 \
             --error-level=5 \
             --columns \
             --context \
             . | tee flawfinder-report.txt
 
-      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         if: always()
         with:
           name: flawfinder-report
           path: flawfinder-report.txt
 
   # Spine is a threaded network daemon; ASan and UBSan are the checks most
-  # likely to catch a real defect here.
+  # likely to catch a real defect here.  The unit tests are what actually
+  # exercises the code, so they are what runs instrumented.
   sanitizers:
     runs-on: ubuntu-latest
     timeout-minutes: 20
+    env:
+      CC: clang
+      ASAN_OPTIONS: detect_leaks=1:exitcode=1
+      UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - name: Install build dependencies
-        run: |
-          set -euo pipefail
-          sudo apt-get update
-          sudo apt-get install -y clang mysql-server libmysqlclient-dev libsnmp-dev \
-            libssl-dev build-essential help2man autoconf automake libtool dos2unix libcmocka-dev
+      - uses: ./.github/actions/build-spine
+        with:
+          extra-packages: clang
+          cflags: -fsanitize=address,undefined -fno-omit-frame-pointer -g -O1
+          ldflags: -fsanitize=address,undefined
 
       - name: Build with ASan and UBSan
-        env:
-          CC: clang
         run: |
           set -euo pipefail
-          ./bootstrap
-          ./configure \
-            CFLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -g -O1" \
-            LDFLAGS="-fsanitize=address,undefined"
           make -j"$(nproc)"
 
-      - name: Run the binary under the sanitizers
-        env:
-          ASAN_OPTIONS: detect_leaks=1:exitcode=1
-          UBSAN_OPTIONS: print_stacktrace=1:halt_on_error=1
+      - name: Run the unit tests under the sanitizers
+        run: |
+          set -euo pipefail
+          make check
+
+      - name: Smoke-test the binary under the sanitizers
         run: |
           set -euo pipefail
           ./spine --version
           ./spine --help > /dev/null
+
   # A release tarball that cannot be compiled is a release blocker, and the
   # only way to catch it is to build from the tarball the way a packager does.
   distcheck:
     runs-on: ubuntu-latest
     timeout-minutes: 20
     steps:
-      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - name: Install build dependencies
-        run: |
-          set -euo pipefail
-          sudo apt-get update
-          sudo apt-get install -y mysql-server libmysqlclient-dev libsnmp-dev \
-            libssl-dev build-essential help2man autoconf automake libtool dos2unix libcmocka-dev
+      - uses: ./.github/actions/build-spine
 
       - name: make distcheck
         run: |
           set -euo pipefail
-          ./bootstrap
-          ./configure
           make distcheck

.github/workflows/fuzz.yml

@@ -20,20 +20,11 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
-      - name: Install build dependencies
-        run: |
-          set -euo pipefail
-          sudo apt-get update
-          sudo apt-get install -y clang libmysqlclient-dev libsnmp-dev libssl-dev \
-            autoconf automake libtool help2man pkg-config
-
-      - name: Generate config.h
-        run: |
-          set -euo pipefail
-          ./bootstrap
-          ./configure
+      - uses: ./.github/actions/build-spine
+        with:
+          extra-packages: clang pkg-config
 
       # A pull request gets a short regression run against the committed corpus.
       # The weekly schedule runs long enough to explore.
@@ -47,7 +38,7 @@ jobs:
           fi
           make -C tests/fuzz FUZZ_SECONDS="$seconds" run
 
-      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
+      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         if: failure()
         with:
           name: fuzz-crashes

.github/workflows/integration.yml

@@ -18,15 +18,18 @@ jobs:
     runs-on: ubuntu-latest
     timeout-minutes: 20
     steps:
-      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
 
       # Brings up MariaDB and an snmpd, then polls a real device through
       # poll_host() and checks the value lands in poller_output.
       - name: Poll an SNMPv3 device end to end
-        run: tests/snmpv3/scripts/run-integration.sh
+        run: |
+          set -euo pipefail
+          tests/snmpv3/scripts/run-integration.sh
 
       - name: Show container logs on failure
         if: failure()
         run: |
+          set -euo pipefail
           cd tests/snmpv3
           docker compose logs --no-color || true

.gitignore

@@ -31,3 +31,21 @@ libtool
 *.o
 *.m4
 tests/unit/bin/
+
+# automake test-suite artifacts
+*.trs
+test-suite.log
+
+# coverage instrumentation from --enable-coverage
+*.gcda
+*.gcno
+*.gcov
+
+# make dist output
+spine-poller-*.tar.gz
+
+# cmocka test binaries and automake stamps
+tests/unit/test_*
+!tests/unit/test_*.c
+.dirstamp
+tests/unit/.dirstamp

CHANGELOG

@@ -26,6 +26,25 @@ The Cacti Group | spine
 -issue#552: Terminate die() output so consecutive fatal messages no longer run together
 -issue#561: Reserve room for the terminator in php_readpipe() so a full script server result cannot write past result_string
 -issue#562: Escalate PHP script server shutdown to SIGKILL after a bounded grace period so a stuck child is not orphaned
+-issue#564: Check the calloc() results in spine.c so an allocation failure dies instead of dereferencing NULL
+-issue#565: Bound the newline appended by spine_log() so a full log line cannot write past flogmessage
+-issue#566: Free the result set on the NULL-row branch of the util.c settings helpers
+-issue#567: Build the log timestamp format once at config load instead of rebuilding it on every log line
+-issue#600: Wipe the local SNMPv3 passphrase copies in snmp_host_init() before freeing them
+-issue#605: Refuse a script server descriptor at or past FD_SETSIZE instead of writing outside fd_set
+-issue#606: Allocate the undefined-value string in php_cmd() only once the retry budget is spent
+-issue#607: Catch SIGPIPE rather than treating it as fatal, while preserving default SIGPIPE behavior in executed scripts
+-issue: Compile output_regex patterns as POSIX extended regular expressions so REGEX_NUMBER works as intended, and apply configured patterns consistently to SNMP, script, and PHP script-server results; script and PHP script-server items previously ignored this field, and a match against multipart output now selects one substring rather than preserving every name:value pair
+-upgrade: output_regex now uses POSIX ERE syntax; escape literal parentheses and plus signs, and review patterns written for the former BRE behavior. Spine's built-in numeric fallback will also begin extracting the first decimal from responses it previously rejected as U (for example, 10.0.0.1 becomes 10.0, an OID such as 1.3.6.1 becomes 1.3, and a version such as 2.6.32 becomes 2.6), so review affected data sources before upgrading
+-issue: Give scripts the configured timeout to exit naturally, then escalate through SIGTERM and SIGKILL during nft_pclose
+-issue: Clear the destination in db_escape() when the column is NULL
+-issue: Zero the details array so an early poller exit cannot free uninitialised pointers
+-issue: Terminate error_string so the host_errors insert cannot read uninitialised heap
+-issue: Match last_snmp_community to the width of the field it is compared against
+-issue: Require MySQL 8.0.20 before emitting the row alias upsert form
+-issue#608: Bound the script server restart so a server that starts but never answers cannot recurse
+-issue#610: Detach the subprocess entry under ListMutex so nft_pclose() owns the child it reaps
+-issue#611: Leave hostinfo untouched when getaddrinfo() fails in init_sockaddr()
 -issue: Correct signed and unsigned printf format specifiers in poller.c, free session.localname on the unknown-version return in snmp.c, and quote shell variables in the build scripts
 -issue: Escape the SNMP result and RRD name before the poller_output INSERT, bound the buffer_output_errors write to the space left in error_string, and validate the --hostlist argument before it reaches SQL
 -issue: Restore the twelve headers and spine.conf.dist missing from the dist tarball so a release tarball can be compiled from