Skip to content

Add NetFlow OCSF mapping package - #169

Merged
mavam merged 8 commits into
mainfrom
tnz-870
Aug 6, 2026
Merged

Add NetFlow OCSF mapping package#169
mavam merged 8 commits into
mainfrom
tnz-870

Conversation

@mavam

@mavam mavam commented Aug 1, 2026

Copy link
Copy Markdown
Member

🔍 Problem

read_netflow produces typed NetFlow v5, NetFlow v9, and IPFIX records, but the Library has no reusable path for normalizing them to OCSF.

🛠️ Solution

  • Add the netflow package and netflow::ocsf::map.
  • Map flow records to OCSF 1.9.0 Network Activity.
  • Represent an unknown connection initiator with initiator_id: 0.
  • Keep flow observation bounds and duration within traffic.
  • Preserve options records and unmapped source fields.
  • Cover real NetFlow and IPFIX captures plus synthetic mapping edge cases.

💬 Review

Please focus on the OCSF semantics for initiators, counters, timestamps, NAT, and options records.

All ten tests pass against the local engine's 1.9.0-dev schema. With the shipping metadata.version: 1.9.0, Tenzir v6.9.0-rc.1 drops events in ocsf::derive because it does not bundle the stable schema yet. Keep this PR blocked until stable OCSF 1.9.0 support lands. Do not ship development metadata.

✅ Closes TNZ-870
📎 Related: tenzir/mono#40

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bfcfce687b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread netflow/operators/ocsf/events/network_activity.tql Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c36afc442

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread netflow/operators/ocsf/events/network_activity.tql Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 191a44878f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread netflow/operators/ocsf/events/network_activity.tql
@mavam
mavam force-pushed the tnz-870 branch 2 times, most recently from 2e60f67 to 1a62feb Compare August 2, 2026 05:13

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a62febe82

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread netflow/operators/ocsf/events/network_activity.tql Outdated
@mavam
mavam force-pushed the tnz-870 branch 3 times, most recently from 4b2fd96 to cd854e2 Compare August 2, 2026 06:37

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd854e2f11

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread netflow/operators/ocsf/events/network_activity.tql Outdated
Comment thread netflow/operators/ocsf/events/network_activity.tql Outdated
Comment thread netflow/operators/ocsf/events/network_activity.tql Outdated
@mavam
mavam force-pushed the tnz-870 branch 3 times, most recently from 5be7dee to b9bd643 Compare August 2, 2026 06:48
Map decoded NetFlow v5, NetFlow v9, and IPFIX records to OCSF Network Activity while preserving uncertain direction, options records, and unmapped source fields.

Assisted-by: gpt-5.6-sol (pi 0.83.0)
mavam added 3 commits August 3, 2026 09:51
Prefer precise absolute timestamps and derive relative uptime
timestamps and durations across 32-bit counter wraps. Select endpoint
addresses by the declared IP version, keep exporter-local interface
IDs unmapped, and avoid guessing an IP version without evidence.

Document why IPFIX flow direction and Community ID do not map
directly, and cover the corrected edge cases.

Assisted-by: gpt-5.6-sol (pi 0.83.0)
Rely on null-propagating timestamp arithmetic while retaining the
non-negative duration check and uptime-based fallback.

Assisted-by: OpenAI gpt-5.6-sol (pi 0.83.0)
Replace the deprecated endpoint-role flag with initiator_id and keep flow timing in the traffic observation window.

Assisted-by: openai-codex/gpt-5.6-sol (pi)
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

mavam added 2 commits August 4, 2026 14:14
Write directly to final OCSF fields, replace statement branches with expressions, and allow absent timing details to remain null.

Assisted-by: openai-codex/gpt-5.6-sol (pi)
Document the otherwise opaque 32-bit millisecond counter constant.

Assisted-by: openai-codex/gpt-5.6-sol (pi)
@tenzir tenzir deleted a comment from chatgpt-codex-connector Bot Aug 4, 2026
@tenzir tenzir deleted a comment from chatgpt-codex-connector Bot Aug 4, 2026
@tenzir tenzir deleted a comment from chatgpt-codex-connector Bot Aug 4, 2026
Rely on null propagation and map the difference between flow bounds directly.

Assisted-by: openai-codex/gpt-5.6-sol (pi)
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Rely on null propagation for optional exporter uptime fields and IP-version inference.

Assisted-by: openai-codex/gpt-5.6-terra (pi)
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@mavam
mavam merged commit 67dee52 into main Aug 6, 2026
3 of 6 checks passed
@mavam
mavam deleted the tnz-870 branch August 6, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant