The benign log is the one that matters

Every detection rule I have ever written passed its first test. That is not a boast; it is an indictment of the test.

You write a rule to catch SSH brute force. You write a log file with forty failed logins from one IP. You run it. It fires. Green tick, commit, move on.

What you have proven is that a rule you wrote to match a thing matches that thing. This is close to a tautology. What you have not proven, and what nobody asked you to prove, is that it stays silent when a backup script fails authentication forty times because someone rotated a key, or when a load balancer health check hammers a closed port, or when a developer fat-fingers their password on a Monday morning and their client retries with exponential enthusiasm.

That second category is where detection packs go to die. Not because they miss attacks, but because they cry wolf until an analyst writes a suppression rule, and then a real one goes past inside the suppression.

So when I built the Wazuh NIST CSF 2.0 rule pack (50 rules, framework-tagged), I made the benign corpus non-optional.

#Two files per rule

Every rule in the pack ships with two synthetic log files:

• trigger.log: traffic the rule must match
• benign.log: traffic the rule must not match

The second file is the interesting one to write, because writing it well requires you to argue against yourself. You have to sit down and think: what is the most attack-shaped legitimate thing that could happen here? Then you put that in the file and dare your own rule to overreact.

For the sudo escalation rule, the benign corpus is a sysadmin doing entirely normal sudo things. For the "execution from /tmp" rule, it is a package manager doing exactly what package managers do, which is execute things out of temporary directories all day long. For "outbound connection to rare port," it is a legitimate service on a high port that happens to be unusual and happens to be fine.

Half of the rules got narrower as a direct result of writing their benign file. That is the mechanism working.

Current state: 57 PASS, 0 FAIL, run two ways: a static validator that needs no Wazuh at all, and a live harness that pushes the same corpus through wazuh-logtest against a real manager on Ubuntu 22.04 and Windows Server 2022.

#Putting the compliance mapping inside the alert

The second problem this pack exists to solve is duller and, if you have ever been near an audit, considerably more painful.

Wazuh detects plenty. Proving to an auditor that a given alert corresponds to a given NIST CSF 2.0 subcategory is a separate job, done months later, in a spreadsheet, from memory. The detection and the evidence live in different systems and start drifting apart the moment either one changes.

The fix is not clever. It is just: put the mapping in the alert.

{
  "rule": {
    "level": 10,
    "description": "SSH brute force attack detected from a single source IP",
    "id": "100018",
    "mitre": {
      "id": [ "T1110.001" ],
      "tactic": [ "Credential Access" ],
      "technique": [ "Brute Force: Password Guessing" ]
    },
    "groups": [
      "authentication_failures", "brute_force",
      "nist_de.cm-01", "nist_rs.ma-02"
    ]
  }
}

That alert arrives already knowing it is DE.CM-01 and RS.MA-02, already carrying T1110.001, already audit-ready. Nobody reconstructs the mapping afterwards, because it was never a separate artefact in the first place.

The discipline that makes this worth anything is refusing to map where the evidence is thin. It is very tempting, when you have 50 rules and five NIST functions, to sprinkle subcategories around until the heatmap looks impressive. The distribution I actually ended up with is lopsided and honest:

Function	Subcategories	Rules
DE Detect	DE.AE-02, DE.AE-03, DE.CM-01, DE.CM-03, DE.CM-09	36
PR Protect	PR.AA-01, PR.AA-05, PR.DS-01	11
ID Identify	ID.RA-03	1
GV Govern	GV.PO-01	1
RC Recover	RC.RP-01	1

Thirty-six of fifty rules land in Detect. Of course they do. It is a SIEM rule pack. A log rule cannot meaningfully implement a governance policy, and claiming otherwise is how compliance mapping became a thing people roll their eyes at. The gaps stay gaps.

#One rule, one file

A smaller decision that has paid for itself repeatedly: every rule lives in its own file, filed under its primary NIST function.

The alternative, big XML files with a dozen related rules and chains of if_sid dependencies, is more elegant right up until someone needs to disable exactly one detection because it is noisy in their environment. Then it is a bomb-disposal exercise. With one rule per file you delete a file. You can fork a single rule, tune it, and never think about the other 49.

Detection packs are not read; they are edited, usually at 2am by someone who did not write them and is not enjoying themselves. Optimise for that person.

#The uncomfortable part

The pack does not work out of the box, and I want to say that plainly.

A large share of these 50 rules never fire unless the endpoints are actually emitting the telemetry they depend on: Sysmon and the right audit policy on Windows, auditd and iptables logging on Linux. Deploy the rules onto a fleet without that groundwork and you get a dashboard full of nothing, which reads exactly like a quiet network.

That prerequisite step is the one everyone skips, including me, for about a week. Detection is a claim about what you can see. If the telemetry is not there, the rule is not a detection. It is a wish.

---

The Wazuh NIST CSF 2.0 rule pack is Apache 2.0, community-maintained, and not an official Wazuh project. The full evidence report and both test harnesses are in the repo. Check my working.