Security checks

Every replay also looks at the pages it visits for common security weaknesses and lists them in the report, each with a severity, what it means for you (impact), and how to fix it.

What this is, and is not

These are passive checks. StepTrails only looks at the traffic your replay already produced (headers, cookies and addresses). It sends no extra requests and attacks nothing, so it is safe to run against any site you can test. It finds a useful subset of common problems. It is not a penetration test and cannot prove a site is secure.

See the findings

Run a replay as usual. Findings appear:

• in the terminal, one line per finding after the steps;
• in the HTML report, in a Security findings table (severity, finding, where, impact, how to fix);
• in the JUnit report, as notes (they do not fail the tests);
• in the Jira bug, as a short list;
• in the --json output, under security.

node bin/replay.js steptrails-recording.json --html report.html

Fail a build on serious findings

Add --fail-on with the lowest severity that should fail the run:

node bin/replay.js steptrails-recording.json --fail-on high

The command exits with an error if any finding is at or above that level (critical, high, medium, low or info). To turn the checks off, add --no-security.

Policy file: tune it to your project

Put a file called steptrails.security.json in the folder you run StepTrails from (or pass --policy my-policy.json). It is optional: without it the defaults above apply. Every mistake in the file is explained and stops the run before a browser opens.

{
  "version": 1,
  "failOn": "high",
  "checks": {
    "server-version": "off",
    "missing-csp": "high",
    "missing-referrer-policy": { "enabled": false }
  },
  "accepted": [
    {
      "check": "missing-nosniff",
      "reason": "Old admin tool, being replaced in March",
      "expires": "2027-03-31",
      "page": "{base}/legacy/*"
    }
  ],
  "baseline": "steptrails.security-baseline.json"
}

Setting	What it does
failOn	The level that fails a run (the command line --fail-on wins if you give both)
checks	Switch a check "off", or give it another severity for this project (for example a missing CSP is "high" for a bank and "low" for a demo)
accepted	Risks you have decided to live with. Each needs a reason and an expires date, so they are looked at again. page limits it to pages matching a pattern (* matches anything). Once the date passes the risk counts again and the run says so
baseline	A file of known findings, see below

Baseline: only fail on what is new

A big site rarely fixes everything at once. Save today's findings as a baseline, and from then on only new findings, or known ones on new pages, count:

node bin/replay.js recording.json --baseline-write steptrails.security-baseline.json
node bin/replay.js recording.json --baseline steptrails.security-baseline.json --fail-on medium

The baseline holds only the name of each check and a generalised page such as {base}/orders/:id, never a host, so it is safe to commit. Writing a baseline never fails the run. The report says how many findings were known, accepted or switched off, so nothing is hidden.

Check that a fix worked

Save a report before the fix, then compare after it:

node bin/replay.js recording.json --json > before.json
# fix the site
node bin/replay.js recording.json --compare before.json

The output lists each finding as FIXED (gone), STILL (present in both) or NEW, worst first. The same list is in --json as securityComparison. Findings are matched by check and page, so a different host (staging against production) does not matter.

Keep a fix fixed: security tests

node bin/replay.js recording.json --security-tests security.spec.js
npx playwright test security.spec.js

For findings about response headers (Content-Security-Policy, HSTS, framing, nosniff, Referrer-Policy, Permissions-Policy, server version), StepTrails writes a Playwright test per page that fails while the weakness exists and passes once it is fixed. Add the file to your build so the weakness cannot return. Set STEPTRAILS_BASE_URL to test another environment. Findings that are not headers (cookies, libraries, exposed data) are listed as not covered.

Reports for other tools: SARIF

--sarif findings.sarif writes the findings in SARIF, the standard format that GitHub code scanning and many other tools read. Each result has its rule, level, a 0 to 10 severity for ranking, a stable fingerprint, the CWE and OWASP tags, and the page as a {base} pattern. With a baseline, results are marked new or unchanged; accepted risks are included as suppressed, with your reason. Addresses are reduced to patterns, so the file does not reveal which environment you tested.

Every finding also carries its CWE number and OWASP Top 10 (2021) category, shown in the terminal and the HTML report.

Severity levels

Level	Meaning
Critical	A confirmed, directly exploitable problem. Reserved for a future optional active scan; the passive checks never report it.
High	Someone on the network or on another site can read or change what users send or see.
Medium	A missing defence against a common attack.
Low	A hardening gap.
Info	A best-practice suggestion.

What is checked

Finding	Level	Why it matters
Page served over plain HTTP	High	Anyone on the network can read or change the page, including passwords.
Insecure (HTTP) resource on an HTTPS page	Medium	An attacker can replace it, for example a script.
HSTS header missing	Medium	Browsers may still try plain HTTP first.
HSTS lifetime short	Low	The protection expires quickly.
Content-Security-Policy missing	Medium	One cross-site scripting bug lets an attacker run code in users' sessions.
Content-Security-Policy allows unsafe scripts	Low	Inline scripts, eval or any source weaken the policy.
Page can be framed by other sites (clickjacking)	Medium	Another site can trick users into clicking hidden buttons.
X-Content-Type-Options missing	Low	Browsers may run a file as a script by guessing its type.
Referrer-Policy missing	Info	Full addresses may leak to other sites.
Permissions-Policy missing	Info	Embedded content can ask for camera, location and so on.
Server reveals software and version	Low	Helps attackers find known flaws.
Cookie without Secure, HttpOnly or SameSite	Low (Medium for session-like cookies)	Cookies can be stolen or sent from other sites.
SameSite=None without Secure	Medium	Browsers reject or downgrade the cookie.
CORS allows any origin with credentials	Medium	Other sites may read private responses.
CORS allows any origin	Info	Fine for public data, wrong for private data.
Anything OWASP ZAP reports (with --zap)	From ZAP	Findings from a passive scan of your journey, each tied to its step. See Scan with ZAP.
JavaScript library with known vulnerabilities	From the data (usually Medium)	An attacker can use a published flaw without finding a bug in your code. See Library checks.
Secrets, card numbers, tokens in addresses or storage, trackers before interaction	Low to Critical	See Data exposure checks.

Pages on localhost, 127.0.0.1 and *.localhost skip the checks that only make sense on the public internet (HTTPS, HSTS and the Secure cookie flag).

Each finding includes

• Severity and a plain-language impact.
• How to fix it, with a copy-paste example such as the exact header line, and examples for common servers where they differ.
• Evidence (the header value or cookie name) and the pages where it was seen. The same weakness on many pages is one finding.

Coming later

An optional deeper scan (OWASP ZAP) that will only run against sites you own or have permission to test. See What's coming.