netcap

Netcap (NETwork CAPture) converts network packets into structured, type-safe Protocol Buffer audit records — designed for security monitoring, forensic analysis, and machine learning. A single Go binary with 83 packet decoders, 40+ stream decoders, and 141+ audit record types, backed by a concurrent architecture and a built-in web UI.

Protocol hierarchy visualization in the Netcap web UI — more screenshots

• 83 packet-layer decoders — Ethernet, IPv4/6, TCP, UDP, DNS, DHCP, ARP, TLS ClientHello/ServerHello, ICMP, NTP, SIP, OSPF, BGP, MPLS, GRE, VXLAN, 802.11, and many more
• 40+ stream decoders — TLS, SSH, HTTP/2, QUIC, SMB, FTP, SMTP, POP3, IMAP, IRC, Kerberos, DCERPC, and more
• Industrial protocols — Modbus, S7Comm, DNP3, OPC-UA, PROFINET, BACnet, CIP, IEC 62351
• Full TCP/UDP stream reassembly with configurable limits

Built-in React (Vite + TypeScript) dashboard in service mode with interactive visualizations:

• Sankey diagrams, treemaps, 3D scatter plots, geo maps, host communication graphs
• Record browsing with JSON/UI views and field-level filtering
• Protocol statistics, connection analysis, host profiling, alert management

See the Gallery for screenshots.

• JA4 fingerprinting — BSD-licensed JA4 TLS client fingerprinting; external JA4+ support is available only in opt-in local source builds subject to the FoxIO License
• YARA rules — file scanning with compiled yara-x rules for malware detection
• Magika AI — Google's AI-based file type classification on extracted files
• Credential harvesting — configurable protocol-aware credential capture
• File extraction — extract files from HTTP, FTP, SMTP, POP3, IMAP, SMB, IRC with hashing (MD5, SHA1, SHA256) and MIME detection
• Detection rules — 30+ YAML rule categories covering reconnaissance, exfiltration, web attacks, industrial ports, and more. The expression engine supports source→distinct-destination cardinality (fan-out) detection, an approved-workstation allowlist (IsApprovedWorkstation), and time-of-day helpers (IsBusinessHours, HourOfDay)
• OT/ICS threat hunting — function-code level Siemens S7comm detection (write / logic download / logic theft / PLC stop / CPU restart) mapped to CISA AA26-231A; ships internal/rules/examples/s7comm_hunt.yml

• Protocol Buffers (default) — compact binary, accessible from any language
• CSV — configurable separators for data analysis pipelines
• JSON — human-readable structured output
• Elasticsearch — direct bulk indexing for ELK stack analysis

• DNS reverse resolution
• GeoIP geolocation and ASN: bundled DB-IP Lite with optional user-installed GeoLite2. Select providers using -geoProviders dbip,geolite2 (default order) or the Databases page; see GeoIP configuration.
• MAC vendor lookup
• Deep Packet Inspection (optional, via nDPI/libprotoident)
• Hyperscan / Vectorscan acceleration (optional) — multi-pattern regex prefilter for nmap service probes (~2.2× faster), CMS/web framework detection (~1.4×) and rule-engine MatchesPattern (up to ~6× on miss-heavy detection traffic), see docs/hyperscan.md

• Prometheus + Grafana — real-time metrics and dashboards
• Elasticsearch + Kibana — full-text search and visualization
• Maltego — 45+ OSINT entity types and transforms

Agents stream audit records to a collection server over mutually authenticated TLS 1.3, with pinned keys, acknowledged delivery and reconnects. See docs/distributed-collection.md.

Pre-built binaries are available on the Releases page. To build from source:

# Build (requires libpcap)
go build -o net ./cmd/net/

# Build without DPI (fewer C dependencies)
go build -tags=nodpi -o net ./cmd/net/

# Build with Hyperscan / Vectorscan acceleration for service probes
# (requires libhs via pkg-config; e.g. `brew install vectorscan` on macOS)
# See docs/hyperscan.md for details.
CGO_ENABLED=1 go build -tags hyperscan -o net ./cmd/net/

# Capture from PCAP file
./net capture -read traffic.pcap

# Live capture
sudo ./net capture -iface en0

# Service mode (starts web UI)
./net capture -read traffic.pcap --service

# Service mode with hot reload (development)
air

net capture enables capture-time network detections independently of baseline approval. -network-detection=false disables them; -network-detection-config policy.json loads bounded JSON thresholds, authorized source IPs/CIDRs and versioned indicators (internal/networkdetect/config.go).

Defaults: 60 s windows, 300 s alert deduplication, 10 distinct DNS names/TCP endpoints/large echoes, 5 SMTP hosts and 10 MiB SSH client payload. State retains at most 4,096 keys, 128 values per window and 1,024 flows with 4 KiB prefixes per direction. A stream gap stops signature/transfer claims for that direction and is reported. Indicator validity uses Unix nanoseconds at capture time. A missing status file or zero configured indicators is displayed as unavailable coverage.

internal/networkdetect/testdata/live/ contains 15 real FlightSim CLI captures at revision 3709d1c6905d9527885c62f66f49a955c7b0d191, generated with Docker --network none and local DNS/API/SSH/SFTP/IRC/Stratum fixtures. C2 intelligence is synthetic lab data; SSH qualification uses a 1 MiB transfer and a 512 KiB threshold. testdata/cases.json separately labels synthetic source-shape and benign controls. Scenario names never enter the detector.

go test -race ./internal/networkdetect
go test -race -tags nodpi,noyara,nomagika ./internal/collector -run '^TestFlightSimCollectorReplay$'

The collector checks replay at 1/2/4/8 workers. The lab requires a 32 MiB tcpdump buffer and a 2 s drain; tests/flightsim-lab/run.sh rejects kernel drops and failed simulator modules before sealing capture hashes.

capture -behavior and agent -behavior enable passive, sensor/interface/VLAN- scoped observations. Capture defaults to <output>/Behavior.json; agents use the user-config netcap/behavior/<interface>/ directory. Select an existing baseline explicitly with -behavior-baseline <path>.

./net capture -read traffic.pcap -out capture -behavior -http localhost:8080
./net capture -iface en0 -out live -behavior -behavior-sensor office-router

The synthetic packet-to-SSE gate (TestBehavioralPacketToSSELatency, 100 samples) measured p95 51.85 ms including decode on Darwin ARM64 / Apple M5 Max, Go 1.27.0, at 19.99 synthetic packets/s. TestBehavioralBrowserDelivery measured receipt to React-render p95 99.10 ms in Chrome 154.0.8037.98, with 1,000 approved facts, 100 decisive SYNs over 5.998 s and 58,000 background SYNs at 9,586 packets/s; zero fact/window overflow. Browser approval, reconnect, evidence and suppression history are asserted against the real backend. TestBehavioralCollectorPressure processed 100,000 stable SYNs through 4 protocol workers and audit persistence in 0.406 s (246,310 packets/s), with zero missing observations or state overflow. The browser workload uses synchronous synthetic ingress; the collector workload uses an offline PCAP. Neither measures kernel-capture drops.

Run the opt-in gates after pnpm install --frozen-lockfile && pnpm build in cmd/capture/webui/frontend; the browser gate also requires Google Chrome:

NETCAP_BEHAVIOR_BROWSER=1 go test -race ./cmd/capture/webui -run '^TestBehavioralBrowserDelivery$' -count=1 -timeout 120s
NETCAP_BEHAVIOR_PRESSURE=1 go test ./internal/collector -run '^TestBehavioralCollectorPressure$' -count=1 -timeout 90s

PCAP oracles in behavior_replay_test.go cover discovery/DNS, rates, DHCP transitions, IPv6 conflicts, geography and lateral patterns with equivalent evidence at 1/2/4/8 workers. These scoped results do not establish latency on arbitrary traffic or hardware. Endpoint visibility is limited to traffic at the selected interface; encrypted SSH/RDP connection patterns do not establish authentication failures.

BenchmarkBaselineDurableStorage compares atomic JSON checkpoints with SQLite WAL / synchronous=FULL on the same Mac (100 iterations, 2026-10-06):

Live state remains bounded in memory with portable JSON checkpoints every 1 s; the measured 10,000-fact checkpoint costs 10.987 ms. The SQLite row result is a lower bound for one observation, not a complete lifecycle/state-store port. Alerts remain separately synced per event. TestCorruptBaselineRecoveryFromValidatedTemplate verifies recovery preserves approval and both the damaged file and backup; cmd/agent/behavior_test.go verifies TLS delivery of identical retained alert evidence after a collector outage and local-engine restart.

internal/behavior/testdata/qualification/ pins fixture contract 1: schema-3 approved snapshots, monitoring PCAPs, exact detector/endpoint/evidence oracles and SHA-256 manifests for discovery/DNS, address changes, lateral patterns, rates and geography. The geography case includes synthetic DB-IP MMDBs. NETCAP_BEHAVIOR_EXPORT=<absolute fresh directory> plus NETCAP_BEHAVIOR_REFERENCE=<Go commit> exports the same corpus while running go test ./internal/collector -run 'TestBehavioral.*(Oracle|WorkerCounts)$' -count=1. Export refuses an existing case directory. Decision timestamps are generated at approval; consumers verify the committed bytes against each manifest.

net investigate flows --read Connection.ncap.gz \
  --start-ns "$START_NS" --end-ns "$END_NS" \
  --filter 'InSubnet(SrcIP, "192.0.2.0/24")' --group-by srcIP

net investigate packet-evidence --read exercise.pcapng \
  --bpf 'host 192.0.2.1 and tcp port 80' --out evidence.zip

net investigate collect-flows --listen 127.0.0.1:2055 --out ./fresh-output --duration 1m
net investigate exported-flows --read ./fresh-output/FlowExports.jsonl \
  --exporter 192.0.2.10:50000 --format netflow-v9 --domain 7 \
  --start-ns "$START_NS" --end-ns "$END_NS" --time-basis receive

START_NS and END_NS are inclusive UTC nanosecond integers. The WebUI API exposes the same flow engine at GET /api/flows/query with startNs, endNs, filter, groupBy, sortBy and limit. Connection downloads accept format=evidence, a TCP/UDP protocol, and optional paired time bounds.

flows --window-mode overlap|contained|start|end selects interval semantics. Optional --bucket-ns emits uniformly interpolated byte/rate estimates, capped at 4,096 bins; raw group counters remain whole-observation counts. Reports also include byte shares and nearest-rank size/packet/duration distributions. The Investigation Evidence page exposes both controls and labels estimated values.

Exports refuse replacement and fail on truncation or exceeded limits. Packet exports cap captured packets at 1 MiB and PCAPNG blocks/metadata at 16 MiB; flow queries cap records at 4 MiB and decoded input at 256 MiB. The 38 book-derived workflow families are qualified in testdata/investigation-coverage.json, with exact fixture references and mandatory integration commands. Coverage is scoped to those fixtures and declared limits; external endpoint, proxy and organizational evidence remains explicitly external.

capture --flow-exports normalizes UDP NetFlow v5/v9, IPFIX and sFlow v5 on --flow-export-ports (default 2055,4739,6343,9995,9996). It records raw datagrams, normalized observations and issues in FlowExports.jsonl before worker dispatch. FlowExportsHealth.json binds final health to the record-file SHA-256. Existing flow artifacts are refused; use a fresh output directory.

capture --conns also writes stream-evidence/stream-*/{client.bin,server.bin,manifest.json}. These bytes contain no ANSI markup. The manifest preserves per-direction offsets, capture times, TCP gaps and UDP datagram boundaries. A gapped stream must not be parsed as contiguous data; the legacy display transcript remains separate.

capture --capture-evidence writes capture-manifest.json with input/configuration hashes, ingress/admission/drop counters and nullable kernel statistics. Add --retain-packets --packet-segment-mb 32 --packet-retention-mb 512 for rotating PCAPNG retention. Expired segments remain recorded as tombstones. A fresh output directory is required; retention failure is an error, not successful collection. The Investigation evidence page (/investigation-evidence) displays this coverage, scoped flow reports and directional stream downloads.

net investigate exchange --spec exchange.json executes one bounded TCP/UDP experiment. Byte fields use JSON base64; Framing supports length-prefix, delimiter and fixed. Response variables capture fresh negotiation values. TLS verifies certificates and hostnames; optional rootCAFile and paired clientCertificateFile/clientKeyFile configure experiment trust and mTLS.

net investigate protocol-fields --spec grammar.json --read client.bin --compare other-client.bin validates framed streams against declared byte/UTF-8/signed/unsigned fields and reports changes by frame ordinal and field name. Fields include offsets, raw bytes and hypothesis descriptions. Frame insertions can shift alignment; a successful parse does not prove the hypothesized application semantics.

net investigate tls-capture --read tls.pcapng --key-log secrets.log --stream 0 uses installed tshark for offline TLS dissection. Output binds input, key-log and plaintext hashes to the tool version; key material is not included. Node ordering follows tshark's reported endpoints, not an assumed client role. NETCAP_REQUIRE_TLS_ADAPTER=1 go test ./internal/protocoltest -run TestTLSCaptureAdapter requires the TLS 1.2/1.3 adapter fixtures instead of skipping when tshark is absent.

net investigate protocol-proxy --spec proxy.json --listen 127.0.0.1:9000 handles one framed TCP connection. Mutations select direction/frame and byte range, with optional length-prefix repair, drop, duplication and delay. Output records original/transmitted bytes. Both tools enforce time/frame/byte limits; response matches are protocol observations, not proof of endpoint effects.

Use each command's --help for the versioned JSON/input contract. FTP data extraction waits for control/data readers to drain; FTPDataHealth.json records associations and exclusions. TCPReassemblyHealth.json records checksum policy and rejection counts. Both sidecars are shown in the Investigation Evidence page and retained by Pro projects.

Pre-built images are available for multiple configurations:

See the docker/ directory for all Dockerfiles and build variants.

• Documentation — full usage guide
• Homepage — project homepage
• DeepWiki — AI-powered codebase exploration
• Thesis — original research paper

Contributions welcome — from protocol decoder additions to core framework improvements.

Development Setup:

• macOS Development Setup Guide
• Installation Guide

Please use the bug report template for issue reports.

Netcap is licensed under the GNU General Public License v3, which is a very permissive open source license, that allows others to do almost anything they want with the project, except to distribute closed source versions. This license type was chosen with Netcap's research purpose in mind, and in the hope that it leads to further improvements and new capabilities contributed by other researchers on the long term.