Reconnaissance

Reconnaissance is the first phase of any attack or security assessment. Before exploiting a system, an attacker gathers as much information as possible about the target: its network layout, exposed services, software versions, and the people who manage it. The more an attacker knows, the more precisely they can strike.

Security professionals perform the same reconnaissance. Understanding what an attacker can discover about your systems lets you reduce your exposure before someone takes advantage of it.

Passive vs. active

Reconnaissance falls into two categories:

Passive reconnaissance collects information without interacting directly with the target. You gather data from public sources: DNS records, WHOIS data, job postings, social media profiles, and archived web pages. The target never sees you.

Active reconnaissance interacts directly with the target. Port scans, banner grabs, and vulnerability probes send packets to the target’s systems. This approach yields richer data but generates logs and may trigger alerts.

What reconnaissance reveals

A thorough reconnaissance phase surfaces:

• Open ports and running services
• Software versions and known vulnerabilities
• Network topology and IP ranges
• Employee names, roles, and email formats
• Domain structure and subdomains
• Technologies in use (web frameworks, CMS, mail servers)

Defenders use this same lens to audit their own attack surface. If you can find it, an attacker can too.

Creating target lists

Generating IP addresses

seq and for loop

Generate consecutive IP addresses with seq:

#!/usr/bin/env bash

# Generate IP addresses from a given range
for ip in $(seq 1 254); do
    echo "172.16.10.${ip}" >> 172-16-10-hosts.txt    
done

echo, sed, and brace expansion

Without sed, echo prints each number on a single line separated by a space. sed replaces that space with a newline character:

echo 10.1.10.{1..254} | sed 's/ /\n/g'

printf

printf doesn’t require piping to sed to replace the space with a newline:

printf "10.1.0.%d\n" {1..254}

Subdomains

You can find a list of subdomains from GitHub gists. Use the following search query:

subdomain wordlist site: gist.github.com

Host discovery

ping

Sends ICMP echo requests to a host to test reachability and measure round-trip time. There are no options to run the command against multiple hosts, so use the following script that reads a list of hosts from a file and prints which ones respond to a ping. Pass the file path as the first argument:

#!/usr/bin/env bash

FILE="${1}"

while read -r host; do
	if ping -c 1 -W 1 -w 1 "${host}" &> /dev/null; then 
		echo "${host} is up"
	fi
done < "${FILE}"

The while loop drives the scan:

• while read -r host; do: reads one line from standard input and assigns it to host. The -r flag prevents backslash interpretation. The loop runs once per line until the file is exhausted.
• if ping -c 1 -W 1 -w 1 "${host}" &> /dev/null; then: pings the host once (-c 1), waits up to one second for a reply (-W 1), and exits after one second regardless of result (-w 1). All output goes to /dev/null. If ping exits with code 0, the host responded and the if block runs.
• echo "${host} is up": prints the host to stdout.
• done < "${FILE}": closes the loop and redirects the file as standard input, feeding one line at a time to read.

Nmap

Nmap is a network scanner used for host discovery, port scanning, and service detection. The -sn flag disables port scanning so Nmap only checks whether hosts are up. This is called a ping sweep.

nmap -sn 172.16.10.0/24
Starting Nmap 7.99 ( https://nmap.org ) at 2026-05-16 06:42 -0400
Nmap scan report for 172.16.10.10
Host is up (0.000030s latency).
MAC Address: EA:21:3B:D3:BE:CD (Unknown)
Nmap scan report for 172.16.10.11
Host is up (0.000034s latency).
MAC Address: 1A:94:51:D8:E2:EE (Unknown)
Nmap scan report for 172.16.10.12
Host is up (0.000072s latency).
MAC Address: 86:5E:F8:1D:71:1B (Unknown)
Nmap scan report for 172.16.10.13
Host is up (0.000056s latency).
MAC Address: CA:3B:EE:2B:B8:5D (Unknown)
Nmap scan report for 172.16.10.1
Host is up.
Nmap done: 256 IP addresses (5 hosts up) scanned in 3.04 seconds

The command scans all 256 addresses in the 172.16.10.0/24 subnet. The -sn flag tells Nmap to skip port scanning and send only host discovery probes: ICMP echo requests, a TCP SYN to port 443, a TCP ACK to port 80, and an ICMP timestamp request.

Each block in the output represents a live host:

• Nmap scan report for <IP>: the address that responded.
• Host is up (Xs latency): round-trip time for the discovery probe.
• MAC Address: the hardware address of the host’s network interface, visible only when scanning a local subnet.

The final line summarizes the scan: total addresses checked, hosts found alive, and elapsed time.

Cleaner output

nmap -sn 172.16.10.0/24 | grep "Nmap scan" | awk -F'report for ' '{print $2}'
172.16.10.10
172.16.10.11
172.16.10.12
172.16.10.13
172.16.10.1

The command pipes Nmap’s output through two filters to extract only the IP addresses.

grep "Nmap scan" keeps only lines that contain Nmap scan, which are the Nmap scan report for <IP> lines. All latency, MAC address, and summary lines are discarded.

awk -F'report for ' '{print $2}' splits each remaining line on the delimiter report for and prints the second field. The second field is everything after that string: the IP address alone.

arp-scan

arp-scan sends Address Resolution Protocol (ARP) requests to hosts on a network and displays responses. ARP maps IP addresses to MAC addresses at layer 2. Because ARP operates at layer 2, arp-scan only works on local networks. It requires sudo to open a raw socket.

The --ouifile and --macfile flags specify the vendor lookup files. Without them, arp-scan looks for the files in the current working directory and fails. Pass absolute paths to avoid this. See Troubleshooting for details.

Single host

sudo arp-scan 172.16.10.10 -I br_public \
    --ouifile=/usr/share/arp-scan/ieee-oui.txt \
    --macfile=/etc/arp-scan/mac-vendor.txt
Interface: br_public, type: EN10MB, MAC: de:06:27:4e:8b:01, IPv4: 172.16.10.1
Starting arp-scan 1.10.0 with 1 hosts (https://github.com/royhills/arp-scan)
172.16.10.10	ea:21:3b:d3:be:cd	(Unknown: locally administered)

1 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.10.0: 1 hosts scanned in 0.194 seconds (5.15 hosts/sec). 1 responded

Scans a single IP address. Use this to confirm a specific host is up and retrieve its MAC address.

The header line shows the scanning interface, its MAC address, and its IPv4 address. Each result line contains the target IP, its MAC address, and the hardware vendor. The summary shows total hosts scanned, elapsed time, scan rate, and how many responded.

Subnet

sudo arp-scan 172.16.10.0/24 -I br_public \
    --ouifile=/usr/share/arp-scan/ieee-oui.txt \
    --macfile=/etc/arp-scan/mac-vendor.txt
Interface: br_public, type: EN10MB, MAC: de:06:27:4e:8b:01, IPv4: 172.16.10.1
Starting arp-scan 1.10.0 with 256 hosts (https://github.com/royhills/arp-scan)
172.16.10.10	ea:21:3b:d3:be:cd	(Unknown: locally administered)
172.16.10.11	1a:94:51:d8:e2:ee	(Unknown: locally administered)
172.16.10.12	86:5e:f8:1d:71:1b	(Unknown: locally administered)
172.16.10.13	ca:3b:ee:2b:b8:5d	(Unknown: locally administered)

4 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.10.0: 256 hosts scanned in 1.983 seconds (129.10 hosts/sec). 4 responded

Scans all 256 addresses in the subnet. Pass the network address with host bits zeroed (172.16.10.0, not 172.16.10.10) to avoid a warning.

File

sudo arp-scan -f /tmp/172-16-10-hosts.txt -I br_public \
    --ouifile=/usr/share/arp-scan/ieee-oui.txt \
    --macfile=/etc/arp-scan/mac-vendor.txt
Interface: br_public, type: EN10MB, MAC: de:06:27:4e:8b:01, IPv4: 172.16.10.1
Starting arp-scan 1.10.0 with 254 hosts (https://github.com/royhills/arp-scan)
172.16.10.10	ea:21:3b:d3:be:cd	(Unknown: locally administered)
172.16.10.11	1a:94:51:d8:e2:ee	(Unknown: locally administered)
172.16.10.12	86:5e:f8:1d:71:1b	(Unknown: locally administered)
172.16.10.13	ca:3b:ee:2b:b8:5d	(Unknown: locally administered)

4 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.10.0: 254 hosts scanned in 1.964 seconds (129.33 hosts/sec). 4 responded

Reads target IP addresses from a file, one per line. The file must be readable by nobody. See Troubleshooting.

Troubleshooting

Permission denied for vendor files

arp-scan drops all privileges to nobody after opening the raw socket. It then looks for ieee-oui.txt and mac-vendor.txt in the current working directory, not in their installed locations. nobody cannot read files in most users’ working directories.

Fix: pass absolute paths with --ouifile and --macfile:

--ouifile=/usr/share/arp-scan/ieee-oui.txt \
--macfile=/etc/arp-scan/mac-vendor.txt

Permission denied for the hosts file

arp-scan opens the hosts file after dropping to nobody. If the file is inside a home directory with 700 permissions (drwx------), nobody cannot traverse the path to reach it.

Two fixes are available. Allow traversal of your home directory without exposing its contents:

chmod o+x /home/<user>

Or copy the file to a world-accessible location:

cp ~/scripts/files/172-16-10-hosts.txt /tmp/

WARNING: host part of X/Y is non-zero

You passed a host address instead of the network address in CIDR notation. Use the network address with host bits set to zero:

# Wrong
sudo arp-scan 172.16.10.10/24 ...

# Correct
sudo arp-scan 172.16.10.0/24 ...

arp-scan still scans the full subnet but logs the warning.

Host monitoring script

The script continuously monitors a subnet for new hosts. When arp-scan finds a host not already in the known hosts file, it records it and sends an email alert.

#!/bin/bash

# sends a notification upon new host discovery
KNOWN_HOSTS="172-16-10-hosts.txt"
NETWORK="172.16.10.0/24"
INTERFACE="br_public"
FROM_ADDR="kali@blackhatbash.com"
TO_ADDR="security@blackhatbash.com"

while true; do 
  echo "Performing an ARP scan against ${NETWORK}..."
  sudo arp-scan -x -I ${INTERFACE} ${NETWORK} | while read -r line; do   # 1
    host=$(echo "${line}" | awk '{print $1}')                            # 2
    if ! grep -q "${host}" "${KNOWN_HOSTS}"; then                        # 3
      echo "Found a new host: ${host}!"
      echo "${host}" >> "${KNOWN_HOSTS}"                                 # 4
      sendemail -f "${FROM_ADDR}" \                                      # 5
        -t "${TO_ADDR}" \
        -u "ARP Scan Notification" \
        -m "A new host was found: ${host}"
    fi
  done 
  sleep 10
done

The outer while true loop runs the scan on a 10-second interval. The inner loop processes each line of arp-scan output:

1. Runs arp-scan with -x to suppress the header and footer lines, then pipes each result line into the inner loop.
2. Extracts the IP address from the first field of the arp-scan output line.
3. Checks whether the host already exists in the known hosts file. The ! negates the condition so the block runs only for new hosts. -q suppresses grep output.
4. Appends the new host to the known hosts file so it isn’t flagged again on the next scan.
5. Sends an email alert with the new host’s IP address.

sleep 10 pauses the outer loop for 10 seconds between scans.

Port scanning

After you discover hosts, you can run a port scanner to discover their open ports and the services they are running.

Nmap

Nmap is the most widely used port scanner. Use it to identify open ports, running services, and software versions on each host you discover.

By default, Nmap performs a SYN scan against the top 1,000 TCP ports. A SYN scan sends a SYN packet to each port and reads the response. An open port replies with SYN/ACK. A closed port replies with RST. A filtered port returns no response, which usually means a firewall is dropping the packets. Nmap reports three port states: open, closed, and filtered.

Single target

Scan a hostname or IP by passing it as the only argument:

nmap scanme.nmap.org
Starting Nmap 7.99 ( https://nmap.org ) at 2026-05-16 08:14 -0400
Nmap scan report for scanme.nmap.org (45.33.32.156)
Host is up (0.074s latency).
Other addresses for scanme.nmap.org (not scanned): 2600:3c01::f03c:91ff:fe18:bb2f
Not shown: 994 closed tcp ports (reset)
PORT      STATE    SERVICE
7/tcp     filtered echo
19/tcp    filtered chargen
22/tcp    open     ssh
80/tcp    open     http
9929/tcp  open     nping-echo
31337/tcp open     Elite

Nmap done: 1 IP address (1 host up) scanned in 8.01 seconds

scanme.nmap.org is Nmap’s official public test host. The output lists each port with three columns: port number and protocol, state, and service name. The Not shown: line reports how many ports were suppressed from output and why. closed tcp ports (reset) means those ports replied with RST. The two filtered ports (7/tcp and 19/tcp) returned no response, likely blocked upstream.

Scanning a local IP produces the same format:

nmap 172.16.10.1
Starting Nmap 7.99 ( https://nmap.org ) at 2026-05-16 08:20 -0400
Nmap scan report for 172.16.10.1
Host is up (0.00020s latency).
Not shown: 999 filtered tcp ports (no-response)
PORT   STATE SERVICE
22/tcp open  ssh

Nmap done: 1 IP address (1 host up) scanned in 4.94 seconds

Not shown: 999 filtered tcp ports (no-response) indicates nearly all ports on this host are firewalled.

Multiple targets

Pass space-separated hostnames or IPs to scan more than one target at once:

nmap localhost scanme.nmap.org
Starting Nmap 7.99 ( https://nmap.org ) at 2026-05-16 08:20 -0400
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0000090s latency).
Other addresses for localhost (not scanned): ::1
Not shown: 999 closed tcp ports (reset)
PORT   STATE SERVICE
22/tcp open  ssh

Nmap scan report for scanme.nmap.org (45.33.32.156)
Host is up (0.070s latency).
Other addresses for scanme.nmap.org (not scanned): 2600:3c01::f03c:91ff:fe18:bb2f
Not shown: 994 closed tcp ports (reset)
PORT      STATE    SERVICE
7/tcp     filtered echo
19/tcp    filtered chargen
22/tcp    open     ssh
80/tcp    open     http
9929/tcp  open     nping-echo
31337/tcp open     Elite

Nmap done: 2 IP addresses (2 hosts up) scanned in 5.55 seconds

Nmap prints a separate report for each host and a summary of total IPs scanned at the end.

Greppable output

The -oG flag writes results in a condensed, single-line-per-host format designed for parsing with grep and awk. Pass - as the filename to send output to stdout instead of a file.

nmap -iL files/172-16-10-hosts.txt --open -oG -
# Nmap 7.99 scan initiated Sat May 16 09:43:38 2026 as: /usr/lib/nmap/nmap --privileged -iL files/172-16-10-hosts.txt --open -oG -
Host: 172.16.10.1 ()	Status: Up
Host: 172.16.10.1 ()	Ports: 22/open/tcp//ssh///	Ignored State: filtered (999)
Host: 172.16.10.10 ()	Status: Up
Host: 172.16.10.10 ()	Ports: 8081/open/tcp//blackice-icecap///	Ignored State: closed (999)
Host: 172.16.10.11 ()	Status: Up
Host: 172.16.10.11 ()	Ports: 21/open/tcp//ftp///, 80/open/tcp//http///	Ignored State: closed (998)

Each host produces two lines. The Status line confirms the host responded to discovery probes. The Ports line lists open ports in the format port/state/protocol//service///. The Ignored State field reports how many ports were suppressed and their state. The leading comment line records scan metadata including the timestamp and command used.

Greppable output works well for extracting all hosts with a specific port open:

nmap -iL files/172-16-10-hosts.txt --open -oG - | grep "80/open"

XML output

The -oX flag writes results as XML. This format integrates with tools that consume nmap XML directly, such as Metasploit, Faraday, and Dradis. Pass - to send output to stdout.

nmap -iL files/172-16-10-hosts.txt --open -oX -
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<?xml-stylesheet href="file:///usr/share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 7.99 scan initiated Sat May 16 09:44:29 2026 as: /usr/lib/nmap/nmap -&#45;privileged -iL files/172-16-10-hosts.txt -&#45;open -oX - -->
<nmaprun scanner="nmap" args="/usr/lib/nmap/nmap -&#45;privileged -iL files/172-16-10-hosts.txt -&#45;open -oX -" start="1778939069" startstr="Sat May 16 09:44:29 2026" version="7.99" xmloutputversion="1.05">
<scaninfo type="syn" protocol="tcp" numservices="1000" services="1,3-4,6-7,9,13,17,19-26,...65129,65389"/>
<verbose level="0"/>
<debugging level="0"/>
<hosthint><status state="up" reason="arp-response" reason_ttl="0"/>
<address addr="172.16.10.10" addrtype="ipv4"/>
<address addr="36:77:21:A5:02:6B" addrtype="mac"/>
<hostnames>
</hostnames>
</hosthint>
<hosthint><status state="up" reason="arp-response" reason_ttl="0"/>
<address addr="172.16.10.11" addrtype="ipv4"/>
<address addr="82:85:42:7E:2F:07" addrtype="mac"/>
<hostnames>
</hostnames>

The <nmaprun> element wraps the entire scan and records the command, start time, and nmap version. <scaninfo> describes the scan type, protocol, and the port range checked. Each <hosthint> element identifies a host detected during the pre-scan discovery phase, before full port results are available. It includes the host’s IPv4 address, MAC address, and hostnames. Full port and service data appears in <host> elements in the complete output.

Service version detection

The -sV flag probes open ports to identify the software and version behind each one. Use -iL to read targets from a file:

nmap -sV -iL scripts/files/172-16-10-hosts.txt 
Starting Nmap 7.99 ( https://nmap.org ) at 2026-05-16 08:29 -0400
Nmap scan report for 172.16.10.1
Host is up (0.00023s latency).
Not shown: 999 filtered tcp ports (no-response)
PORT   STATE SERVICE VERSION
22/tcp open  ssh     OpenSSH 10.2p1 Debian 6 (protocol 2.0)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Nmap scan report for 172.16.10.10
Host is up (0.0000070s latency).
Not shown: 999 closed tcp ports (reset)
PORT     STATE SERVICE VERSION
8081/tcp open  http    Werkzeug httpd 3.0.1 (Python 3.12.3)
MAC Address: EA:21:3B:D3:BE:CD (Unknown)

...

The output adds a VERSION column showing the detected application and version string. Use Service Info lines to identify the operating system.

To see only open ports across all scanned hosts, pipe through grep:

nmap -sV -iL scripts/files/172-16-10-hosts.txt | grep open
22/tcp open  ssh     OpenSSH 10.2p1 Debian 6 (protocol 2.0)
8081/tcp open  http    Werkzeug httpd 3.0.1 (Python 3.12.3)
21/tcp open  ftp     vsftpd 3.0.5
80/tcp open  http    Apache httpd 2.4.58 ((Ubuntu))
80/tcp open  http    Apache httpd 2.4.57 ((Debian))
22/tcp open  ssh     OpenSSH 9.6p1 Ubuntu 3ubuntu13.16 (Ubuntu Linux; protocol 2.0)

This strips all header, footer, and closed/filtered lines, leaving a flat list of every open port across the entire scan.

Or use --open to have Nmap filter the output itself:

nmap -sV -iL scripts/files/172-16-10-hosts.txt --open
Starting Nmap 7.99 ( https://nmap.org ) at 2026-05-16 08:33 -0400
Nmap scan report for 172.16.10.1
Host is up (0.00016s latency).
Not shown: 999 filtered tcp ports (no-response)
Some closed ports may be reported as filtered due to --defeat-rst-ratelimit
PORT   STATE SERVICE VERSION
22/tcp open  ssh     OpenSSH 10.2p1 Debian 6 (protocol 2.0)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Nmap scan report for 172.16.10.10
Host is up (0.0000070s latency).
Not shown: 999 closed tcp ports (reset)
PORT     STATE SERVICE VERSION
8081/tcp open  http    Werkzeug httpd 3.0.1 (Python 3.12.3)
MAC Address: EA:21:3B:D3:BE:CD (Unknown)

...

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 254 IP addresses (5 hosts up) scanned in 15.96 seconds

Unlike the grep approach, --open preserves the per-host structure and includes Service Info lines. It reports only hosts with at least one open port and omits closed and filtered results from each report.

RustScan

RustScan is a fast port scanner written in Rust. It uses asynchronous I/O to scan ports significantly faster than Nmap. On Kali, rustscan runs as a Docker container. The command wraps docker run behind the scenes.

The typical workflow is to use RustScan to identify open ports quickly, then hand those results to Nmap for service detection.

If a previous scan was interrupted, the container may still exist. Remove it before rerunning:

docker rm -f rustscan

Basic scan

rustscan -a 172.16.10.0/24
...
Open 172.16.10.11:21
Open 172.16.10.1:22
Open 172.16.10.13:22

-a specifies the address or CIDR range to scan. The output prints one Open IP:port line for each open port found across the subnet.

Port range with greppable output

rustscan -g -a 172.16.10.0/24 -r 1-1024

-r 1-1024 limits the scan to ports 1 through 1024 instead of all 65,535. -g switches to greppable output, which writes one line per host in the format IP -> [port1, port2, ...]. This format is easier to parse with standard text tools.

Parsing greppable output

Strip the -> delimiter with awk to separate the IP from the port list:

rustscan -g -a 172.16.10.0/24 -r 1-1024 | awk -F'->' '{print $1, $2}'

-F'->' sets -> as the field separator. $1 is the IP address and $2 is the port list in brackets.

To remove the brackets and produce plain output, pipe through tr:

rustscan -g -a 172.16.10.0/24 -r 1-1024 | awk -F'->' '{print $1, $2}' | tr -d '[]'

tr -d '[]' deletes every [ and ] character from the stream, leaving just the IP addresses and port numbers.

Netcat

Netcat (nc) is a general-purpose network utility for reading and writing data across network connections. Use it for quick port checks when Nmap or RustScan are unavailable, or when you need a simple one-line scan of a small port range. Netcat is available on nearly every Unix-like system without installation.

The -z flag puts Netcat in zero-I/O mode: it connects to each port without sending data, then closes the connection. The -v flag enables verbose output so open ports appear in the results.

nc -zv 172.16.10.11 1-1024
172.16.10.11: inverse host lookup failed: Unknown host
(UNKNOWN) [172.16.10.11] 80 (http) open
(UNKNOWN) [172.16.10.11] 21 (ftp) open

The command scans ports 1 through 1024 on the target. Netcat reports only the ports it successfully connected to.

The inverse host lookup failed warning means nc could not resolve the IP to a hostname. It does not affect the scan. (UNKNOWN) in each result line reflects the same failed lookup. The port number, protocol name, and state follow in brackets.

Organizing scan results

When you scan a large subnet, nmap returns hundreds of lines of output. Organizing that output lets you quickly identify all hosts running a specific service, feed targeted lists into downstream tools like Nikto or Hydra, or map your attack surface by service type.

Common strategies:

• By port: One file per open port, each listing the hosts that expose it. Feed port-22.txt directly into a brute-force tool without extra filtering.
• By host: One file per IP, listing that host’s open ports. Useful for building a complete profile of individual targets.
• By service version: Group hosts by the banner nmap returns. Useful for finding all hosts running a specific vulnerable version.

The script uses the by-port strategy: it parses nmap output and writes each host’s IP to a file named after the open port.

#!/bin/bash
HOSTS_FILE="/home/ryan/scripts/files/172-16-10-hosts.txt"
RESULT=$(nmap -iL ${HOSTS_FILE} --open | grep "Nmap scan report\|tcp open")   # 1

while read -r line; do
    if echo "${line}" | awk -q "report for"; then                             # 2
        ip=$(echo "${line}" | grep open | awk -F'/' '{print $1}')             # 3
    else
        port=$(echo "${line}" | grep open | awk -F'/' '{print $1}')           # 4
        file="port-${port}.txt"                                               # 5
        echo "${ip}" >> "${file}"                                             # 6
    fi
done <<< "${RESULT}"

The while loop processes each line of the filtered nmap output:

1. Runs nmap against every host in HOSTS_FILE (-iL), reports only open ports (--open), then filters output to keep only scan report headers (Nmap scan report for) and open port lines (tcp open).
2. Checks whether the current line is a scan report header. Note: awk -q is not a valid pattern-match flag; this should use grep -q "report for" instead.
3. Extracts the host IP from the report header and stores it in ip. Note: grep open matches nothing on a report header line; the correct extraction is awk '{print $NF}' to get the last field.
4. Extracts the port number from the port line. awk -F'/' '{print $1}' splits on / and returns the first field—for example, 80 from 80/tcp open http.
5. Constructs a filename from the port number—for example, port-80.txt.
6. Appends the current IP to the port file.

done <<< "${RESULT}" feeds the entire RESULT variable into the loop as standard input using a here-string.

Detecting open ports

Port scanners show you what’s open right now. In lab environments and CTF challenges, you often need to know the moment a specific port becomes available: when a target finishes booting, when a service restarts after a crash, or when a firewall rule changes and exposes a previously closed port. Polling manually wastes time. A watchdog script automates the wait by scanning in a loop, alerting you the instant the port opens, and immediately running service detection to capture version information.

Common scenarios:

• Waiting for a GNS3 device to finish booting before attempting SSH or Telnet
• Monitoring a CTF target for a service that starts after a delay
• Detecting when a service restarts after exploitation or a configuration change
• Catching a port that opens intermittently under specific conditions

Port watchdog script

The script takes a target IP and port as arguments, polls with RustScan until the port opens, then runs an Nmap service scan and logs the results.

#!/bin/bash
LOG_FILE="watchdog.log"
IP_ADDRESS="${1}"                                                         # 1
WATCHED_PORT="${2}"

service_discovery() {
    local host
    local port
    host="${1}"
    port="${2}"

    nmap -sV -p "${port}" "${host}" >>"${LOG_FILE}"                      # 2
}

while true; do
    port_scan=$(docker run --network=host -it --rm --init \              # 3
        --name rustscan rustscan/rustscan:2.1.1 \
        -a "${IP_ADDRESS}" -g -p "${WATCHED_PORT}")
    if [[ -n "${port_scan}" ]]; then                                     # 4
        echo "${IP_ADDRESS} has started responding on port ${WATCHED_PORT}!"
        echo "Performing a service discovery..."
        if service_discovery "${IP_ADDRESS}" "${WATCHED_PORT}"; then     # 5
            echo "Wrote port scan data to ${LOG_FILE}"
            break
        fi
    else
        echo "Port is not yet open, sleeping for 5 seconds"
        sleep 5                                                           # 6
    fi
done

Run it with the target IP and port as arguments:

bash watchdog.sh 172.16.10.11 22

The service_discovery function runs Nmap service detection and logs the results. The while true loop polls with RustScan on a five-second interval until the port responds:

1. Reads the target IP and port from the first and second positional arguments.
2. Runs a service version scan (-sV) against the specific port and appends the output to LOG_FILE. >> appends rather than overwrites, preserving earlier scan data.
3. Runs RustScan in a Docker container and captures its output. --network=host gives the container access to the host network. --rm removes the container after it exits. -g enables greppable output: RustScan returns an empty string when the port is closed and a result line when it is open.
4. Tests whether port_scan is non-empty. A non-empty result means the port is up.
5. Calls service_discovery and checks its exit code. If Nmap succeeds, break exits the loop.
6. Waits five seconds before the next poll when the port is still closed.

After the script runs, each port-N.txt file contains one IP per line: every host in the subnet with port N open.

Banner grabbing

When you connect to a remote network service, the service publishes a text message to greet the client before any data is exchanged. This message is called a banner. It typically identifies the software, its version, and sometimes the underlying operating system. Banner grabbing is the process of extracting these banners to map what is running on each open port.

After a port scan tells you which ports are open, banner grabbing tells you what is behind them. Use it to:

• Identify software and versions running on open ports
• Find services running versions with known vulnerabilities
• Narrow your attack surface to specific targets before exploitation

Passive banner grabbing

Passive banner grabbing collects banner information without connecting directly to the target. Instead, you query third-party databases that have already scanned the internet and indexed the results. Shodan, ZoomEye, and Censys store banner data collected from their own scans and let you search by IP, port, service, or version string.

Use passive banner grabbing when:

• You need to stay undetected and avoid generating logs on the target
• You are in the early stages of external reconnaissance
• You want a quick overview of a target’s exposed services without touching it

Active banner grabbing

Active banner grabbing connects directly to a service and reads its response. Tools like Netcat, Telnet, and curl work for text-based protocols. Nmap’s -sV flag automates this across all open ports simultaneously.

Use active banner grabbing when:

• You are operating in an authorized pentest or lab environment
• You need current, accurate version information rather than cached data
• Passive sources don’t have data on the target

Banner grabbing script

The script reads a list of IP addresses from a file and attempts to grab the banner on a specified port from each host.

#!/bin/bash
FILE="${1}"
PORT="${2}"

if [[ "$#" -ne 2 ]]; then                                                # 1
    echo "Usage: ${0} <file> <port>"
    exit 1
fi

if [[ ! -f "${FILE}" ]]; then                                            # 2
    echo "File: ${FILE} was not found."
    exit 1
fi

if [[ ! "${PORT}" =~ ^[0-9]+$ ]]; then                                   # 3
    echo "${PORT} must be a number."
    exit 1
fi

while read -r ip; do                                                     # 4
    echo "Running netcat on ${ip}:${PORT}"
    result=$(echo -e "\n" | nc -v -w 1 "${ip}" "${PORT}" 2> /dev/null)   # 5
    if [[ -n "${result}" ]]; then                                        # 6
        echo "============"
        echo "+ IP Address: ${ip}"
        echo "+ Banner: ${result}"
        echo "============"
    fi
done < "${FILE}"

Run it with a hosts file and port number:

bash banner-grab.sh hosts.txt 22

Three guard clauses validate input before the loop runs. The while loop reads each IP from the file and attempts a banner grab:

1. Checks that exactly two arguments were provided. If not, prints a usage message and exits.
2. Checks that the file exists. If not, prints an error and exits.
3. Validates that the port is a number using a regex match (^[0-9]+$). If not, prints an error and exits.
4. Reads each IP address from FILE one line at a time and runs Netcat against it.
5. Sends a newline to the target port and captures the response. -v enables verbose output. -w 1 sets a one-second connection timeout. 2> /dev/null suppresses connection error messages.
6. If result is non-empty, prints the IP address and banner in a formatted block.

HTTP banners

To grab banners from web servers, use curl with the --head flag to send an HTTP HEAD request. The HEAD method retrieves only the response headers without fetching the full response body, making it faster and less intrusive than a GET request.

Web servers typically advertise themselves in the Server response header, often including the application name and version. This makes HTTP banner grabbing a reliable way to fingerprint web technologies.

curl --head 172.16.10.10:8081
HTTP/1.1 200 OK                                # 1
Server: Werkzeug/3.0.1 Python/3.12.3          # 2
Date: Sun, 17 May 2026 21:26:48 GMT
Content-Type: text/html; charset=utf-8         # 3
Content-Length: 7176                           # 4
Connection: close                              # 5

The request sends a HEAD to port 8081 on the target. The response reveals:

1. The server accepted the request and the resource exists.
2. The web server is Werkzeug 3.0.1 running on Python 3.12.3, indicating a Python web application.
3. The resource serves HTML content encoded in UTF-8.
4. The response body is 7,176 bytes, though HEAD does not return it.
5. The server closes the TCP connection after the response.

#!/bin/bash
DEFAULT_PORT="80"

read -r -p "Type a target IP address: " ip                                        # 1
read -r -p "Type a target port (default: 80): " port                              # 2

if [[ -z "${ip}" ]]; then                                                          # 3
    echo "You must provide an IP address"
    exit 1
fi

if [[ -z "${port}" ]]; then                                                        # 4
    echo "You did not provide a specific port, defaulting to ${DEFAULT_PORT}"
    port="${DEFAULT_PORT}"
fi

echo "Attempting to grab the Server header of ${ip}..."

result=$(curl -s --head "http://${ip}:${port}" | grep Server | awk -F':' '{print $2}' )  # 5

echo "Server header for ${ip} on port ${port} is: ${result}"                      # 6

The script prompts interactively for a target IP and port, then extracts the Server header from the HTTP response. Two guard clauses handle missing input before the request runs:

1. Prompts for the target IP address and assigns it to ip.
2. Prompts for a port number and assigns it to port. If the user presses Enter without a value, port is empty.
3. Checks whether ip is empty. If so, prints an error and exits.
4. Checks whether port is empty. If so, sets port to DEFAULT_PORT and continues.
5. Sends a silent HEAD request (-s suppresses progress output), pipes the output through grep to isolate the Server header line, then uses awk to extract the value after the colon.
6. Prints the server header value for the target IP and port.

Nmap scripts

Nmap includes a scripting engine that extends its functionality with Lua scripts stored in /usr/share/nmap/scripts. The banner.nse script connects to open ports and reads their banners, combining port discovery and banner capture in a single pass.

Use -sV with --script=banner.nse and -iL to grab banners from a list of hosts:

nmap -sV --script=banner.nse -iL files/172-16-10-hosts.txt

• -sV: enables service version detection
• --script=banner.nse: runs the banner script against each open port
• -iL files/172-16-10-hosts.txt: reads targets from a file rather than specifying them on the command line

Each banner result begins with |_banner or |_http-server-header. Pipe through grep to extract only those lines across all hosts:

nmap -sV --script=banner.nse -iL files/172-16-10-hosts.txt | grep "|_banner\||_http-server-header"
|_banner: SSH-2.0-OpenSSH_10.2p1 Debian-6
|_http-server-header: Werkzeug/3.0.1 Python/3.12.3
|_banner: 220 (vsFTPd 3.0.5)
|_http-server-header: Apache/2.4.58 (Ubuntu)
|_http-server-header: Apache/2.4.57 (Debian)
|_banner: SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.16

The grep pattern matches two output prefixes:

• |_banner: raw banners from non-HTTP services such as SSH and FTP
• |_http-server-header: the Server header extracted from HTTP responses

The output identifies four distinct server types across the subnet: two SSH hosts (OpenSSH on Debian and Ubuntu), one FTP server (vsFTPd 3.0.5), and three web servers (Werkzeug on Python, and two Apache instances on Ubuntu and Debian).

Detecting OSs

Nmap can guess a target’s operating system using TCP/IP fingerprinting as part of its OS detection scan. Every OS implements the TCP/IP stack differently: packet window sizes, TTL values, TCP options, and how the stack responds to unusual or malformed packets all vary by implementation. Nmap crafts packets in various ways, analyzes the responses, and compares the results against a database of known OS fingerprints to identify the most likely match.

Use the -O flag to enable OS detection. It requires at least one open and one closed port on the target for reliable results, and must run as root to craft raw packets:

sudo nmap -O -iL files/172-16-10-hosts.txt
Starting Nmap 7.99 ( https://nmap.org ) at 2026-05-17 17:57 -0400
Nmap scan report for 172.16.10.1
Host is up (0.00019s latency).
Not shown: 999 filtered tcp ports (no-response)
PORT   STATE SERVICE
22/tcp open  ssh
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose
Running: Linux 2.6.X|5.X
OS CPE: cpe:/o:linux:linux_kernel:2.6.32 cpe:/o:linux:linux_kernel:5 cpe:/o:linux:linux_kernel:6
OS details: Linux 2.6.32, Linux 5.0 - 6.2
Network Distance: 0 hops

Nmap scan report for 172.16.10.10
Host is up (0.00016s latency).
Not shown: 999 closed tcp ports (reset)
PORT     STATE SERVICE
8081/tcp open  blackice-icecap
MAC Address: F2:7B:0C:80:01:81 (Unknown)
Device type: general purpose
Running: Linux 4.X|5.X
OS CPE: cpe:/o:linux:linux_kernel:4 cpe:/o:linux:linux_kernel:5
OS details: Linux 4.15 - 5.19
Network Distance: 1 hop

Each host report includes OS detection fields after the port table:

• Warning: OSScan results may be unreliable: Nmap needs at least one open and one closed port to fingerprint reliably. 172.16.10.1 has only open and filtered ports, so the result is less certain.
• Device type: the general category of the device, such as general purpose, router, or printer.
• Running: the detected OS family and kernel version range. The | separates multiple candidates.
• OS CPE: Common Platform Enumeration identifiers — standardized strings that reference specific OS versions in vulnerability databases.
• OS details: the most specific version match Nmap found based on the fingerprint comparison.
• Network Distance: the number of hops between the scanning host and the target. A distance of 0 means the target is the scanning host itself. A distance of 1 means the target is directly connected.

OS detection script

The script runs an Nmap OS detection scan against one or more hosts and prints a clean summary of each IP and its detected OS.

#!/bin/bash
HOSTS="$*"                                                               # 1

if [[ "${EUID}" -ne 0  ]]; then                                          # 2
    echo "The Nmap OS detection scan type (-O) requires root privileges"
    exit 1
fi

if [[ "$#" -eq 0 ]]; then                                                # 3
    echo "you must pass an IP or an IP range"
    exit 1
fi

echo "Running an OS Detection Scan against ${HOSTS}..."

nmap_scan=$(sudo nmap -O ${HOSTS} -oG -)                                 # 4

while read -r line; do                                                   # 5
    ip=$(echo "${line}" | awk '{print $2}')                              # 6
    os=$(echo "${line}" | awk -F'OS: ' '{print $2}' | sed 's/Seq.*//g')  # 7

    if [[ -n "${ip}" ]] && [[ -n "${os}" ]]; then                        # 8
        echo "IP: ${ip} OS: ${os}"
    fi
done <<< "${nmap_scan}"                                                  #9

sudo bash os_detection.sh 172.16.10.0/24
Running an OS Detection Scan against 172.16.10.0/24...
IP: 172.16.10.10 OS: Linux 4.15 - 5.19
IP: 172.16.10.11 OS: Linux 4.15 - 5.19
IP: 172.16.10.12 OS: Linux 4.15 - 5.19
IP: 172.16.10.13 OS: Linux 4.15 - 5.19
IP: 172.16.10.1 OS: Linux 2.6.32|Linux 5.0 - 6.2

Two guard clauses validate privileges and input before the scan runs. The while loop processes each line of greppable Nmap output:

1. Assigns all positional arguments to HOSTS as a single space-separated string, allowing the script to accept an IP address or a CIDR range.
2. Checks EUID (effective user ID). A value other than 0 means the script is not running as root. OS detection requires raw packet privileges, so the script exits.
3. Checks that at least one argument was provided. If $# is 0, no targets were given and the script exits.
4. Runs the OS detection scan in greppable output format (-oG -). The - sends output to stdout so it can be captured in the variable rather than written to a file.
5. Reads each line of nmap_scan one at a time.
6. Extracts the IP address from the second field of the greppable output line.
7. Splits the line on OS: and takes everything after it, then strips from Seq onwards. The Seq token marks the start of sequence number data that follows the OS field in greppable output.
8. Prints the IP and OS only when both variables are non-empty, filtering out lines that contain no OS detection data.
9. Feeds the entire nmap_scan variable into the loop as standard input using a here-string, processing the captured Nmap output line by line without writing it to a file.

Analyzing websites and JSON

When you identify an open web port, fingerprint the service to map the technology stack before probing further. WhatWeb identifies web frameworks, server software, and version information by analyzing HTTP headers, cookies, and HTML. The default output gives a quick summary. When you need to extract specific values or feed results into scripts, use the --log-json flag to produce structured JSON and pipe it to jq to isolate exactly what you need.

whatweb 172.16.10.10:8081
http://172.16.10.10:8081 [200 OK] Country[RESERVED][ZZ], HTML5, HTTPServer[Werkzeug/3.0.1 Python/3.12.3], IP[172.16.10.10], Python[3.12.3], Title[Menu], Werkzeug[3.0.1], X-UA-Compatible[ie=edge]

Running whatweb against a host produces a one-line summary of everything the tool detected:

• 200 OK: the server responded successfully
• Country[RESERVED][ZZ]: the IP is in a private range, not publicly routed
• HTTPServer[Werkzeug/3.0.1 Python/3.12.3]: the server runs Werkzeug, a Python WSGI library used by Flask
• Python[3.12.3]: the Python version running the application
• Title[Menu]: the page title, which may hint at the application’s purpose
• Werkzeug[3.0.1]: the Werkzeug version. Combined with the Python version, this narrows the attack surface to known vulnerabilities in that release

whatweb 172.16.10.10:8081 --log-json=/dev/stdout --quiet | jq
[
  {
    "target": "http://172.16.10.10:8081",
    "http_status": 200,
    "request_config": {
      "headers": {
        "User-Agent": "WhatWeb/0.6.3"
      }
    },
    ...
  }
]

--log-json takes a file path and writes JSON-formatted results to that file. Passing /dev/stdout works because stdout is exposed as a file on Linux, which sends the output to the pipe instead of disk. --quiet suppresses the normal text output so only the JSON reaches the pipe. Piping to jq with no filter pretty-prints the full JSON structure, letting you explore the available fields before writing a targeted query.

whatweb 172.16.10.10:8081 --log-json=/dev/stdout --quiet | jq '.[0].plugins.HTTPServer.string[0]'
"Werkzeug/3.0.1 Python/3.12.3"

whatweb 172.16.10.10:8081 --log-json=/dev/stdout --quiet | jq '.[0].plugins.IP.string[0]'
"172.16.10.10"

The jq path .[0].plugins.HTTPServer.string[0] navigates the JSON structure:

• .[0]: the . represents the current input. Every jq path starts from . as the root. [0] selects the first element of that array. WhatWeb returns an array of results, one per target
• .plugins: accesses the plugins object containing all detected technologies
• .HTTPServer: selects the HTTPServer plugin entry
• .string[0]: each plugin stores its detected values in a string array. [0] retrieves the first value

The fourth example follows the same pattern, substituting .IP for .HTTPServer to extract the server’s IP address from the same JSON structure.