6.3.1. Network Diagnostic Methodology

# Layer 1-2: Is the link up?
ip link show eth0                # State: UP or DOWN
ethtool eth0                     # Link speed, duplex, auto-negotiation
ip -s link show eth0             # Counters: TX/RX errors = NIC/driver/cable faults;
                                 #   RX dropped = packets discarded by kernel/driver (often a full ring buffer, see ethtool -g)
# If DOWN: check cable, NIC, switch port

# Layer 3: Does this host have a valid IP?
ip address show eth0             # IP address and prefix length
# If no IP: check NetworkManager / Netplan / DHCP

# Layer 3: Is the default gateway configured?
ip route show                    # Look for: default via X.X.X.X
ping -c 2 <gateway_ip>           # Can we reach the gateway?
# If no route: add it or fix networking config
# If can't ping gateway: check ARP (ip neigh show; legacy: arp -n), check firewall on gateway

# Layer 3: Internet routing
ping -c 2 8.8.8.8                # Can we reach internet by IP?
# If fail but gateway OK: ISP routing or NAT issue

# Layer 7: DNS resolution — DNS is an APPLICATION-layer service (L7), not L5.
# It is tested here, before the Layer 4 port checks below, because everything
# addressed by NAME depends on it — a working L4 path still fails if DNS is broken.
dig google.com                   # Full DNS resolution test
dig google.com @8.8.8.8          # Test against specific resolver
nslookup google.com              # Simpler resolution test
cat /etc/resolv.conf             # What DNS servers are configured?
resolvectl status                # systemd-resolved status
# If IP ping works but domain names fail: pure DNS problem

# Layer 4: Can we reach the specific port?
nc -zv 192.168.1.10 443          # TCP connectivity test to port 443
nc -zv -w 3 192.168.1.10 443    # 3 second timeout
curl -I https://example.com      # HTTP-level test (also tests TLS, app)
telnet 192.168.1.10 80           # Legacy port test

# Path diagnostics
traceroute 8.8.8.8               # Where does the path fail?
mtr 8.8.8.8                      # Combined ping + traceroute, continuous

Firewall Troubleshooting:

# Is firewalld blocking?
firewall-cmd --list-all          # What's allowed?
firewall-cmd --list-ports
firewall-cmd --list-services

# Is nftables/iptables blocking?
nft list ruleset                 # Full nftables rule set
iptables -L -n -v                # iptables rules with packet counters
# Look for DROP/REJECT rules affecting your traffic

# Temporarily disable to test (restore after!)
systemctl stop firewalld         # Test without firewall (restore after!)
firewall-cmd --panic-on          # Block ALL traffic (for emergency containment)
firewall-cmd --panic-off         # Restore normal operation

# SELinux blocking network?
getsebool httpd_can_network_connect   # Is this boolean enabled?
ausearch -m avc -ts recent       # Recent SELinux denials

In a cloud VM, traffic passes several independent filters: subnet-level rules, instance/NIC-level security groups, and the OS firewall inside each guest. Opening one layer opens none of the others, so check every layer that applies to the path — including the guest firewall on both VMs.

SSH Connection Debugging:

ssh -v alice@server              # Verbose: shows each step of connection
ssh -vvv alice@server            # Very verbose (key exchange/auth negotiation detail — SSH uses its own transport, not TLS)
# Look for: "Authentications that can continue"
#           "Permission denied (publickey)"
#           "Connection refused" (SSH not running or port wrong)
#           "Connection timed out" (firewall or routing)

# On the server
sshd -t                          # Test sshd_config syntax
journalctl -u sshd -f            # Live SSH authentication log
tail -f /var/log/auth.log        # Debian: authentication log
tail -f /var/log/secure          # RHEL: authentication log

DHCP failure indicators

• ip addr show shows a 169.254.x.x (APIPA) address — the client asked for a lease and never got an offer. APIPA is the self-assigned fallback, so it is a symptom of DHCP failure, never a working configuration
• journalctl -u NetworkManager shows a DHCP timeout — the server is unreachable, or its pool is exhausted
• Verify in this order: is the DHCP server running? is the client on the interface you think it is? and — if the server is on another subnet — is a DHCP relay configured? DHCP discovery is a broadcast, and broadcasts do not cross a router, so a remote DHCP server is unreachable without a relay agent forwarding those requests (for example ISC dhcrelay -i <client-side-interface> <server-ip>, which picks up the broadcasts and unicasts them to the server)

Routing problems

• ip route show — is the correct default gateway present?
• traceroute <destination> — where does the path stop?
• Routing is bidirectional. A perfectly correct outbound route still fails if the remote host has no return path to you. When outbound looks right and traffic still dies, suspect the return leg.
• One service fails while others on the same host reach the same destination: the path is fine, so look at what differs for that service — the interface or source address it binds to (ss -tnp shows the local address each connection uses), a per-service proxy, or a remote allowlist that doesn't include that source IP.

Link, VLAN and throughput problems

• MTU mismatch: small packets (ping) succeed but large transfers stall or reset. Compare the MTU on both ends (ip link show), probe with ping -M do -s 1472 <host> (1472 + 28 bytes of headers = 1500, Don't-Fragment set), and use tracepath to find the path MTU.
• VLAN subinterface receives nothing: an 802.1Q interface such as eth0.100 needs the 8021q kernel module (lsmod | grep 8021q, modprobe 8021q); ip link add link eth0 name eth0.100 type vlan id 100 is runtime-only, so persist it with NetworkManager or Netplan; and the switch port must trunk that VLAN.
• Bond doesn't fail over: a bond can only fail over if it detects link loss — MII polling (miimon=100) or ARP monitoring (arp_interval + arp_ip_target). With monitoring off (miimon=0 and no ARP monitor), active-backup keeps sending on a dead member. cat /proc/net/bonding/bond0 shows the mode, polling interval and active member.
• Throughput far below link speed on a high-latency path: TCP can have at most one window of unacknowledged data in flight per round trip, so throughput ≤ window ÷ RTT (the bandwidth-delay product). A 1 Gbps path with 50 ms RTT needs about 6 MB of window; check that window scaling is on (net.ipv4.tcp_window_scaling) and the maxima in net.ipv4.tcp_rmem / tcp_wmem.

Split-horizon DNS

If dig @<internal-resolver> host fails but dig @8.8.8.8 host succeeds — or the reverse — you are probably looking at split-horizon DNS, where internal and external resolvers deliberately answer the same name differently. That is a configuration question ("which resolver should this host be using?"), not a broken-DNS question.

Stale or missing answers from one resolver

• Resolvers cache every answer for the record's TTL. After a record changes, a resolver holding the old answer keeps returning it until the TTL expires or its cache is flushed (resolvectl flush-caches for the local systemd-resolved cache). If dig @<authoritative-server> name already returns the new value, the record is fine — the cache is stale.
• If the local resolver returns NXDOMAIN for an internal name that the internal authoritative server answers, the resolver is asking the wrong servers: it needs a conditional forwarder (or routing domain) that sends the internal zone to the internal DNS server.

⚠️ Exam Trap: A "Connection refused" error means the destination host is reachable but nothing is listening on that port (or a firewall is actively refusing). A "Connection timed out" means packets are not reaching the destination at all — blocked by firewall or routing failure. These require different fixes and the exam tests whether you can distinguish them.

Reflection Question: A developer can ping a web server by IP but curl http://webapp.example.com fails with "Could not resolve host." What layer of the stack is failing, and what three things would you check to diagnose it?