2.1.2. OS Concepts for Security Analysts
💡 First Principle: Attackers live in the operating system — they use legitimate OS features (scheduled tasks, registry, processes) to execute and persist. Analysts who understand normal OS behavior can recognize when those features are being abused.
The OS is not just a platform the attacker targets; it's the toolkit they use after gaining access. Understanding normal OS behavior is the prerequisite for recognizing abnormal behavior.
Windows Registry is a hierarchical database storing OS and application configuration. From a security perspective, attackers use it for persistence (adding entries to auto-run keys so malware survives reboots) and for storing configuration data that would be suspicious as a file. Key locations to know:
| Registry Key | Security Significance |
|---|---|
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run | Runs programs at system startup — common persistence location |
HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Run | Runs programs at user login — user-level persistence |
HKLM\SYSTEM\CurrentControlSet\Services | Service configurations — attackers register malicious services here |
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options | Used in "debugger" hijacking attacks |
System hardening reduces attack surface by removing or disabling unnecessary features before a system is deployed. CIS Benchmarks provide specific, scored configuration recommendations for every major OS. A hardened system starts with fewer exploitable services, weaker default credentials, and less attack surface — making post-compromise analysis cleaner because "normal" is more constrained.
File structure matters because attackers hide malicious files in predictable locations. On Windows, C:\Windows\System32\ should only contain legitimate system files — malware dropped there mimics legitimate binaries. On Linux, /tmp/ and /var/tmp/ are common staging areas because they're writable by all users. Analysts should know the expected file structure to recognize anomalies.
On Linux, chmod changes permission bits (read/write/execute for owner, group, and others; chmod 755 is rwxr-xr-x), chown changes ownership, and ls -l shows both. World-writable files and unexpected SUID binaries are common persistence and escalation clues.
System processes have expected parent-child relationships. explorer.exe should spawn user applications, not cmd.exe or powershell.exe spontaneously. lsass.exe (Local Security Authority Subsystem) should only have one instance running — a second instance is a classic indicator of credential dumping malware. Process tree analysis is a core forensic technique.
svchost.exe is the generic host process for Windows services. Legitimate instances run from C:\Windows\System32\, are started by services.exe, and carry a -k <group> argument (for example -k netsvcs) naming the service group they host. A wrong path, a parent other than services.exe, or no -k is suspicious, which is why it is a favorite masquerade and injection target. When a correct-looking svchost.exe makes odd connections the path alone proves nothing: map its PID to the hosted service(s) (tasklist /svc) and inspect that service's entry (its ServiceDll) under HKLM\SYSTEM\CurrentControlSet\Services.
⚠️ Exam Trap: The Windows Registry is not only used for persistence — it can also store encrypted payloads, configuration data for fileless malware, and evidence of past activity (MRU lists, recently accessed files). When exam questions ask about registry analysis, think beyond just startup keys.
User-activity artifacts. To see which files a user recently opened, check the Recent folder (%APPDATA%\Microsoft\Windows\Recent, holding .lnk shortcuts) and Jump Lists (per-application recent-file lists under Recent\AutomaticDestinations), plus registry MRU keys. Security Event 4663 (object access) also records file access, but only where file-system auditing (a SACL) has been configured.
Process parentage in a memory image. Volatility's pstree shows each process under its parent, and comparing psscan (which carves process structures out of memory) with pslist (which walks the active process list) exposes processes unlinked to hide them. Legitimate svchost.exe instances are children of services.exe, which runs for the life of the system, so an svchost.exe whose parent PID matches no running process was not started through the normal service chain: its launcher exited or was hidden, a red flag for process hiding or injection-based persistence.
Reflection Question: An endpoint protection tool flags an instance of svchost.exe running from C:\Users\Public\ rather than C:\Windows\System32\. Why is this suspicious, and what category of attacker technique does this represent?
