3.1.3.4. Agent Infrastructure: Hosted, Self-Hosted, and Caching
3.1.3.4. Agent Infrastructure: Hosted, Self-Hosted, and Caching
Triggers start pipelines; agents execute them. The choice of agent infrastructure affects build security, speed, and what resources pipelines can reach.
Agent infrastructure determines where pipeline code executes. Microsoft-hosted agents provide ephemeral, pre-configured VMs — fresh per job with no cross-build contamination, but no VNet integration for reaching private resources. Self-hosted agents (especially VMSS-based) offer full network control, sitting inside your VNet with NSG-controlled access to private feeds, databases, and registries. The trade-off: hosted agents are maintenance-free but network-limited; self-hosted agents require OS patching, image maintenance, and agent updates. Pipeline caching reduces build times by keying on lockfile hashes — when package-lock.json changes, the cache misses and runs a fresh install. VMSS agents auto-scale based on queue depth, spinning up when demand increases and scaling down during quiet periods. Keep separate agent pools per security tier — never run production deployment agents alongside dev build agents.
Content for Agent Infrastructure - see flashcards and questions for this subsection.
Agent pool design affects security, performance, and cost. Microsoft-hosted agents are ideal for open-source projects and standard builds — no infrastructure to manage, pre-installed tools, automatic updates. But they can't access private VNet resources, and each job starts from a clean image (no persistent cache beyond the Cache task).
Self-hosted VMSS agents provide the best of both worlds for enterprise scenarios: auto-scaling based on pipeline queue depth, VNet integration for private resource access, and custom images with pre-installed tools and cached dependencies. The VMSS scale set connects to Azure DevOps and automatically provisions or decommissions agents based on demand.
Agent image management is a pipeline itself. Build a custom VM image with Packer: install required SDKs, pre-pull Docker base images, pre-install npm/NuGet packages. Store the image in Shared Image Gallery. When the image updates, VMSS instances gradually reimage to the new version. This reduces "first build" time from 15 minutes (downloading everything) to 2 minutes (everything pre-cached).
Network security for self-hosted agents requires NSG rules allowing outbound to Azure DevOps service tags and internal resources, blocking all other internet access. Separate agent pools by security tier: development agents can have broader network access, while production deployment agents should only reach production resources through restricted paths.
Container-based agents (Docker-in-Docker or Kubernetes pods) offer the fastest scaling: new agents spin up in seconds from pre-built images. Azure DevOps supports Kubernetes-based agent pools where each pipeline job runs in a fresh pod. The trade-off: Docker-in-Docker has security implications (privileged containers), and pod-based agents require Kubernetes cluster management. For most teams, VMSS agents provide the best balance of scale and simplicity.
Demand-based scaling for VMSS agents uses a formula: desiredIdleAgents + queuedJobs = targetAgentCount. Configure minimum idle agents (for fast response to new jobs) and maximum agents (cost cap). The scale-down delay prevents premature decommissioning — agents wait idle for a configurable period before deallocating, avoiding the startup cost for closely spaced pipeline runs.
Container-based agents use Docker images as the agent environment, providing perfect reproducibility — the same image produces the same build everywhere. Teams maintain Dockerfiles for their build environments, versioning build tool dependencies alongside application code.
