3.1.3.1. YAML Pipeline Fundamentals: Variables, Expressions, and Secrets
3.1.3.1. YAML Pipeline Fundamentals: Variables, Expressions, and Secrets
YAML-based pipeline configuration (Pipeline as Code) stores pipeline definitions alongside application code in version control. Changes go through PR review, creating an auditable history of who changed what and why. The three variable expression types serve different evaluation times: ${{ }} template expressions evaluate at compile time (parameters, static values), $[ ] runtime expressions evaluate during execution (system variables, output variables), and $(var) macro replacement handles variable groups and pipeline variables. Output variables pass computed values between jobs using ##vso[task.setvariable] logging commands — the source step must have a name: property for downstream reference. Variable groups linked to Key Vault use service connections for authentication; if a new secret doesn't appear, verify the service principal's access policy and the variable group's authorization status.
YAML pipeline structure follows a hierarchy: pipeline → stages → jobs → steps. Stages are deployment units (Build, Test, Deploy-Staging, Deploy-Prod). Jobs run on agents and can execute in parallel within a stage. Steps are individual tasks or scripts.
Template parameters provide compile-time customization. Unlike runtime variables, template parameters are resolved before the pipeline starts, enabling conditional stage/job inclusion. Template expressions (${{ if eq(parameters.environment, 'prod') }}) control which stages appear in the compiled pipeline, allowing a single template to generate different pipeline structures per environment.
Variable precedence follows a specific hierarchy: command-line overrides > pipeline-level > stage-level > job-level > template defaults. Understanding this prevents debugging sessions where a variable has an unexpected value because a higher-priority source overrode it. Queue-time variables (settable when manually running a pipeline) override pipeline-defined values, which is useful for debugging but dangerous if not controlled — restrict queue-time overrides to non-sensitive variables.
Conditional execution with condition: expressions enables skip-logic: condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main')) runs a stage only on main branch builds. Combined with parameters, this creates reusable templates that adapt behavior based on context.
Extends and template repositories enable organizational governance at scale. A central template repository defines approved pipeline patterns — build templates, deployment templates, scanning templates. Teams reference these templates using resources: repositories and extends: syntax. Required template checks on environments ensure no team can deploy without using the approved template.
Pipeline debugging uses system.debug: true to enable verbose logging, stepTarget: host to run steps on the host rather than in a container for debugging, and diagnostic logging (##[debug]) for custom trace output. For complex YAML issues, the Azure DevOps pipeline editor provides a "View YAML" mode that shows the fully compiled pipeline after template expansion.
Variable templates enable sharing variable definitions across pipelines. A variables-common.yml template defining image names, service connections, and environment-specific values is referenced by all pipelines, ensuring consistency. When the base image version updates, changing one template file updates all consuming pipelines. This is preferable to variable groups for values that should be version-controlled alongside pipeline definitions.
Secure files in Azure Pipelines store certificates, provisioning profiles, and SSH keys that tasks download at runtime. Unlike variable groups, secure files are binary blobs encrypted at rest. They appear in the agent's working directory only during task execution and are cleaned up afterward.
Reusable templates enable organizational standards. A central template repository provides approved pipeline patterns that teams extend with extends syntax. This ensures every pipeline includes mandated security scanning and approval stages without each team writing them from scratch.
