Outline updated November 2026

Foundation21-28 hours delivered over 3-4 daysLive, online

Request this course

Data Engineering: Ansible, Airflow, and Snowflake

Class Duration

21-28 hours of live training delivered over 3-4 days. The longer format adds depth on Snowpark, dbt, and production practices.

Student Prerequisites

  • Practical experience with Python
  • Basic Linux command line skills
  • Familiarity with SQL fundamentals
  • Basic understanding of cloud computing concepts

Target Audience

Data engineers, DevOps engineers, and Python developers who build automated data pipelines. Teams who want more depth on orchestration can continue to Apache Airflow 3 for Developers.

Description

This course combines Ansible, Apache Airflow, and Snowflake for data engineering. Ansible provisions and configures the environment, and Airflow schedules and monitors the pipelines. In Snowflake, students load data with stages, COPY INTO, Snowpipe, and Openflow. They transform data with SQL, Snowpark Python, Dynamic Tables, and dbt Projects on Snowflake, and call Cortex AI Functions from SQL. Final modules connect the three tools in ELT pipelines and cover testing, CI/CD, and cost control.

Learning Outcomes

  • Provision infrastructure with Ansible playbooks, roles, and collections.
  • Manage dynamic inventories and secrets with Ansible Vault.
  • Write Airflow DAGs with the TaskFlow API, assets, and dynamic task mapping.
  • Size Snowflake virtual warehouses and control cost.
  • Load data with stages, COPY INTO, Snowpipe, and Openflow connectors.
  • Transform data with Snowflake SQL, Snowpark Python, and Dynamic Tables.
  • Capture changes with streams and tasks.
  • Build and run dbt projects with dbt Projects on Snowflake.
  • Call Cortex AI Functions from SQL while managing cost and governance.
  • Orchestrate Snowflake ELT pipelines with Airflow and the Snowflake provider.

Training Materials

Comprehensive courseware is distributed online at the start of class.

Software Requirements

Students will need a free, personal GitHub account to access the courseware. Students will need permission to install Python and Visual Studio Code on their computers, along with Python packages and VS Code extensions. Courseware targets ansible-core 2.22, Apache Airflow 3.4, and current Snowflake features. A Snowflake trial or training account will be used. Openflow is not enabled on trial accounts by default, and some Cortex AI Functions depend on region, so both are confirmed during class planning. If students are unable to configure a local environment, a cloud-based environment can be provided.

Training Topics

Introduction to Ansible

  • Overview and architecture
  • Declarative, push-based configuration
  • ansible-core and collections
  • Fully qualified collection names
  • YAML syntax review
  • Command-line tools

Ansible Setup and Inventory

  • Installing and configuring ansible-core
  • Configuration files
  • Defining hosts and groups
  • Dynamic inventory for cloud environments

Ansible Playbooks, Variables, and Templates

  • Playbook writing basics
  • Tasks and handlers
  • Commonly used modules
  • Variables and facts
  • Jinja2 templates and template modules
  • Linting playbooks with ansible-lint

Ansible Roles and Reuse

  • Creating and using roles
  • Role directory structure
  • Ansible Galaxy and collections
  • Writing reusable code

Apache Airflow Essentials

  • Workflows as code
  • Directed acyclic graphs (DAGs)
  • Airflow services
  • Installing with a constraints file
  • Metadata database options
  • Executor types and configuration

Developing DAGs

  • The airflow.sdk authoring interface
  • TaskFlow API and Operator API
  • Connections, variables, and XComs
  • Assets and asset-based scheduling
  • Dynamic task mapping
  • Monitoring runs and task logs

Introduction to Snowflake

  • Cloud data platform architecture
  • Separation of storage and compute
  • Virtual warehouses
  • Snowflake editions and pricing
  • Snowsight and Workspaces

Snowflake Objects and Data Modeling

  • Databases, schemas, and tables
  • Views and materialized views
  • Data types and constraints
  • Clustering keys
  • Time Travel and Fail-safe
  • Apache Iceberg tables

Loading Data into Snowflake

  • Internal and external stages
  • The COPY INTO command
  • CSV, JSON, and Parquet file formats
  • Snowpipe for continuous loading
  • Snowpipe Streaming for low-latency ingestion
  • Openflow managed connectors
  • Openflow Snowflake deployments and BYOC

Transforming Data in Snowflake

  • Snowflake SQL extensions and window functions
  • Semi-structured data with VARIANT
  • Snowpark Python DataFrames
  • Snowpark UDFs and stored procedures
  • Dynamic Tables for declarative pipelines
  • Streams and tasks for change data capture
  • Query optimization
  • Query History and Query Profile

dbt Projects on Snowflake

  • dbt models, sources, and tests
  • dbt documentation
  • dbt projects in Snowflake Workspaces
  • Deploying dbt projects with Snowflake CLI
  • Scheduling dbt runs with tasks or Airflow

Cortex AI Functions in Pipelines

  • AI_COMPLETE and AI_CLASSIFY
  • AI_EXTRACT and AI_AGG
  • Summarization, sentiment, and document parsing
  • Cost controls for AI Functions
  • Choosing a model family
  • Regional availability
  • AI governance in data pipelines

Security and Access Control

  • Role-based access control
  • Users, roles, and privileges
  • Key-pair and OAuth authentication
  • Service users
  • Row access and masking policies
  • Network policies
  • Secure Data Sharing

Orchestrating Snowflake with Airflow

  • Snowflake provider for Airflow
  • SQLExecuteQueryOperator and SnowflakeSqlApiOperator
  • Connection configuration
  • Running Snowpark and dbt workloads from DAGs
  • Error handling and retries
  • Idempotent pipeline design

Using Ansible for Data Infrastructure

  • Provisioning Airflow environments
  • Managing Snowflake users, roles, and warehouses
  • Secrets management with Ansible Vault

Production Considerations

  • Data quality checks
  • Testing data pipelines
  • CI/CD for data pipelines
  • Monitoring, alerting, and cost management
  • Warehouse sizing and performance tuning

Private team training

Bring this course to your team

Tell us about your team and we will tailor this outline to your tools, stack, and schedule. Courses can also be combined into a longer program.

  1. Tell us what you needYour tools, stack, team size, and timeline.
  2. We send a proposalA course or program, an outline adjusted to your team, and possible dates.
  3. Your team trains liveOnline or on-site, with courseware for every student.

Prefer to talk? Call +1 434-509-6890 or email help@cloudcontraptions.com.

We reply within 1 business day.

Training partner?