Azure Data Factory (ADF) Syllabus

1. Azure Data Factory Fundamentals

• ADF Concepts: Modern Data Engineering Architecture, ADF Overview, Architecture & Components
• Core Components: Linked Services, Datasets, Pipelines, Integration Runtime
• Data Movement: Copy Activity Deep Dive, Dynamic Content & Expressions

2. Azure Data Lake Storage (ADLS Gen2) Integration

• Storage Architecture: ADLS Gen2 Architecture, File Systems & Containers
• File Processing: CSV, JSON, Parquet & Excel Handling
• Data Organization: Folder Structure Design, Dynamic File Handling, File Validation Framework

3. File-Based Ingestion Frameworks

• File Processing: Single File, Multiple File & Multiple Folder Processing
• ADF Activities: Lookup Activity, ForEach Activity, Get Metadata Activity
• Framework Design: Parameterized Datasets & Metadata-Driven Ingestion Framework

4. Database Connectivity & Ingestion

• Database Integration: Azure SQL, SQL Server, Oracle, PostgreSQL to ADLS
• Data Migration: Snowflake to Azure SQL
• Enterprise Frameworks: Multi-Table & Multi-Server Ingestion Frameworks
• Automation: Stored Procedure Activity

5. SQL for Data Engineering

• SQL Fundamentals: Joins, Aggregate Functions, String Functions, Date Functions
• Advanced SQL: Window Functions, Common Table Expressions (CTE), Recursive CTE
• Optimization: Duplicate Handling Techniques & Advanced SQL Interview Scenarios

6. Incremental Load Frameworks

• Loading Strategies: Full Load vs Incremental Load, Delta Processing
• Frameworks: Watermark Framework, Last Modified Date Framework
• Enterprise Design: Multi-Table Incremental Load, Control Table Design, Enterprise Incremental Architecture

7. Scheduling & Trigger Frameworks

• ADF Triggers: Schedule Trigger, Tumbling Window Trigger, Event-Based Trigger
• Automation: File Arrival Trigger & Dependency-Based Trigger
• Best Practices: Trigger Design & Monitoring Strategies

8. Enterprise Metadata-Driven Frameworks

• Metadata Design: Metadata Table Design & Dynamic Source Processing
• Dynamic Processing: Dynamic Sink Processing & Dynamic Schema Mapping
• Reusable Solutions: Single Pipeline for Multiple Sources & Reusable Framework Design

9. REST API Integration

• API Fundamentals: REST API Concepts & Authentication Mechanisms
• Security: Bearer Token Authentication
• Data Processing: API Pagination, JSON Processing & Error Handling
• Integration Framework: REST API to ADLS Automation Framework

10. Mapping Data Flows

• Transformations: Data Cleansing, Validation, Derived Columns
• Advanced Processing: Aggregate, Join, Pivot & Unpivot Transformations
• Data Warehousing: SCD Type 1 & SCD Type 2 Implementations

11. Security & Automation

• Security: Azure Key Vault Integration & Secure Credential Management
• Automation: Logic Apps Integration & Email Notifications
• Monitoring: Pipeline Status Notifications & Automated Alert Frameworks

12. Monitoring & Production Support

• Monitoring: Pipeline Logging, Audit Framework & Error Logging
• Optimization: Retry Mechanisms, Concurrency Management & Performance Tuning
• Support: Production Support Scenarios & Troubleshooting Techniques

13. Real-Time Industry Projects

• Project 1: SQL Server to ADLS Enterprise Ingestion Framework
• Project 2: Multi-Table Incremental Load Framework
• Project 3: Metadata-Driven Enterprise Pipeline
• Project 4: REST API to ADLS Automation Framework
• Project 5: Event-Based File Processing Solution
• Project 6: End-to-End Azure Data Factory Enterprise Project

14. Interview Preparation Program

• Interview Readiness: 100+ Azure Data Factory Interview Questions
• SQL Preparation: 100+ SQL Interview Questions
• Career Support: Mock Interviews, Resume Building Assistance
• Industry Exposure: Production Support Scenarios & Interview Best Practices

15. Course Highlights

• 90+ Hands-On Labs & Enterprise-Level Frameworks
• Metadata-Driven Design Patterns & Incremental Load Architectures
• REST API Integrations & Production Support Knowledge
• Lifetime Learning Community Access
• Interview & Career Guidance