Apache Airflow Experts in Frankfurt

• Led design, development, and deployment of data solutions supporting a major pharma acquisition for Takeda Pharmaceutical Company, delivering transparency reporting systems across Azure,Databricks (Python and Shell Scripting) platforms.
• Owned,Designed and developed scalable ELT pipelines to process Customer and Product data using Azure, complex SQL, Databricks, and shell scripting, enabling efficient data integration and processing across multiple sources including job orchestration and workflow automation.
• Implemented performance optimization techniques (query tuning, parallelism, workload optimization), improving system efficiency and processing time.
• Applied strong analytical and problem-solving skills to assess technical solutions and support business requirements for compliance and transparency reporting.
• Designed scalable data foundations suitable for downstream analytics and AI workloads.
• Led data quality initiatives by assessing multiple source data, defining quality metrics, and establishing processes for monitoring and continuous improvement.

I am an experienced data engineer specializing in end‑to‑end data integration, cloud DWH architectures, and high‑quality, governed data products.

I delivered following projects and engagements as a freelancer.

• Data Migration of CRM System for AL-FA Objekt Service Gmbh
• Microsoft Software Resales Partnership

I am looking for freelance roles like: Freelance Data Engineer Cloud Data Warehouse Architect Data Modeling & Architecture Consultant MDM & Data Governance Specialist BI & Analytics Developer

Technical Focus Areas

• Data Engineering & Integration: SQL Server/SSIS, Informatica PowerCenter/IDQ, Talend, Kafka, Azure Data Factory – Delta/CDC/ELT patterns, robust pipelines, monitoring/recovery, data lineage & impact analysis, medallion architecture Bronze/Silver/Gold layers
• DWH & Cloud: Azure SQL / Data Lake / Synapse, AWS Redshift/S3, on‑prem SQL/Oracle – scalable data marts with a strong cost/benefit focus.
• Data Modeling: Atomic (Inmon) and Dimensional (Kimball), Data Vault (Linstedt), Domain‑Driven Design, clear lineage & contracts.
• MDM & Governance: Informatica MDM, IBM MDM, stewardship processes, data quality rules, survivorship/XREF, catalog/glossary, SIF/BES/REST publication.
• Analytics/BI: Power BI, SSAS, Cognos – business‑ready, maintainable data products.

• Introduction of methods for developing and automated deployment of cloud-native software and machine learning applications in OpenShift clusters
• Development of various programs in Python
• Technologies: Kubernetes, OpenShift, Kustomize, ArgoCD, Tekton, Docker, PodMan, Airflow, IntelliJ, PyCharm, Git, Bitbucket, Jira, Confluence, Python

• Datastage 11.7, dbt, Oracle 19, Python 3.12 / PySpark 3.5, Azure GitHub, Azure DevOps, Automic
• Development of migration jobs to transfer data from the collection DWH to the new Risk Mart, as well as development of ETL pipelines to migrate historical data from the old Mart to the new Risk Mart.
• Storage of the silver layer on Hadoop and the gold layer in Oracle.
• Translation of DataStage jobs into dbt to publish reporting data in Google Cloud to a PostgreSQL database.
• Creation and optimization of complex SQL queries for data extraction from a data vault, taking into account historical data in the point-in-time tables.
• Creation of Oracle table definitions (DDL) and adjustment of existing stored procedures.
• Versioning changes in GitHub and deployment via the CI/CD portal.
• Refactoring long-running DataStage jobs into Python using PySpark to reduce server load.
• Migration of SAS scripts to PL/SQL, including new development of distribution functions that have no direct equivalent in Oracle.
• Development of Automic jobs to run DataStage pipelines and Python scripts (PySpark jobs) that control the population of the SME and institutional risk tables in the Risk Mart and perform business calculations.
• Participation in the agile process, including creating user stories, estimations, and planning in Azure DevOps.
• Handling Azure DevOps tickets and close collaboration with testers and business teams for error analysis and resolution.

• Advising on the use of analytics and BI tools and services in the Microsoft Azure stack (e.g. MS Fabric, Synapse Workspaces and dedicated SQL pools, SQL Database, PostgreSQL, Snowflake, Databricks, Data Factory, SSIS, Analysis Services, Function Apps, Power BI, ML)
• Independently designing analytics solutions with Python, SQL, etc.
• Designing and implementing ETLs and data pipelines
• Creating and maintaining APIs
• Independently applying CI/CD, testing, and version control
• Data modeling
• Model development and optimization
• Anomaly detection with AI
• Predictive analytics

Used technologies:

• Snowflake
• Fabric
• Azure Synapse Analytics
• Azure DataFactory
• Azure Data Lake
• Azure DevOps
• Databricks
• Spark
• CI/CD
• SQL Database
• Python
• Power Platform

• Presentation introducing generic AI and large language models
• Explanation of legal frameworks (EU AI Act, US CLOUD Act, GDPR)
• Systematic review of AI tools along the SDLC and holistic systems
• Comparison of on-prem LLMs vs. cloud-based, as well as change management and works council
• Facilitated the discussion and derived next steps for introducing AI development tools

• Implementation of specified DevOps solutions to automate infrastructure (Terraform, Bicep, CloudFormation, Ansible) on-premises datacenter (Ovirt, Proxmox, Ceph Cluster, MinIO) and private cloud.
• Administration, configuration and implementation of CI/CD DevOps pipelines (GitLab, GitFlow) to support development process (Artifactory, Prometheus, Istio, service mesh, Helm Chart, OpenShift (Red Hat Enterprise) / Kubernetes cluster), Red Hat Satellite.
• Administration, setup, monitoring and patching of Linux infrastructure based on Red Hat Enterprise for Dev, Test and QA.
• Use of Scrum and Kanban methods.
• Administration, configuration and implementation of security standards for deploying on Dev, Test, QA and Prod stages of the new ePA applications.
• Development of new plugins and add-ons needed on current infrastructure.
• Database support.
• Data analytics support (Python, Spark, Pandas, Power BI, Splunk Enterprise).
• Implementation of best practices for DevSecOps and BizDevOps using GitOps (ArgoCD), Streamlit framework, Semaphore Ansible UI.
• Configuration and testing of iperf, uperf, sysbench using benchmark-operator for external source data and IoT/MDM devices, creating reports via ELK / OpenSearch.
• Building a new Databricks platform to collect and analyze big data from different sources and IoT devices into Hadoop framework (Python, Pandas, PySpark, Power BI, Apache Airflow).
• Building backend data aggregation and processing to automate configuration deployment between different OpenShift clusters and big data framework (Python, Pandas, PySpark, Apache Spark, PostgreSQL, Django 2, Ansible Automation, Jira JSM).
• Building a new ML pipeline platform using Kubeflow, TensorFlow, KServe.
• Data extraction, transformation and loading from different data sources including structured and unstructured data to analytic DWH / big data cluster using Python, Pandas, Polars, Power BI, Django backend and PostgreSQL.
• Setup of new DevOps Test and QA HashiCorp Vault cluster for PKI and IAM.
• Configuration and testing of automated patching based on CVSS score, SIEM-integrated CVEs.
• Use of Nexpose and InsightVM to scan vulnerability events in network, host, container and application.
• Design and implementation of secure and scalable AWS architectures including VPC, EC2, S3, RDS and Route53 and similar setups on Azure and GCP.
• Automated system provisioning and deployment using CloudFormation templates.
• Configuration of IAM roles, policies and permissions to ensure secure access control.
• Patch management, backup automation and disaster recovery setup on AWS infrastructure.
• Monitoring and optimization of system performance using AWS CloudWatch and AWS Trusted Advisor.
• Support of VMware services (vSphere, Aria, Horizon) and the virtual desktop environment.
• Development and maintenance of CI/CD pipelines using Jenkins, GitLab CI/CD and AWS CodePipeline with interface to Nutanix.
• Configuration of AWS CloudWatch to monitor application performance and system events.
• Planning and execution of migration of on-premises applications to AWS cloud platforms.
• Deployment of containerized applications using Docker and Kubernetes in AWS environments.
• Deployment of internal software packages between availability zones using AWS CodeDeploy.
• Building and deploying ML models using Scikit-learn, XGBoost and Spark MLlib including hyperparameter tuning, model evaluation and production deployment.

• Development of a central billing app for cloud costs at DB
• AWS
• Python
• AWS CDK
• RDS
• Spark (PySpark)
• Glue
• Lambda
• CI/CD (GitLab)
• React/Typescript
• data optimization
• Scrum

• Practice-oriented coaching on core aspects of data science
• Teaching advanced concepts in Python as well as automation (with Make and n8n) and ETL processes with Apache Airflow
• Weekly preparation and delivery of practice-oriented programming courses using real-world examples
• Promoting practical programming skills among participants through interactive exercises and individual support
• Developing didactic materials and adapting content to participants' skill levels
• Close collaboration with the team for continuous improvement of course quality and learning outcomes