Apache Pig Experts in Germany

• Head of overall technical integration NSC (National Secure Cloud, project with approx. 60 employees).

• Technical integration of all subprojects into one product, definition of interfaces and basic components of a cloud including hardware, technical architecture of the IABG platform.

• Development of a Cloud Management Platform (CMP) capable of creating private/mixed clouds of any complexity based on a textual description with one click or interactively.

• CMP also includes the complete hardware management lifecycle.

• Kubernetes, OpenStack and Hadoop are used as the foundation.

• The management layer includes Harbor, Gitea, Longhorn, Keycloak, Rancher and Jenkins, which are configured automatically.

• Private cloud can run any customer workloads, including a full Hadoop layer with HDFS, Spark, MapReduce, Mesos, HBase and around 20 additional ML/DL technologies.

• Hadoop worker clusters can also be installed automatically without Kubernetes on bare metal or commodity hardware.

• OpenStack with Nova, Neutron, Ironic, Swift, Cinder, Ceph.

• Development of a Java application Rudi: SOAP, REST, containers, DB.

• Technologies: Kubernetes (K3s, Rke2, Minikube, Harbor, Gitea, Jenkins, Longhorn, Keycloak, Rancher), OpenStack (Nova, Neutron, Keystone, Swift, Ceph, Cinder, Sahara, Magnum, Kayobe, Kolla, Bigrost, Ironic), Hadoop (HDFS, Ambari, Solr, Livy, Ranger, YARN, Tez, HBase, Kafka, Hive, Zookeeper, MapReduce, Spark, Oozie, Flink), virtualization (Kubernetes (K3S), VMware, Oracle), scripting (Ansible, Puppet, Juju, Shell, Groovy, Gradle, Maven).

• Defined a generic, universal system platform for XR (AR, VR) use cases as part of the European IPCEI initiative
• Evaluated available options for backend, streaming and client implementations
• Developed PoC implementations and conducted technical evaluation of potential components
• Led the development team and defined the target system architecture for a complete end-to-end solution
• Designed and integrated GenAI PoCs (LLM, time series, prediction, agents)
• Researched NeRFs and Gaussian Splatting for XR content creation and analyzed methods to measure and reduce system latencies
• Worked on GPU scheduling, CUDA and related topics
• Defined and implemented backoffice and end user dashboards using Grafana and Superset
• Used Jaeger tracing to analyze specific events
• Defined a GenAI-based automatic alerting feature for specific events and event clusters with KPI values
• Integrated AR/VR/XR IoT devices
• Collaborated with business development and UX teams in a parallel design thinking track to align technical and customer journey scopes
• Integrated the new services and platform into existing telco environments and analyzed tools like Camara for partial integration and handover scenarios

• Lead Architect to design and implement the data lake and ETL Pipeline using AWS Stack
• Performance Optimization of Data Ingestion of ETL Pipeline
• Development of Data Validation using Great Expectations
• Leading of the data migration for two sources exchanges
• Data Modeling in AWS Redshift

MLOps

• Model inference implementation by mlflow and AWS SageMaker
• Feature Engineering for the running ML Models ( Recommender Engineer, Clustering )
• Implementatino of Model Registry and artifactory using mlflow
• Historization an Profiling of the Input Data Using AWS Glue Crawler and AWS Data Catalog
• Feature importance using mlflow

Tech. Stack: Python 3, AWS Glue, AWS Step Fucntion, AWS Lambda, AWS EventBridge, AWS IAM Role, AWS SageMaker, AWS EC2, AWS Glue Crawler, AWS CloudWatch, MLFlow, ETL, Data lake, GitHub Action, Terraform, Jenkins, Ansible playbooks (Infrastructure as Code), CI/CD, GitLab, SQL, PySparkSCRUM, Agile, Jira, BigData, VSCode, DBeaver, MSSQL, MySQL, grafana, Docker, Linux, Bash, MapReduce, Data Modeling (ORM), Pandas, YAML, SQL-Alchemy

• Contributed to over 20 successful projects

• Global marketing analytics for Hitachi Energy as part of a global data modernization initiative aiming to enhance data retention, historical data availability and provide Eloqua's 2-year retention for remote interaction reporting and analytics.

• Analyzed Eloqua's default retention policy and identified risk of data loss for records older than two years.

• Designed and implemented historical data preservation strategy by creating transformed tables in the target data platform to archive older data while ensuring data quality dashboards.

• Collaborated with the Power BI team to re-point dashboards from raw Eloqua imports to the newly created archival layer.

• Leveraged Jira to track and manage data engineering tasks, bugs, and feature requests across Agile sprints; coordinated backlog prioritization and task assignment to align data pipeline development with business needs.

• Power BI dashboard optimization:

• Worked closely with business stakeholders to assess and understand reporting needs for reverse customer data.

• Designed and implemented incremental refresh in Power BI to ensure daily updates without full data reloads.

• Collaborated with Azure data engineers to optimize data processing and publication pipelines.

• Stakeholder communication & data modeling:

• Acted as liaison between Group Data Office and Technology Office to align data modelling standards.

• Gathered requirements from data engineering team and participated in weekly status meetings to provide implementation updates and resolve blockers across teams in Germany, Poland, and India.

• Documentation & quality assurance:

• Prepared end-to-end technical design documentation, data flow diagrams, and Power BI audit guides for future reference.

• Participated in UAT sessions with business users to validate data outputs and report accuracy.

• Implementation, maintenance and operation of an ERP system and a document exchange platform for a corrugated cardboard manufacturer.
• Tools and systems: Unix (Debian 6.x), C, SVN, Windows, C#, MS SQL Server, SCRUM.