RLHF Experts in Germany

Conventional Paradigm Test (CPT) – AI Evaluation & LLM Research

Development of an experimental evaluation approach to examine “paradigmatic closure” in Large Language Models — that is, the question of how far LLMs can recognize the basic assumptions, values, and limits of the paradigms within which they generate answers.

Design and testing of an additional approach to classic AI benchmarks that does not primarily measure factual correctness or task performance, but instead examines a model’s ability to recognize alternative perspectives, make implicit assumptions visible, and reflect on the limits of its own answer or interpretation framework.

Focus areas: development of evaluation criteria and test questions · LLM evaluation and comparative model analysis · prompt and response analysis · qualitative classification of model answers · study of epistemic compression and value leakage · benchmark and literature research · development of structured assessment and analysis methods

As part of CPT, existing AI evaluation approaches and benchmarks were analyzed, and a minimalist test protocol was developed that classifies model answers by response patterns such as DIRECT, CLARIFY, PLURALIST, REFUSE, and META-AWARE. TruthfulQA was used as the basis for experimental application and comparison with existing reference answers.

Technologies & Methods: Large Language Models (LLMs) · Generative AI · Prompt Engineering · AI Evaluation · TruthfulQA · Benchmark Analysis · Human-in-the-Loop Evaluation · Qualitative Content Analysis · Research & Literature Review

Project: Digitalization of Warehouse Processes | Building a Data Analytics Platform.

• Built a web-based workforce allocation system that digitized daily shift planning by matching worker expertise to operational zones, replacing manual coordination with a structured workflow adopted across the site, saving supervisors time on daily planning.
• Developed a real-time operational visibility dashboard giving supervisors a live view of task throughput and outstanding workload across warehouse zones throughout the day, helping reduce overtime and idle labour costs.
• Developed a slotting optimization solution to improve warehouse picking efficiency and reduce picking time per order, working directly with operations teams from concept through production deployment.

Technologies used: Python, Django, PostgreSQL, Pandas, NumPy, HTML, Java, JavaScript, Docker, Kubernetes, AWS, Power BI, GitHub Actions CI/CD, GitOps, Claude, OpenAI

• Architected and deployed a production-scale GraphRAG system using Neo4j, embeddings, and multi-hop reasoning over 120M+ nodes, improving answer precision by 32%, reducing hallucinations by 41%, and lowering retrieval latency by 38%.
• Designed and implemented an enterprise agent ecosystem using Model Context Protocol (MCP), exposing internal APIs, databases, and services as secure callable tools for autonomous workflows and system integration.
• Designed and deployed a production LLM-based email routing agent using Microsoft Graph API, MCP, and Azure OpenAI, achieving 96% routing accuracy, reducing manual triage workload by 65%, and decreasing response times from 18 hours to under 4 hours.
• Implemented autonomous agent self-correction pipelines using iterative feedback loops (Ralph Wiggum), enabling reliable error detection, automated remediation, and production-safe execution.
• Developed a multimodal semantic search platform using multimodal LLMs and vector embeddings, enabling semantic discovery across 250k+ image and video assets and improving search recall by 48%.

• Lead Product Owner for 4 development teams
• Leading and coordinating a greenfield project with parallel implementation of core components by independent teams; managing dependencies and resources
• Establishing a data lakehouse approach, including analysis of data volumes and future requirements as part of a cloud migration (best-of-breed approach)
• Responsible for requirements analysis, selection, and piloting of a LIMS/ELN system, supported by advising decision-makers and managing external vendors
• Introducing and managing an OpenWeb UI and Azure OpenAI-based RAG system to support knowledge extraction and data-driven analyses
• Setting up, configuring, and managing Jira projects, as well as developing project-specific workflows and automations
• Implementing classic Scrum processes with all ceremonies and taking on the Scrum Master role for all involved teams
• Assisting in hiring through interviews and assessments from a product owner's perspective
• Making key architectural decisions, including selecting the platform for the data lakehouse (Databricks) and the strategic integration of LIMS and analytics platforms

• Conduct structured evaluation of LLM outputs using Content Review Standards (CRS) and AI safety frameworks.
• Assess responses across high-risk domains including violence and criminal facilitation.
• Assess responses across high-risk domains including hate speech and harassment.
• Assess responses across high-risk domains including suicide and self-harm.
• Assess responses across high-risk domains including regulated advice (medical, legal, financial).
• Assess responses across high-risk domains including misinformation and fabricated claims.
• Assess responses across high-risk domains including defamation and intellectual property.
• Assess responses across high-risk domains including child safety and sexual exploitation.
• Assess responses across high-risk domains including political and sensitive content.
• Apply youth-protection and age-appropriateness guidelines to prevent unsafe facilitation or restricted substance guidance.
• Classify prompts as adversarial, borderline, or benign based on contextual intent and risk analysis.
• Evaluate model behavior types including correct refusal, partial refusal, over-refusal, under-refusal, improper compliance, and ignorance-based outputs.
• Identify policy misapplications and user-intent misinterpretation patterns.
• Designed structured adversarial and borderline multi-turn conversation flows to stress-test AI boundary enforcement and reasoning stability.
• Identified failure modes including hallucination, unsafe compliance, excessive refusal, contextual drift, and inconsistent safety logic.
• Applied a structured four-dimension evaluation rubric covering accuracy & safety, relevance & completeness, clarity & structure, and tone & appropriateness.
• Provided structured feedback supporting supervised fine-tuning and reinforcement learning from human feedback processes.
• Rewrote unsafe or misaligned outputs into compliant, accurate, and helpful responses.
• Performed Persian ↔ English translation and translation validation of AI-generated content.
• Assessed semantic accuracy, contextual consistency, and safety alignment across languages.
• Identified mistranslations, cultural nuance issues, and cross-lingual policy inconsistencies.
• Recognized with the Above & Beyond Award – Q3 2025 for exceeding quality standards and embracing innovation.

• Designed and evaluated high-quality prompts and completions for Large Language Models (LLMs), focusing on improving response accuracy, instruction-following behavior, and factual consistency.
• Annotated and rated LLM-generated outputs for grammar, coherence, relevance, and truthfulness.
• Developed RLHF-style preference data by ranking model completions to inform reinforcement learning fine-tuning cycles.
• Participated in prompt engineering experiments to assess the effect of instruction format, verbosity, and phrasing on model behavior.
• Conducted error analysis and quality assurance on large-scale NLP datasets, identifying edge cases and linguistic ambiguity affecting LLM performance.

• Conceived, built and iterated an applied-AI MVP that turns in-depth audio interviews into structured, long-form narrative outputs across multiple genres (e.g. memoir, institutional knowledge, thematic essays) using agentic orchestration and multi-step reasoning.
• Designed and implemented core workflows in a Next.js-based stack, working with structured representations (JSON and other formats), retrieval-augmented generation and emerging knowledge graph structures to maintain context and consistency over long documents.
• Defined and tested agent behaviors across realistic storytelling scenarios, including ideal user journeys, edge cases and failure modes, with explicit criteria for coherence, factual alignment and user intent satisfaction.
• Currently running targeted user tests with selected partners to validate use cases and inform the next product iterations.

• Created and refined technical and compliance-oriented datasets for AI, ensuring high-quality structured documentation.
• Conducted supervised fine-tuning (SFT) and RLHF tasks, maintaining strict alignment with industry and security guidelines.
• Produced detailed technical reports and feedback for audits and QA teams.
• Collaborated with cross-functional teams on documentation strategies for large-scale AI deployments.

• Independently write, edit, and publish 50+ page study guides for major literary works, including detailed chapter summaries, character breakdowns, quote analyses, and thematic reviews tailored for diverse audiences
• Translate complex literary content into structured and accessible resources through deep research, synthesis, and editorial precision within editorial deadlines

• Specializing in coaching and consulting for technical transitions and business development.
• Empowering leaders to increase team performance in agile, high-pressure environments.
• AI Training & Optimization: Leveraging RLHF (Reinforcement Learning from Human Feedback) to optimize technical KI models and prompt engineering.