Job Description
Position overview:
CHAI currently seeks a Technical Advisor (TA), Data Engineer to work with the Ministry of Health (MOH) National Health Intelligence Center to help improve the design, building, and maintenance of systems to collect, store, and analyze large sets of structured and unstructured data. The role involves working with data pipelines, databases, and big data tools to ensure that data is accessible, reliable, and optimized for use by data scientists, analysts, and other stakeholders. They will be seconded to the National Health Intelligence Center (NHIC) and will report in parallel to CHAI, Program Manager, Digital Health for specific CHAI supported initiatives.
The TA, Data Engineer, will provide need-based technical assistance during the review and implementation of a new data analytics architecture at MOH/NHIC. This effort is a cornerstone to MOH’s goal to disrupt how data is managed and used, including big data, to inform important policy and operational decisions at all levels of implementation.
The TA, Data Engineer will help design and implement the framework for improved data architecture, and governance and build capacity within the MOH and the Health Intelligence Center (NHIC). In addition, the incumbent will work closely with the digital team at MOH to incorporate and translate data needs into system requirements.
Responsibilities
The key functions and deliverables of this role will include:
Data Engineering Core
- Design, develop, and maintain scalable data pipelines for ingesting, processing, and storing large volumes of health data.
- Build and optimize ETL processes to ensure smooth data flows from source systems to storage and analytics platforms.
- Work with various database technologies (SQL, NoSQL, columnar, etc.) to manage, query, and optimize data effectively.
- Integrate data from internal systems, external APIs, and third-party services, ensuring accuracy, consistency, and quality.
- Develop and maintain a data catalogue and data dictionary to support effective data management.
Infrastructure & DevOps
- Architect, build, and manage scalable cloud-based infrastructure (AWS, GCP, Azure) including compute, storage, networking, and security.
- Implement Infrastructure as Code (IaC) solutions using tools such as Terraform and Ansible for consistent and repeatable deployments.
- Administer Kubernetes clusters across environments (development, staging, production), including setup, scaling, and advanced configurations (operators, custom resources, RBAC, service mesh).
- Develop and manage GitOps workflows using ArgoCD (or similar), ensuring declarative deployments and configuration management.
- Design and maintain robust CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins, etc.) with automated testing, artifact management, and deployment strategies (blue-green, canary).
Security, Compliance & Monitoring
- Implement data governance, security, and privacy protocols to ensure compliance with regulatory frameworks (GDPR, CCPA, and relevant national standards).
- Monitor and troubleshoot performance of data systems, optimizing workflows to reduce latency and improve reliability.
- Deploy observability solutions (Prometheus, Grafana, ELK, distributed tracing) for system health and performance monitoring.
- Ensure platform and data security through access controls, certificate management, vulnerability scanning, and adherence to organizational security policies.
Data Quality & Advanced Analytics
- Establish and enforce data quality frameworks to ensure the accuracy, integrity, and reliability of data used in decision-making.
- Recommend and integrate tools to monitor and improve data quality within the health data ecosystem.
- Support the development of predictive and prescriptive analytics capabilities using advanced analytics and machine learning tools.
Capacity Building & Stakeholder Engagement
- Work closely with data scientists, analysts, and business stakeholders within the National Health Intelligence Center to deliver clean, usable datasets for analysis, reporting, and modeling.
- Develop training programs and resources to keep data teams up to date with the latest advancements in data engineering and public health informatics.
- Engage with stakeholders across the health sector — including government agencies, NGOs, and international partners — to facilitate data sharing, interoperability, and collaborative analytics.
- Document pipelines, infrastructure, and processes to ensure transparency, knowledge transfer, and long-term maintainability.
Continuous Improvement
- Stay current with emerging technologies, tools, and best practices in data engineering and public health data systems.
- Continuously evaluate and improve data architecture, platforms, and workflows to increase efficiency, scalability, and impact.
- Perform other tasks as assigned by the program manager to support the overall success of the program.
Qualifications
- Master’s degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience).
- Minimum of 6 years of proven experience in software or data engineering, including building and maintaining data systems and pipelines.
- Strong proficiency in SQL, with experience across relational and non-relational databases (e.g., PostgreSQL, MySQL, MongoDB).
- Hands-on experience with data modeling, dimensional modeling, and ETL tools (e.g., Apache NiFi, Talend, Airflow).
- Practical experience with cloud platforms (AWS, Google Cloud, Azure) and big data frameworks (Hadoop, Apache Spark, Kafka).
- Expertise in GitOps, Kubernetes, infrastructure automation (Ansible, Terraform, Puppet/Chef), security and monitoring, and scalable platform architecture for multienvironment deployments.
- Familiarity with modern data warehousing solutions (Snowflake, Redshift, BigQuery).
- Strong analytical and problem-solving skills, attention to details, with a focus on data quality and system reliability.
- Effective collaborator with cross-functional teams, with excellent written and verbal communication skills.
Disclaimer: Applications will be received until this position is filled
