Goal of the Project:
- Implement a scalable cloud-based data platform to enable sustainable, reusable, secure, efficient ingestion, transformation, and storage of enterprise data across multiple sources.
- Improve data consistency, reliability and controlled accessibility by automating ETL/ELT workflows, enforcing data quality checks, and reducing manual intervention by 30–40%.
- Knowledge and experience of near real-time analytics capabilities to support business intelligence, predictive modeling, and faster decision-making for strategic initiatives.
Primary Responsibilities:
- Optimize, operate at scale and enhance cloud-based data platform on Microsoft Azure (incl DataBricks)
- Involve in cloud data platform enhancement project lifecycle with external vendor including design, development, testing, deployment, and documentation.
- Work with Data Analyst to perform data preparation, cleansing, build optimized data models and visualization in Power BI
- Perform proof of concept for cloud data products
Secondary Responsibilities:
- Validate data workflows and integration points through rigorous testing (e.g., pipeline validation, schema checks, and performance benchmarking) to ensure solutions meet business and technical requirements.
- Create operational playbooks and automation scripts to streamline deployment, monitoring, and troubleshooting, enabling efficient handover and long-term maintainability of data solutions.
- Document data pipeline architecture, deployment processes, and operational runbooks to support effective troubleshooting and post-go-live maintenance by internal teams
Additional Beneficial Data Knowledge:
- ERP
- CRM
- Marketing
- Asset Management
- Construction