For our Client, global organization with strong brands and position on the market, we are seeking to recruit a Data Engineer.
As a Data Engineer for Finance Data Operations you will build robust, fault tolerant, near real-time, and scalable data pipelines to supply our Finance community with operational, analytics and reporting ready data.
Data Engineer
Miejsce pracy: Warszawa
Key responsibilities include:
- Design, build, test and operate robust and scalable data pipelines for streaming and batch data;
- Design, build, test and maintain a scalable and reliable data repository for operational, analytical and reporting uses;
- Develop best practices and frameworks for automation and CI/CD for data pipelines;
- Refactor legacy pipelines for modern infrastructure and frameworks;
- Create and maintain an optimized cloud based infrastructure;
- Embed end-to-end security and privacy;
- Tenaciously solve problems and make decisions under uncertainty;
- Actively contribute to team ceremonies including: sprint planning, daily stand-up, sprint review and retrospectives;
- Demonstrate a strong ownership in understanding business needs and driving business valued outcomes;
Required qualifications:
- University degree in a quantitative concentration: Computer Science, Engineering, Mathematics, Economics or equivalent;
- 3+ years of designing, building, testing and operating data pipelines in AWS;
- 3+ years of hands on experience building data pipelines with Python, Scala, or Java;
- 3+ years of hands on experiences building on the Hadoop stack especially Spark;
- 3+ years of experience modelling data in normalized, denormalized and graph structures;
- Experience developing and operating workflow orchestration (Apache Airflow);
- Excellent proficiency with Python, SQL, PySpark, Spark SQL;
- Strong experience with schema design and data modelling;
- Fluent in English;