Senior Data Engineer
[online recruitment]
[online recruitment]
Location: Warsaw
Reference number: VR-54566
Project Description:
As a part of data migration into Azure Cloud - extract data from Oracle Hive cluster, convert into Parquet format and load to the Azure Blob Storage.
Utilize Spark for further data manipulations and analysis in a distributed manner.
Responsibilities
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure 'big data' technologies;
- Implement data flows connecting operational systems, BI systems, and the big data platform;
- Build real-time, reliable, scalable, high-performing, distributed, fault tolerant systems;
- Clean and transform data into a usable state for analytics. Build data dictionary;
- Create data tools for analytics and data scientist team members that assist them in their ML endeavors;
- Design and develop code, scripts and data pipelines that leverage structured and unstructured data;
- Implement measures to address data privacy, security, compliance.
Skills
HiveQL, Scala, Java, Apache HBase, Python, Kafka Streams, Big Data, Apache Kafka, Hadoop
- Experience with designing data and analytics architectures in Microsoft Azure cloud;
- Experience with Big Data technologies like Spark, Hadoop, Hive, HBase, Kafka etc.;
- Fluency in several programming languages such as Python, Scala, Java, with the ability to pick up new languages and technologies quickly;
- Experience with data warehousing, data ingestion, and data profiling;
- Demonstrated teamwork, strong communication skills, and collaborative in complex engineering projects.
Our offer
Career and Professional Development
- International projects for clients all over the world
- Competitive salary
- Individual development plan
- Managerial Targeted Training programs
- BRIDGE Mentoring Program
- Luxoft Training Center
- Language Classes
- Self-learning online library
- Global Relocation Program
- Internal Mobility (a chance to gain experience in varied projects and technologies)
- Professional communities for knowledge-sharing (Agile, Tech, Business)
- Group Life Insurance
- Travel Insurance
- Private Healthcare (dental care, unlimited consultations of specialist physicians)
- Medical costs reimbursement for employees
- Benefit Program (Cafeteria and Multisport Card)
- LuxGood Program (a wide range of health and well-being initiatives)
- After-hours groups (sport, trips, board games, cultural activities)
- Company and Team events
- BeLux - discount offers program (banking, car leasing, other)
- Convenient locations in modern offices
Career and Professional Development
- International projects for clients all over the world
- Competitive salary
- Individual development plan
- Managerial Targeted Training programs
- BRIDGE Mentoring Program
- Luxoft Training Center
- Language Classes
- Self-learning online library
- Global Relocation Program
- Internal Mobility (a chance to gain experience in varied projects and technologies)
- Professional communities for knowledge-sharing (Agile, Tech, Business)
Well-being
- Group Life Insurance
- Travel Insurance
- Private Healthcare (dental care, unlimited consultations of specialist physicians)
- Medical costs reimbursement for employees
- Benefit Program (Cafeteria and Multisport Card)
Work-Life Balance
- LuxGood Program (a wide range of health and well-being initiatives)
- After-hours groups (sport, trips, board games, cultural activities)
- Company and Team events
- BeLux - discount offers program (banking, car leasing, other)
- Convenient locations in modern offices
Please send your CV with consent for processing your personal data for recruitment purposes using the below button
APPLY