Senior BigData Transformation & ETL Engineer
Luxoft is a global leader in high-end software development.
Luxoft is looking for talents with a passion for technology & ready to create original solutions. Once on board, you are invited to expand your knowledge & skills, offering you a continuous learning experience helping you stretch your potential.
So if you’re enthusiastic by the idea of accessing cutting edge technology & innovation to make an impact, why don't you join us?
Senior BigData Transformation & ETL Engineer
Miejsce pracy: Warsaw
Nr Ref.: VR-31709
Responsibilites
- Designs, develops and modifies code based on functional and system requirements
- Participate in testing process through unit and functional testing
- Perform development tool and environment design and configuration
- Support bug fixes, change requests and functional enhancement on the code
- Validates scope, plans and deliverables for assigned projects
- Responsible for overall technology direction for the assigned module/project
- Interacts with Design team and Test team for requirements and defects
- Write reusable, testable and efficient code
- Design and implementation of data processing systems and software
- Implementation of low-latency and highly available APIs
- Integration of existing databases in processing pipeline
- Support with implementation of user interfaces
- Follow the change and Incident Management process
Requirements
Essential:
- Minimum 3+ years’ experience in a similar role, preferably within financial services
- Fluent with SQL, good knowledge of BigData ETL tools like HIVE
- Develop, debug, monitor, and maintain a CI/CD pipeline that feeds into a production deployment, plus one or more staging and testing environments
- Hands on Big Data: RHEL Linux, Cloudera CDH, DevOps tools and Python scripting
- Ensure that our architectural designs are future-proofed, from a scale, load, and performance perspective
- Kerberos, LDAP
- Strong ability to work independently
- Outstanding problem solving skills
- Comfortable working in an Agile environment
Important:
- A computer science or related degree
- Bonus points for experience with Apache AirFlow, Saltstack, Ansible
- Spark
- Subversion
- PaaS experience
Useful Technical Skills:
- AWS, MySQL, SQL DBA, Bash/Python scripting
- Any Tomcat, Sensu, Nginx, MongoDB would be advantageous
- Continuous Integration
We offer
We are looking for a hands-on Big Data Transformation and ETL engineer to work alongside other developers and QAs in an Agile team. The candidate should be a proven problem solver; someone who enjoys and embraces the variety of tasks a role like this provides.