Our client - a company, which technologies have been built to extract maximum value from the search for brands, partners and businesses all over the world and also to innovate & improve the consumer experience - is now looking for Senior Big Data Engineer to join their team in Kyiv.

The company’s world-class engineers, semantic specialists, and product teams are building the future of Search and as part of the Engineering Team, you will play a key part in developing its offering.

The Big Data team is enabling the optimal storage, organisation and timely retrieval of the data. They are responsible for optimisation of ETL pipelines, maintaining over 60 Spark jobs. Building a data lake for data scientists and analysts.

 KEY RESPONSIBILITIES

• You will help architect and build solutions to business-critical problems.
• You will be participating in interesting projects such as:
• Carrying out efficient integration with the data providers via various API endpoints and data representation formats.
• Building and deploying an in-house distributed ETL pipeline for processing petabytes of data per day.
• Enable accurate, comprehensive and reliable data storage in our distributed data warehouses based on the needs of other teams.
• Providing continuous improvements in the way data is being processed and stored based on the feedback and needs of the business or other teams.
• Setting up monitoring for key performance metrics and overall systems' behaviour to promptly react in case any anomaly detected.
• You will be responsible for optimisation of ETL pipelines, maintaining over 60 Spark jobs. Building a data lake for data scientists and analysts.
• Experimenting with new tools and technologies to produce cutting-edge solutions to business problems.
• Be a part of a self-organising, results-oriented agile team using Kanban to complete new product launches.

 WHAT YOU SHOULD HAVE:

• 5+ years of experience in software development and have exceptional problem-solving skills.
• Solid hands-on Scala programming language.
• Solid hands-on Apache Spark framework.
• Solid hands-on Apache Spark SQL.
• Proven production experience with at least one of the following: Apache Impala and/or Presto and/or Amazon Athena.
• Proven working record with RDMS.
• Proven production experience with at least one of the following resource-sharing frameworks Apache Mesos and/or Kubernetes and/or Yarn.
• Proven production experience with monitoring tools and distributed logging frameworks.
• Proven production experience with at least one of the cloud services providers AWS Cloud services and/or Google Cloud services.
• Unix OS family advocate: shell scripting and essential commands knowledge.
• Familiarity with Docker.
• Passion for following best industry practices: TDD, BDD, CI/CD etc.
• Understanding and working production experience with HTTP/HTTPS protocols; ability to setup and deploy a simple web server if needed; 3d party HTTP/HTTPS integrations.
• Proven experience of building production-ready ETL processing pipelines.
• Proven track record of building integration with third parties via various APIs.
• Good written and verbal English communication skills.

WOULD BE A PLUS:

• Experience with striming frameworks Apache Kafka and/or Apache Flink.
• Experience with distributed key value stores: Aerospike, HBase etc.
• Experience with Python and JavaScript.
• Experience with workflow frameworks (e.g. Apache Airflow etc.).
• Some experience in the AdTech sector is advantageous.
• Previous experience in Bank or any Accounting field in a role of analyst is an advantage.
• Experience with working in an international team.

COMPANY OFFERS:

• Flexible working.
• Medical Insurance.
• Snacks and drinks in the office.
• Sharing culture and opportunities to learn from each other's projects.
• Diversity.
• Courses, training, conferences.
• 20 days vacation, plus 2 bonus days off including your birthday and 11 of the Ukraine public holidays off.
• Accountant access.