Experienced Java & Python developer, specializing in Big Data Software. Eager for new knowledge, discovering best practices and bringing innovation to the team.
Development of backend components in Python and Scala integrating with PostgreSQL and Presto.
Implementing a scalable Data Streaming solution with Flink on Java integrating with Kinesis and Kafka.
New methodologies and best practices pusher, mentoring the Junior team members.
Senior Backend Engineer @
Natural Intelligence
In Natural Intelligence (NI) I had the role of a senior engineer in the team.
During the assignment I built ETLs using Spark technology in Scala language, managing spark jobs, watching them with it’s resource manager, writing ad-hoc etls in Zeppelin, etc.
In addition, I implemented multiple batch jobs on top of Spring Batch which fetched data from 3rd party APIs, wrote it to AWS S3 service, integrating with AWS Aurora. This part was implemented in Java. There I saw that writing a clean maintainable code is a weakness in the team, leading to me starting to mentor the team in terms of better and cleaner code and design.
Also, had an interaction with Airflow as their prime scheduling mechanism.
The first assignment was to integrate all their java service with Datadog monitoring and logging service.
So to summarize:
Spark with scala
Java batch jobs with Spring Batch
Airflow
AWS cloud infrastructure (EMR, Aurora, Redshift, and more)
Mentoring the team about cleaner code, better testing, etc.
Monitoring and log shipping to DataDog