About the Role
We are seeking a
Data Engineer / Tech Lead
with a strong background in
real-time data streaming
to lead the design and delivery of cutting-edge streaming solutions. This role combines
hands-on engineering
with
technical leadership
– you will both architect scalable data streaming systems and guide a team to successful implementation.
Key Responsibilities
Act as
technical lead
for projects with a strong streaming/data engineering focus.
Design
end-to-end architectures
for data streaming use cases, ensuring scalability, reliability, and performance.
Implement
Change Data Capture (CDC)
and other real-time extraction mechanisms.
Drive adoption and best practices in
Confluent Kafka
and
Spark Structured Streaming.
Lead the development of streaming pipelines using
Python, SQL, Bash, Node.js, and Linux/Unix Shell.
Oversee
code quality, versioning, release management, and CI/CD processes.
Guide the team in applying
Dev Ops and cloud-native tools : Git Hub Actions, Terraform, Kubernetes, etc.
Collaborate with business stakeholders, data scientists, and platform teams to align streaming solutions with organizational goals.
Mentor junior/mid-level engineers, fostering growth and technical excellence within the team.
Essential Skills & Experience
Proven experience in data engineering with a strong focus on streaming.
Strong expertise in Confluent Kafka
and
Spark Structured Streaming.
Hands-on experience with CDC and real-time data extraction.
Solid programming background:
Python, SQL, Bash, Node.js, Linux/Unix Shell.
Strong knowledge of data engineering lifecycle : versioning, release management, deployment, agile delivery.
Experience with
cloud-native & Dev Ops tools : Git Hub Actions, Terraform, Kubernetes.
Previous experience in a
technical leadership role
(designing solutions, guiding teams, mentoring).
Nice to Have
Multi-cloud exposure (AWS, Azure, GCP).
Knowledge of wider
big data & analytics platforms
(Databricks, Snowflake, Delta Lake).
Experience in stakeholder management and solution consulting.
