Staff Data Engineer

About the role

This is a Staff Data Engineer (Data Lake) opportunity with a Series C New York healthcare data company whose platform gives life sciences organisations data and AI-driven insight on doctors and trials. You’ll design and scale distributed ETL and ELT pipelines and ingestion frameworks for structured and unstructured healthcare data, evolve the data lake architecture for scale, reliability and cost, improve data quality and normalisation across thousands of global sources, build batch and near-real-time processing on cloud-native platforms, optimise Spark workloads, and mentor engineers and lead architecture reviews. The stack includes Python (PySpark), Java, Scala, SQL, Spark, AWS (EMR, Glue, S3, Athena, Redshift), Argo or Airflow, Docker and Kubernetes. The ideal candidate will have 8+ years in data or software engineering building distributed data platforms, advanced SQL, deep Spark experience and technical leadership. Large-scale data cleaning and healthcare, life sciences or entity-resolution datasets are preferred. This is a hybrid position in New York City. We're looking for someone curious, pragmatic, and eager to work across the stack with a small senior team. You'll own end-to-end delivery of major features, influence architecture, and mentor engineers.

What you'll do

• Ship customer-facing features on a weekly cadence.
• Partner with design and product on scoping and trade-offs.
• Drive technical quality through review, testing, and docs.
• Mentor one or two engineers on your team.

Requirements