The ML Engineering team within the Marketing Data Science team at Wayfair develops scalable data-processing platforms and deploys hundreds of machine learning models that power algorithmic decision-making across many marketing channels and customer touchpoints. While data science is at the heart of everything we do, it is our ML engineering team that enables us to scale our impact and quickly turn our ideas into models, models into decisions, and into new customer experiences. Our team aims to redefine how we think about label and feature generation, model development, deployment, and monitoring.
The Senior ML Engineering Lead will be responsible for growing and leading a lean and mean high-performance engineering team that deploys and maintains 500+ ML models scoring 100M+ customers/devices on a daily basis. The portfolio of ML model and analytical services our team develops powers the way millions of customers interact with us. This role represents significant technical, analytical, and leadership opportunities with enormous potential for business impact. You will also have the opportunity to dive deep into the technical details, future-proof our ML tech stack, and pioneer novel platforms while simultaneously defining and communicating out your technical vision to the broader organization.
If you are a seasoned engineering leader, a continuous learner and technology enthusiast, and enjoy working on solving complex real-life problems using massive datasets - this could be the one! Above all, youll get to work on problems that are both intellectually-challenging and drive real, measurable impact -- first and foremost, for our customers -- and as a result for Wayfair at large. To get a better sense of the type of projects we actually work on, check out our Data Science & Machine Learning blog posts here!
What You'll Do
- Lead, develop, and grow a team of highly skilled data & machine learning engineers
- Own the technical vision, architecture, and system design for your portfolio of services
- Build highly scalable distributed data processing platforms & low-latency modeling scoring services for 100s of ML models powering millions of daily marketing decisions
- Extend our existing ML libraries, technologies, and frameworks that enable data scientists to quickly train and deploy new ML models & online learning systems to prod
- Serve as an engineering thought-leader for the DS&ML organization and foster a culture of that values innovation, scalability, decoupled microservices, & continuous deployment
- Pressure-test and future-proof our data science & ML tech stack; be a pioneer in assessing and deploying new & emerging technologies that show promise
- Work alongside the DS&ML leadership team to define and pressure-test team vision
- Build strong x-functional partnerships with various engineering leaders across the org to ensure leverage existing platforms & identify any gaps/risks to delivering our roadmap
- Navigate organizational complexity & execute against bold new ideas
Examples of some Existing & Upcoming Products:
- Mercury: a highly-scalable signal processing & feature generation platform that reliably generates 20k+ features with built-in consistency b/w training & production scoring. We utilize a lambda architecture and a pub-sub paradigm to enable rapid model training.
- Batch Model Scoring Service: responsible for scoring 500+ ML models, each using up to 20k input features, for 100M+ customers/devices on a daily basis
- Label Store: a technology that enables auto model retraining, auto calibration, and auto performance monitoring. Data scientists simply register their label-generation code using an intuitive DSL (domain specific language) & the rest is taken care of.
What You'll Need
- 6+ years of combined experience, with 3+ years as an individual contributor (SDE, Data Engineer, or MLE) and 3+ years managing a team of engineers, MLEs, and/or scientists
- Demonstrated experience leading teams and deploying ML solutions for environments with massive datasets (technology, ad-tech, fintech, anything with billions of records)
- A manager do-er. This is fairly new/small team that has made significant progress on a few key initiatives but require strong engineering leadership to get us to next level
- You enjoy getting into the weeds and providing hands-on technical mentorship
- A good balance between pragmatism / bias for action vs. developing solutions that scale
- Experience working in a high-growth environment is a plus
- Strong OOP programming skill, experience in Python, Java, etc., and some exposure to ML libraries (numpy, sklearn, tensorflow, ML-LIB)
- Significant experience working with distributed computing and big data technologies (Hadoop, Spark, Hive, SQL, Yarn); some experience w/ streaming data (kafka or spark)
- Experience with most of the following: CI/CD (ex Jenkins), version control (ex Git), DAGs (Airflow, Luigi, Kubeflow, etc.), artefact management, and containerization (Docker, etc.)
- Ability to communicate across multiple technical teams with demonstrated experience navigating a complex organization, generating buy-in, and removing roadblocks
It's a bonus to have:
- Experience with cloud or hybrid infra
- Experience working with streaming data (ex: Kafka or Spark Streaming)