Engineering Spotlight, is a recurring series on VentureFizz. It's a deep dive into a Boston tech company's engineering team and what it's like working there.
Each spotlight gives you an inside look at the company's technical stack, the complexity of problems the engineers get to solve, the team's culture, what to expect for an interview and more.
Below is a list of the companies we've featured so far this year along with a couple of snippets from their Engineering Spotlight feature.
"We’re using a lot of different technologies but AWS is probably the biggest with how we’re scaling. Our engineers are focused on finding ways to ingest security data at a petabyte scale and identify performance analytics to drive intelligence and value for our customers. There are a lot of challenges at the petabyte scale and those challenges include efficiency, performance, and costs. It requires really ingenious technical solutions, such as AWS."
"The security space is really interesting in so many ways. Here is one project that illustrates this: With security being so visible now with high-profile breaches and even talk of election tampering, customers are really interested in getting the best possible security capabilities for their organizations. But security is also a really nuanced subject, with lots of details and this can be a high barrier to entry to some customers. This makes for a really fun project as we try to combine technology and user experience (UX) to make security both effective and accessible for customers."
Focus for Facebook Boston's Engineering Team:
"Our Boston team is currently focused on five areas, all of which impact Facebook's 10-year roadmap and mission: 1) Connectivity Lab, 2) Developer Tools, 3) Location Infrastructure, 4) Traffic, & 5) Machine Learning"
"The people who come to work at Facebook Boston are passionate about working on hard, technical challenges at scale. This passion translates into a culture that's driven by collaboration, and a desire to make an impact. Our engineers genuinely enjoy working with the world's best minds in creating unique and enjoyable experiences for more than 2 billion people around the world.
This culture of collaboration is one of the many reasons why Glassdoor named Facebook the #1 Best to Work in 2018. One Boston review for this Glassdoor award spoke to smart coworkers and having the ability to work and succeed in areas that will help the company. "
"Healthcare data is notoriously messy and this problem is amplified when collecting data from various sources. One of the interesting challenges that the Engineering team has to tackle is to normalize all this data into a usable repository. Another aspect to our work is Patient Linking - the ability to identify patients between data sources in a HIPAA compliant matter is a very complex and challenging problem being researched and fine-tuned in academia.
Defining and calculating patient outcomes at scale is an inherent challenge to OM1’s mission.
We also tackle how to visually represent that patient data in a repeatable, insight-driven way that is digestible to end users."
Product Planning Process:
"We use an Agile process with two-week sprints with a traditional sprint planning meeting to finalize commitments for the upcoming sprints. Prior to that, product owners groom the backlog with Engineering. In that grooming, engineers work closely with product owners to understand the value brought to users and can offer their creativity."
Development Process on Starry's Tech:
"Starry Station, our take on the router, took about a bit over a year to develop. The software and hardware comprising our fixed wireless Internet system went through roughly two-and-a-half years of research and development."
Databases: MongoDB, Redis, Elasticsearch, Graphite/Whisper
Other: AWS, Linux, Docker, Kubernetes, Salt, Nginx, HAProxy"
How Cross-Functional Teams Are Structured:
"Our organization is broken into three major units: Brand/Design (physical product development and brand story/content), Growth (business success, customer understanding, and performance marketing), and Platforms (scaling the business through technology, process, and people systems). Within each business line, members from each of these teams work together to accomplish growth in specific parts of our business. For instance, for our wedding invitation business, we have a General Manager who is responsible for growing the business profitably, wedding concierges that assist couples in designing the perfect invitation, a product manager, and a small engineering team colocated. They are all responsible for driving the marketing, roadmap, and technology-focused specifically on the wedding consumer.
Within technology, we follow an API driven strategy. We are always focused on the consumer and looking to find ways to help people give more and better, but we are also focused on finding shared patterns/architecture that can be used across all of our business lines. To accomplish this we have started to specialize “squads” within our teams that focus on our selling platforms (web applications, customer workflow services) and our capabilities (e.g. 3D rendering, customer modeling). The platform teams manage their own backlogs working closely with the product managers of the multiple consuming teams and building new services/APIs to support the needs of all of the other teams."
"We are all pretty passionate about the product we are building at Lovepop which doesn’t leave a ton of time for side projects. However, we do make an effort to contribute back to the community however we can which takes a variety of forms - we will PR against open-source projects where it makes sense but we also are making an effort to get out to meet-ups this year and share some cool solutions we have come up with. Meetups we particularly like are the ReactJS Boston and Boston EmberJS Group."
Thoughts on Email Still Remaining Dominant:
"Email is an incredibly effective and growing communication channel. Email clients and ISPs continue to give consumers more choice and better tooling to optimize how they consume their personal stream of emails. At Klaviyo we are very focused on ensuring our customers see very high engagement rates with their campaigns. Our goal is to have the world send one less spammy email and for our customers to have long-term relationships with their own customers.
Internally we celebrate our users with the most engaging content and successful marketing strategies. Our product features and support teams actively coach customers to send a better email and we routinely fire customers which abuse email privileges."
"You should expect to really be pressed on the details of interesting projects you’ve worked on and your personal contribution. We like engineers that know their numbers and can go deep on the various components and technical decisions that comprised a feature.
Our interviews are a combination of collaborative coding and whiteboarding exercises. We also look for engineers that are passionate about customers and have experience or at the very least passion for building features that improve the lives of end users.
Every new engineer is expected to level up Klaviyo in some way so in the interview we look for how they raise the bar for Klaviyo engineering."
"Privy first came to be as a Rails 3 app. Just over seven years later, after several major Rails upgrades, we're comfortably serving over 1.6B requests per month on Rails 5.1.
Our current frontend stack revolves around React & Redux (ES6), which interfaces with an older Backbone application written in Marionette.js. Frontend assets are tested with Jest andMocha, and bundled using Webpack.
On the backend, in our Rails app, the main technologies we use are PostgreSQL, Redis, MongoDB, Sidekiq, and elbow grease.
All our engineers use Docker for development, with SSL support for development/production parity following the principles of The Twelve-Factor App."
Engineering Team Culture:
"Lots of freedom accompanied by strong accountability, in a very tangible way: build something poorly, and you'll be inundated with bug reports and change requests from customers and the internal team; getting bitten by that once or twice builds up some scar tissue. It's more fun to get things right the first time around!
We try to keep things light at the office - there's a desk dedicated to all the snacks and baked goods people bring in every week, pets make regular appearances, and we host board game nights with friends."
Technology and Business:
"Indigo started with the idea that microbes could increase crop yields. As the science developed, and we started to think about how to go market, it became clear that farming is a really tough industry to be in. So rather than just sell seeds, we're looking to provide a platform that allows farmers to make more money off the hard work that they're already doing. At our core we're building a data platform to track and store as much information as possible about what's happening on farms, in our scientific research, and out in the market. We can then apply modern AI techniques and machine learning to find true insight from all of that data. Then on the front end, we're building out a series of apps to pass those insights back to the farmers and researchers that we're serving. If you think about the career of a farmer, they essentially have 40-50 chances across their lifetime to figure out how to maximize their farm’s performance. We can analyze data across thousands of farms in a single year and pass the insights we learn back to our customers. The end result will put money back in the farmers pocket and keep the harmful chemicals out of the food supply, therefore making it healthier in the process. Win-win!"
Interesting Projects at Indigo:
"There is no shortage of interesting projects or big sky ideas at Indigo. A lot of these projects are being developed in-house from scratch, but we’re also looking across the industry to find the best technologies that we can package together to collect more data and improve farm insight. This means investigating the latest in drone, satellite, and sensor technologies. Internally our biggest project right now is sending a text message. The messaging part is easy, it’s what’s in the message that makes this project so challenging. We’re building an end to end platform that will pull data from thousands of farms and devices on those farms, analyze the data using machine learning, make recommendations from those models and then send the insight that we generate from those models directly to the farmer via text messages. We’re also working on projects to disrupt the way crops are distributed, how research is performed and how farmers are marketed to sell to."
BitSight's Typical Customers:
"Customers vary greatly regarding size and industry; from Fortune 500 companies to universities to medium-sized businesses located across the globe -- North America, Europe, Asia and other regions. The common denominator is that they are motivated to gain insight into the cyber risks in their computing infrastructure and that of their key vendors.
As for use cases, there are several that stand out:
- Third-party risk management: This is getting insight into the cybersecurity performance of other organizations, such as the vendors in your supply chain.
- Benchmarking: This is insight into your own organization’s cybersecurity performance, comparing that performance to industry peers and communicating key indicators to the board.
- Cyber insurance underwriting: Cyber insurers use our ratings and the underlying data to make informed, data-driven insurance underwriting decisions. They also use our platform to provide tools to their policyholders, to limit their risk.
Now, a common concrete use case is when the security community identifies a new vulnerability, such as the recent Meltdown & Spectre. Our customers want rapid visibility into the potential presence of these types of attacks in their infrastructure and in their vendor ecosystem. We’ve invested considerably in our APIs and UI to enable users to determine this very quickly."
Interesting Projects at BitSight:
"There are numerous projects in the team that our engineers are excited about.
One is being driven by our DevOps team: a transition to Kubernetes for container management. This, along with our CI/CD pipeline, enables our teams to deliver code to production faster and in a more repeatable manner. Developers like this as it’s letting them create transient production-like environments for fast, streamlined collaboration; for example, they will quickly spin up a cluster with a feature that they are building to have internal users experiment with it & provide feedback. The DevOps team really likes it because we can isolate changes to each application’s containers, making deployments repeatable and understandable.
On the front end, we’re transitioning our main website from a number of jQuery powered Django templates to a modern React+Redux single page app and developing lots of new features in that single page app. A key part of that process is building out multiple custom and reusable components and linking them up to our in-house design system. As our component library and single page app infrastructure has matured over the past year, we’ve gone from development cycle of months for new features to weeks or even days. We’ve also been experimenting with new technologies like GraphQL and headless Chrome to improve both our state management and integration testing."
"We are totally cloud-based and try to use the technologies that best serve our customers and deliver real value. Sometimes that’s working with Microservice Architectures, Lambda, EC2 or API Gateway solutions. Other times, we’re exploring other brand new technologies to figure out how to best implement them within our product to deliver real value. And, like any company with a product that’s been around a while, sometimes we’re doing the decidedly unglamorous work of paying down technical debt or supporting older parts of the app with PHP, Java and Node.js."
Engineering Culture at Placester:
"We focus on creating a great experience for our customers and delivering them real value. As a team, we take great pride in our codebase and continually improving every part of it. We solve problems collaboratively, have the same goals and work for the same team. Going into every project, learning is at the forefront. We value a culture where individuals and teams are able to learn from one another... and we have some super smart people always willing to share things they’ve learned over the course of their careers."
What Acquia Does:
"Acquia products are used to create and deliver world-class customer experiences. Acquia is leading the charge in transforming web content management to multi-channel, personalized experiences across platforms including voice, IoT devices, and chatbots. Acquia solutions leverage leading technologies like machine learning, containerization, and microservices."
"We are growing our team significantly as we have no shortage of interesting projects to tackle. Our machine learning team is working on solutions to simplify data-intensive profiling tasks to make marketing teams’ lives easier. We have an industry-leading governance solution for customers that run large volumes of independent brand experiences and need to manage them collectively. We’re working on extending those governance features to customers who run one experience, a few experiences, or thousands of customer experiences."
"There are four main pillars of our engineering culture.
Learn it: Notarize has a strong focus on mentorship, with regular 1:1s, reviews, and check-ins to ensure that you are learning & growing as an engineer. Your manager will help you set personal goals, support your learning, and track your progress against those goals, to encourage you in the growth of your career, wherever it may take you.
Communicate it: You'll be part of a small, cross-functional team (no silos) with a highly collaborative process that values thoughtful communication, not just with other engineers but with all parts of the organization.
Ship it: We have a great cadence for releasing software immediately. We ship code once a week with some nightly builds (both web & mobile), with a focus on small features & improvements (no monster merges).
Fix it: We’re a small startup with big plans, and there’s so much to do. Whether it’s an antiquated legal process, an edge-case user interaction, a quirky piece of code, or a desk that hasn’t been put together yet, we show up every day with the goal of digging in and leaving something better than we found it."
"We love Railsbridge--we helped sponsor their most recent event and one of our engineers volunteered to teach. Having a diverse engineering team is non-negotiable for us; my mom was a software developer and I grew up hearing horror stories about her experiences as a woman in tech. Our vision is for Notarize to be an inclusive place where people of all backgrounds feel welcomed, appreciated, and valued."
In a unique spin on our Engineering Spotlight series, we interviewed Travis Dunn, the company's CTO. He shared the details on his background, lots of interesting information from DraftKings' engineering team, plus some details on their hiring plans and interview process.
“We’re hiring at every level, from Director of Engineering all the way down to college hires. We're looking for software engineers, data engineers, and pretty much all roles across engineering.”
“What we're looking for is whether you can explain the systems you worked on, and whether you learned them in-depth. We look for curiosity and being able to see the big picture, and how you fit into something. We're also looking for people who can tell stories of working with teams. Working here is super collaborative, and not every company is. People who do well are leaning over the shoulder, giving feedback, offering code reviews, and going up to the whiteboard for design.”
Divided Engineering Teams:
"Geography plays a minor role in how we divide work, but it is a huge factor in how we execute work. Across the group we’re organized into a few teams, 3-5 engineers per team. We’ve found that allowing our teams to specialize around domains, a combination of technical and business needs, we’ve been able to pretty naturally segment the work. Teams run themselves pretty autonomously, so when there are members across both locations, they determine the best way to break things down given the context. Geography is a huge factor when it comes to communication and how we work together. We’ve invested pretty heavily in our remote conferencing tools (Zoom is amazing) and we constantly have a few team members visiting from out of town."
On Staying Sharp:
"As a team we’re constantly looking for opportunities to improve. This includes pushing the team to external sources. Last year we sent more than 70% of the team to conferences. Additionally, we sent team members to various classes to deep dive into a particular subject. We’re not just focused on technology, we also push training for our engineering managers to improve their teams."
"The company was spun out the MIT Senseable City Lab. The research completed there was to understand how people move around cities led directly to the Copenhagen Wheel, our first product. We’re trying to make it easier to move around cities as they continue to grow, in ways that are sustainable, convenient, and often fun."
The Tech Behind the Copenhagen Wheel:
"The Copenhagen Wheel and surrounding software are a complex ecosystem, but with the goal to make something that is still as simple as riding a bike.
The hardware inside the wheel is a sophisticated combination of precision sensors, control systems, firmware, battery and motor. All these work together with the purpose of reacting to human pedaling so quickly that the addition of motor power makes the rider feel as if they are stronger, not like they’re being pushed. From there we track sensor and biometric data at a high rate and communicate it over Bluetooth Low Energy to our smartphone apps, where we automatically track GPS traces combined with biometric data for the user.
The combination of firmware and apps summarize the sensor data being collected into diagnostic data about the wheel’s performance and environment, which we can use to diagnose problems in the field, improve performance in future firmware, or iterate on the hardware itself. Our backend systems allow us to introspect those diagnostics from a support or research perspective."
How the Hopper App Works:
"We receive, from several GDS partners, real-time “shadow traffic” containing the results of consumer airfare searches. We collect 10 to 15 billion airfare price quotes every day (300 billion a month) and have built a huge historical archive of several trillion prices from over the past several years. Our data includes priced availability over time along with consumer demand (search interest). We use our historical data to create trip-specific pricing forecasts up to a year in advance of departure based on current and historical yield trends as well as pricing volatility.
We’re taking a very similar approach with hotels where were connecting with a number of different hotel data providers. With hotels on Hopper, we became the first company to offer price predictions for hotels. With 95% recommendation accuracy up to six months in advance, Hopper monitors prices and alerts users to save an average of $34/night, and up to $90/night in best case scenarios. Contrary to popular belief, the best hotel rates aren’t always available at the last minute. We found that on average the best hotel rates are typically available 2-3 months in advance."
"We maintain a high-quality application with state of the art animations and design that 27M users have enjoyed. The core of our app relies on a prediction algorithm that consumes trillions of data points, and making personalized recommendations in the travel industry means dealing with data at a scale and complexity unmatched by most companies taking a similar approach (i.e. Netflix, Amazon), but also with significantly less frequent feedback (users taking 1-2 trips per year).
In addition to our historical archive of trillions of flight prices, we now have millions of users setting over 100k new trip watches everyday which has enabled us to build a huge graph of Hopper users’ preferences. Last year, we began testing a recommendation algorithm -- sending users notifications about deals to alternate origins, alternate destinations, different months or weekends, with every conversion further strengthening the algorithm.
We also have a lot of interesting architecture problems. We’re using a state machine to run the booking engine, which is a combination of a lot of disparate hard-to-coordinate systems that we’ve brought order to. We’ve scaled that up to serve thousands of bookings we get a day. We need to maintain API compatibility between a micro-service architecture on the backend and multiple clients spread across multiple versions and operating systems. We also maintain a low latency service while separating concerns on the backend."
"We are running Java and Node.js. Services implement their business logic by leveraging different types of persistence layers or databases like: Oracle, Casandra, Postgres, Mongo or JanusGraph—our architects and developers are able to choose whatever best suits their needs and objectives. All services, old and new, communicate through REST and messages using different middleware options such as Coherence, Infinispan, AMQ, and RabbitMQ. Of course, we try to balance the independence with which our teams can make technology choices with opportunities to consolidate as the platform continues to evolve."
On Security for Engineers:
"For an engineer at Intralinks, security starts from mandatory secure coding training—they need to take annual training in technology relevant to their day-to-day work. Our Definition of Done for any new component includes passing the threat modeling report review. Lastly, if they have made mistakes in coding or included open source component with security vulnerabilities—which should not happen often—the build will be broken, and they will need to address reported issues in order to pass the security gate. Some engineers may choose to take additional training and become Intralinks Security Champions and help Scrum teams to follow secure development lifecycle and produce working secure code."
"gita, PFF’s first product, is a mobile-carrier that follows people on-the-go carrying up to 44 pounds. It is the answer for getting out the door without having to reach for the keys to the car. Through products like gita, PFF is creating a new mobility ecology of freedom, sustainability, health, social connection and pleasure for all people."
Engineering Projects Outside of Work:
"The team is very hands-on and most members of the engineering team have personal projects they’re working on, but they don’t directly overlap with PFF work. However, we do use a lot of open-source technology and we contribute back as much as we can.
Right now, everyone is focused on the development of our first product, gita. However, we often bring our personal interests into the office. These interests can range from the PFF car club, cooking classes taught by some of the team, a running club, and cross-functional projects (like designing and making our own cornhole boards)."