Engineering Spotlight: BitSight Technologies
The BitSight Security Ratings Platform rates a company’s cybersecurity performance similar to how consumers are rated by a credit score. The company currently has nearly 300 employees and is aiming to grow to over 400 by the end of this year. You can read our recently profile on the company by clicking here.
We connected with BitSight's VP of Engineering, Sanjeev Banerji, to get an inside look at the culture of the engineering team and how they operate day to day. We also learned about what the team does for fun, which includes both old school movies and ski trips. Read on below!
Also, BitSight is hiring! Check out its BIZZpage for all the company’s openings!
Quick Hit Company Details
- Year Founded: 2011
- Number of employees: 300
- Number of engineers: 55
- Industry: Cybersecurity / SaaS
Can you share a summary of what BitSight Technologies does?
BitSight transforms how organizations manage information security risk. The BitSight Security Ratings Platform, a cloud-based SaaS system, applies sophisticated algorithms to Web scale amounts of security & risk data. The platform subsequently produces daily security ratings ranging from 250 - 900. After receiving the score, they can assist in managing third-party risk, underwriting cyber insurance policies, benchmark performances, conduct M&A due diligence, and assess aggregate risk. Organizations worldwide, including seven of the top 10 cyber insurers, 20% of Fortune 500 companies, and 3 of the top 5 investment banks, use BitSight’s proven Security Ratings technology on a daily basis to make integral risk and business decisions. With over 1,000 customers and the largest ecosystem of users and information, BitSight is the most widely used Security Ratings Service.
Who are your typical customers? Are there any use cases from a customer that stand out to the engineering team?
Customers vary greatly regarding size and industry; from Fortune 500 companies to universities to medium-sized businesses located across the globe -- North America, Europe, Asia and other regions. The common denominator is that they are motivated to gain insight into the cyber risks in their computing infrastructure and that of their key vendors.
As for use cases, there are several that stand out:
Third-party risk management: This is getting insight into the cybersecurity performance of other organizations, such as the vendors in your supply chain.
Benchmarking: This is insight into your own organization’s cybersecurity performance, comparing that performance to industry peers and communicating key indicators to the board.
Cyber insurance underwriting: Cyber insurers use our ratings and the underlying data to make informed, data-driven insurance underwriting decisions. They also use our platform to provide tools to their policyholders, to limit their risk.
Now, a common concrete use case is when the security community identifies a new vulnerability, such as the recent Meltdown & Spectre. Our customers want rapid visibility into the potential presence of these types of attacks in their infrastructure and in their vendor ecosystem. We’ve invested considerably in our APIs and UI to enable users to determine this very quickly.
What types of data are collected and processed by BitSight?
To produce security ratings on companies with a worldwide presence, BitSight collects data from sensors deployed globally throughout the Internet. All of our data feeds are used to produce more accurate and useful security ratings, and therefore the majority of them are network security-relevant in nature. For example, one of our most important data feeds is generated by our botnet sinkhole infrastructure. This system intercepts communications from massive worldwide botnets and allows us to identify when malware infects companies. It is the largest botnet sinkhole on Earth and generates roughly 12 billion events per day.
Other data feeds that we collect include Internet-scale measurements of software configuration and vulnerabilities, a feed of global DNS traffic, visibility into the BitTorrent distributed hash table, and communications from misconfigured network appliances and the Internet of things devices.
This data is all processed by our data pipeline, which ingests hundreds of billions of records per day in order to update our security ratings daily.
What are some of the different technologies that the engineering team uses?
We use leading-edge technologies across our teams and tech stack.
Moving from the back end to the front end, first, we have our Big Data pipeline team. This team develops the piece of our system that does the “heavy lifting” data processing, handling tens of billions of security events per day on AWS clusters using thousands of cores. We use Spark, Hadoop, HBase, and Elasticsearch, and we code in Java, Scala, and Python. All of this is in service of addressing problems in distributed computing, scalability, and reliability.
Next comes back-end services team, which straddles the Big Data pipeline and the front end. The services transform the billions of security events we ingest every day, into a form consumable for our Web client and customer APIs, utilizing Django/Python, Scala, and Java. We are rapidly transitioning much of this code base to microservices.
Our DevOps team embeds with development teams to ensure reliable and consistent delivery of changes into production. We are always looking to accelerate and streamline processes, leveraging the latest technologies like Docker and Kubernetes to do just that.
What are some of the interesting projects that the engineering team is tackling?
There are numerous projects in the team that our engineers are excited about.
One is being driven by our DevOps team: a transition to Kubernetes for container management. This, along with our CI/CD pipeline, enables our teams to deliver code to production faster and in a more repeatable manner. Developers like this as it’s letting them create transient production-like environments for fast, streamlined collaboration; for example, they will quickly spin up a cluster with a feature that they are building to have internal users experiment with it & provide feedback. The DevOps team really likes it because we can isolate changes to each application’s containers, making deployments repeatable and understandable.
On the front end, we’re transitioning our main website from a number of jQuery powered Django templates to a modern React+Redux single page app and developing lots of new features in that single page app. A key part of that process is building out multiple custom and reusable components and linking them up to our in-house design system. As our component library and single page app infrastructure has matured over the past year, we’ve gone from development cycle of months for new features to weeks or even days. We’ve also been experimenting with new technologies like GraphQL and headless Chrome to improve both our state management and integration testing.
What is the culture like at BitSight Technologies for the engineering team?
Based on informal internal discussions and observations of how our team operates & behaves a few elements really jump out. BitSight employees share the following characteristics:
Collaborative. Great outcomes are rarely the outcome of solo efforts. We know that engineering is a team sport.
Humble. We are proud of our accomplishments thus far, but we know that there is much hard work ahead.
Diverse. Our team has many different types of people & personalities from a wide range of backgrounds.
Impact-oriented. We value results. Not just progress but driving things to completion, be it building a feature, addressing a customer escalation or providing guidance to our customer-facing teams, for example.
Inquisitive. We are a smart & talented bunch but are very aware that each of us has more to learn. And due to our growth and success thus far, there are always new opportunities to exercise that curiosity and contribute in new ways.
I’ll add that our founders, Stephen Boyer and Nagarjuna Venna, deserve immense credit for laying the groundwork for these values. These values are in the DNA of the team and of the company; that is a testament to who they are as people. And these attributes continue to spread organically as the team & company grow, as we’ve become a global organization with offices in Cambridge, Lisbon and Raleigh, NC.
What does BitSight Technologies look for in a potential employee? What can someone expect during the interview process?
Some of this is tied to our culture, as described above. We want people who highly value collaboration and producing results, for example. We are also keenly aware that it’s invaluable to have people who actively evolve our culture; we are growing & changing and our culture must do the same. We try to find people who will contribute to our culture, rather than simply conform to it.
Our interview process follows a fairly straightforward path from the phone screen to the on-site interview. We are an engineering team, so a good chunk of the interview process is technical. For hands-on engineering positions, there are lots of technical discussions -- whiteboard problem solving, design, software dev process, testing, etc. There are some open-ended conversational discussions, not focused on any particular topic. For leadership positions, there is an additional focus on leadership and management acumen; we like to dive deep into both your approach and your actual experiences. Lastly, we line up time with executives outside of engineering and always try to include one of our founders. We’ve found that the opportunity to talk with senior leadership is greatly valued by candidates, as they get to see the broader perspective that those leaders have.
Are you involved in any local tech organizations or Meetups? And what about outside of the office?
We are actively working to increase our involvement in the local tech communities that we are a part of. We’ve presented at local Big Data and front-end tech meetups and attended local symposia such as devopsdays Boston. In the next few weeks, we’ll be presenting at DevOps meetups about our experiences with Kubernetes. We also make it a point for engineers to attend the major tech conferences that are relevant to us, such as AWS re:Invent, Kubecon, and SMASHING. Internally, we have an annual hackathon with both Product & Engineering; from our most recent one, two ideas are now being productized, so we’re quite excited about that.
Apart from work, the team and the company have tons of planned and spontaneous social activities. A favorite is the annual ski trip. This year, we had about 50 of us up at Loon Mountain, NH, for a day of learning on the greens to straight-lining on the double blacks. We have a film series of sci-fi movies on Wednesday afternoons, streamed on a big screen TV in our cafe area, complete with popcorn and soda. We also do lots of impromptu things; a memorable time is when one of our product managers brought in oysters from Duxbury -- not from the store but actually from the oyster farm in Duxbury! Plus, he shucked them himself for all of us to enjoy.
Rapid Fire Q&A
What’s on tap?
Cold Brew Iced Tea!
Star Wars or Star Trek?
At the certainty of dating myself, I stood in line for this when it was originally released many moons ago, so it’s gotta be Star Wars.
iPhone or Android?
Coffee - hot or iced?
Both! Of course, we have the hot stuff and in the summer we brew our own cold brew.
Favorite employee perk?
The annual ski trip for the Cambridge team is very popular.
Favorite app used by engineering team members?
What music is playing in your office?
A wide variety, all piped through people’s headphones.
Ethan Geil, Technical Director, Data Pipeline
What attracted me to BitSight (and keeps me excited about working here) is the combination of the great team and the fascinating problems we face. We get to address a meaningful, important challenge (improving security and reducing the risk of breaches); we do this by analyzing all sorts of security data at Internet scale (we process hundreds of billions of records per day). We make extensive use of standard big data tools (including Spark, MapReduce, and HBase), but because of the combination of scale and the unique questions we ask of our data, we often have to develop novel techniques and architecture, too.
- Ph.D. in physics from Cornell University (experimental condensed matter)
- Favorite work is at the intersection of data science and data engineering
- Mentor aspiring data scientists and engineers via Insight Data Science and other programs
Caroline Gallagher, Software Engineer
- Bachelor’s (double major) in computer science and economics from Wellesley College
- Worked as a software engineer at Kyruus for a year and a half after graduation before joining BitSight.
Images courtesy of BitSight Technologies.