One of the best things about Kubernetes is how every company using it is transforming their business. One of our customers, Flatfile, is set up for huge growth in 2021 as they invest in solving problems associated with mass amounts of data - a category called data onboarding. As described in VentureBeat, “The fundamental role Flatfile plays is, somewhat ingloriously, that of data janitor.”
The team built an API that helps companies import data in hours (as opposed to days, weeks, or months). Of course, supporting the company’s goal requires scalable and secure infrastructure.
We recently met up virtually with Robert Trencheny, Head of Infrastructure at Flatfile to ask a few questions about his career, Kubernetes and Fairwinds Insights.
Straight out of high school, I got a job at a big digital agency on the east coast. I built and maintained the Heisman Trophy website in 2010. That was the first year they decided to announce the winner on the website at the same time the trophy was presented in person. That’s when I tuned into infrastructure because it’s pretty crazy to watch 10 million people hit a single website at the same time and obviously everything failed over immediately. After that, I co-founded Hackbright Academy which teaches Python to women and helps them find great jobs in the industry. We sold that in 2016 at which point I had moved on to other startups and projects.
The project that got me really into infrastructure was Gyroscope, a health data startup, where I built their initial data infrastructure. They started with only a couple hundred users but with billions of data points a day. You have to process it through a huge data pipeline to build the beautiful system it is. After that, I went on to build Campuswire, a communications platform for colleges. After a few years as the lead engineer there, I went on to co-found BloomJoy which worked with social media influencers to help them monetize their audience by writing branded content. We built websites for these influencers, a few of which entered the Alexa top 50 rankings which obviously brought tons of scaling challenges along with it.
I then moved on at Flatfile having known and worked with the CEO and other team members for years. I am head of infrastructure and also head of internal tooling and IT. We are hiring in 2021 to enable us to grow fast.
Containers just make sense. It’s the new normal, making things so much easier. For example, we are working with a few companies that built their own tool set to help clients ingest data into an existing application. We couldn’t do a traditional on-prem deployment for them and they couldn’t use our cloud product because of compliance issues. Containers have allowed us to give clients raw Docker images in two or three environment variables. Ten years ago that would have been impossible - you’d be giving 60 different bash scripts to hopefully get everything set up properly.
With Kubernetes specifically, we are barely scratching the surface - basically using it as a Heroku replacement. Our engineering goal is to enable user-driven data pipelines to allow users to actually write functional code to control how data should be mutated or translated. That means I need to have that code run in a non-trusted environment, to do it at scale and to run it on anything from a 500 byte file to a 5 TB file.
Being able to orchestrate along with AWS tools to send data to the right places, have a central plane to monitor everything and to be sure we do not have any security issues will be the power in Kubernetes for us. It’s a critical foundation to enable this scale.
We’ll also be adopting multi-region and private cloud. Containers make it easier because we can have one set up and easily duplicated across multiple regions. We do all our provisioning with Terraform so having a simple set of Terraform files that can roll out containers to a Kubernetes cluster is super powerful.
Right now we are trying to stay pretty close to Kubernetes best practices. But one thing that isn’t necessarily cutting edge, but that I’m thankful everyday for is operators. Specifically, we love using the Prometheus operator because it just takes care of our monitoring stack for us without really having to juggle anything on our own. We are looking to expand use of operators and cloud connectors.
This is a year of growth for us. We are moving to containers to run our entire company. We will standardize on bs-code which will run on Docker, standardize on Kubernetes and have systems so that our engineers can understand them quickly.
The biggest thing on my radar and what keeps me up at night is compliance. I need to be sure that only certain people have the minimum amount of permissions to clusters and code. I need to be certain that everything is getting double and triple checked and reviewed before going out. We deal with sensitive data so compliance is a huge requirement for our business. We are HIPAA, GDPR and SOC 2 compliant, but as we get more into the US federal government more will be required.
Scale and security has been solved, but making sure you have a centralized way to monitor and make sure that there’s nothing nefarious going on and be able to respond to breaches if they happen is a huge worry.
Fairwinds Insights is in a suite of products that help me sleep better at night. I don’t have to actively monitor Insights, but I know that if something goes wrong, I’ll get notified. Because Fairwinds is constantly improving the product, I don’t have to worry about constantly tweaking and tuning things. I know that you’re going to be monitoring for best case scenarios for me.
Fairwinds Insights gives me peace of mind, honestly. The Slack notifications alert me and if we see an issue, we fix it fast. That’s fantastic because it means we are doing hundreds of millions of rows of data ingested per week right now. I haven't had to hire a full time security engineer yet. Fairwinds is like half of that equation for us with Sqreen being the other. Having this effective security blanket for us is huge.
In the dashboard, when I click on an alert, you don’t just give me a one liner explanation and then expect me to figure out on my own how to fix it. You actually tell me this is how you fix it or at least try to get as close as possible. The dashboard is really key for me - it’s such an easy tool to use and intuit really quickly.
I haven’t done a ton of market research into this because I’ve been blown away with everything Insights does and its absolute focus on Kubernetes over anything else.
We are in the middle of signing a large enterprise client. We are in the second of three reviews by independent committees. I’ve been able to say to them that we have a constant monitoring solution in place that makes sure Flatfile infrastructure is running against best practices and that things are running well - It’s a huge value add. If not for Fairwinds, I’d need to rely on infrastructure engineers on teams or technology not built for Kubernetes.
It hasn’t been possible for us yet to use this, but the value is huge. It’s of immense value to be able to make sure that we don’t have any issues getting into production that shouldn’t be. I love the workflow.
It’s knowing that if something goes wrong, I’ll get a Slack notification. When I woke up this morning the first thing I did was check our infrastructure alerts channel and I saw five new Insights alerts there. One of my infrastructure team members clicked in and fixed it pretty quickly with just a pull request. The notifications are key for us.
