Jim Zheng Posts About Now Photos

early problems

I’ve been thinking about hiring a lot lately. One thought that I keep running into is: why join a startup in its earliest stages? What makes the engineering challenges fun? I’ve always felt there was something unique about joining a startup as opposed to a big company. Not as a founder, but anywhere between being the first engineer to being the 50th. The feeling has to do with being able to work on a lot of “early” problems.

What are “early” problems? I’ve seen two types.

The first type relate to greenfield projects. They’re hard to grasp because you don’t even know how to phrase them as problems. Likely, if you’re working on a product in a new domain, you’re learning about the domain as you go along. Take the first people to working on email, medicine, and billing systems. They probably found writing software for these domains to be super interesting. All they had were a vision and guiding principle.

Let’s take gmail. Building the search functionality for a web-based email client must have been fun. The scale was tiny; data volumes for gmail were a million times smaller than the vast volumes in gmail today. Now search is such a solved problem. There’s Algolia or AWS ElasticSearch, paid services that do all the heavy lifting for you. But in the early days, you had to come up with your own clever tricks to make search fast.

At Flexport, when we were building the first versions of projects, we had no idea what the “right” thing was. One example of this was a document parsing pipeline. Providers emailed us tabular data in unstructured formats. They scattered rows across screenshots, pdfs, and excel files. To figure out what to build, we had to learn how business teams operated. To understand how this would run, we needed to learn the nature of the industry.

One of the issues with working on these types of problems is: you have to figure out which ones are the most valuable. And to do that, you need to have a nose for what’s valuable for the business and the industry. Take our document parsing project. We knew it was the right thing to build: we built the features that alleviated the most pain for our business team. The most valuable features turn out to be the ones that solve these mundane, ev pains.

The second type of “early” problems are at the scaling phase. This is when your systems are no longer working because your org has evolved. You need a re-architecture. You have the same problems to solve, but you need a new paradigm for solving them. Maybe this is a move to service-oriented architecture. Maybe it’s a transition from on-prem to the cloud, or from cloud to on-prem like dropbox.

These problems are very unique, and you likely only get to experience these a few times in your career. They exist because you joined a company early enough to see the business grow. A company only needs to go through this painful process once. For example, you never do an SOA migration twice. If you’re lucky enough to get domain modeling right, you have much less work. If you get it wrong, you might have to combine and remove some services. But it’s much less painful the second time.

At Flexport, we’re going through our own - the data mesh transformation. It’s an “early” problem because there’s few existing examples in the industry, and it’s a big change. Most enterprise companies only migrate bits and pieces of their systems. It requires proving out the value of this new pattern, and we won’t see the true benefits until years down the line.

Interestingly, the two types of problems all have “pain” as a common denominator. In the first type, you’re solving your customers’ pains. In the second, you’re incurring the pain of refactoring or re-architecting your systems.

Early problems are painful, pretty hard, and you’ll likely make mistakes along the way. But they’re also the most informative learning experiences. In a career where you bias for learning, these experiences will always force you to stretch and grow. But what else is there but to push forward?

other thoughts

Discussions

Discuss on Twitter