C&B Notes

Amazon’s Evolutionary Echo of Walmart

Zack Kanter, an avowed student of Amazon and Walmart, takes a fascinating look at the evolution of retail through the lens of these two companies.  In the piece, which is long but well worth the investment of time, he argues that Amazon’s most important structural advantage is that the company found a way to kill its bureaucracy in advance of its explosive growth over the last two decades.  Kanter also explores the conflicts of interest inherent in Amazon’s retail ads business and why they could become a big problem.

To make sense of what started to happen after Amazon rolled out Marketplace, you have to understand that things get really weird when you run an unbounded search at internet-scale.  When you remove “normal” constraints imposed by the physical world, the scale can get so massive that all of the normal approaches start to break down.

Walmart had solved problems of vendor management, product management, and bureaucracy at an almost unfathomable scale.  It engineered intricate systems, aligned incentives, and built a culture of thinking small to stamp out inefficiencies wherever it could find them.  Walmart solved problems that were almost impossible to solve at Walmart’s scale, creating a wonder of the modern world, perhaps the pinnacle of what is possible with complex coordination.  And Walmart, at its heart, is a company of merchants; it is a human-powered company, and its advantage in the marketplace is that it merchandises better than any other company on the planet.  Walmart understands its customers extraordinarily well, and its merchants play a hand in every product that shows up in its aisles.

Amazon, by contrast, is an illustration of what happens when a massive global market is freed by the internet from the geographical constraints that previously kept it manageable; it is an illustration of what happens when you enter a problem space so large that you have to bypass the human element altogether.  What was just barely solvable with carefully-built systems at Walmart’s scale of shelf space would have been impossible to solve with shelf space that stretched on to infinity.  Amazon had to find a way of abdicating responsibility for solving these problems altogether; with Marketplace, Amazon had begun to grasp at a solution that would do exactly that.

After removing the vendor bottleneck, Amazon had discovered the next constraint to filling its theoretically-infinite shelf space: computing power and data storage.  To his horror, Bezos had discovered that Amazon’s software engineers were waiting weeks for technical resources like servers and storage to be provisioned.  Instead of being limited by how fast they could write code, they were limited by how fast they could deploy that code to Amazon’s infrastructure, and so, alongside an effort to dramatically simplify and improve its codebase — which had evolved into a mess of ‘spaghetti code’ in the ten years that Amazon had been in business — Amazon began to build a platform that would allow its software engineers to provision on-demand resources immediately. In a radical move, the platform — Amazon’s own technological infrastructure — would be made available to external developers, too.  It would be called Amazon Web Services…

And so, circa 2002, we start to see the emergence of a pattern: 1) Amazon had encountered a bottleneck to growth, 2) it had determined that some internal process or resource was the bottleneck, 3) it had realized that it could not possibly develop and deploy enough resources internally to remove that bottleneck, so 4) it instead removed the bottleneck by building an interface to allow the broader market to solve it en masse.  This exact pattern was repeated with vendor selection (Amazon Marketplace), technology infrastructure (Amazon Web Services, or AWS), and merchandising (Amazon’s Catalog API).

Amazon was becoming a platform; that is, an aggregation of resources made available through a series of interfaces.  In the case of Marketplace, the resource was customer demand, and the interface was a web portal called ‘Seller Central,’ which allowed sellers to list items in Amazon’s catalog and process the resulting orders.  With AWS and Catalog, the resources were computing power and a monetizable ecommerce catalog, respectively, and the interfaces were corresponding web portals and APIs (application programming interfaces) that software developers could access programmatically.

Platforms spring up as a necessity borne from unbound searches running at internet scale. A company like Walmart, despite being positively massive in terms of revenue, can operate as a monolith — that is, a tightly-coupled collection of internally-facing resources — because it is dealing with a constrained problem space. Walmart’s problem space (for argument’s sake, 100,000 SKUs and 100,000 square feet) was, for the most part, limited enough that Walmart could, with sufficient effort and innovation, solve its own problems internally.  It could manage vendor selection, it could merchandise its own catalog, and it could manage, and mitigate, the growth of its own bureaucracy.

There is a notable exception here: the scale of Walmart’s purchase order volume was so large that it could not feasibly continue to manage the purchasing process on its own.  To solve this, Walmart built Retail link — perhaps Walmart’s first platform — to expose its purchasing ‘resources’ externally.  Retail Link gave Walmart’s vendors tools to manage purchase orders and much more, taking an enormous burden off of Walmart itself.

Walmart began sharing its inventory data with key suppliers, too.  The problem of coordinating Walmart’s inventory had grown too large for Walmart to solve on its own.  By sharing its inventory levels and internal projections with vendors — by making its internal numbers available externally — Walmart could draw on the wisdom of the broader ‘market’ — its vendors — to arrive at better outcomes than it could have achieved within a silo.

Like with Amazon’s various platforms, Walmart built Retail Link out of pure necessity; without it, the purchasing process would have remained a constraint to Walmart’s growth.  The difference is that, with its unconstrained shelf space, Amazon was encountering these problems — and implementing platform solutions — everywhere.

* * * * *

As these examples of the same pattern — Marketplace, AWS, and catalog — emerged around the same time in 2002, Jeff Bezos had the most important insight he would ever have: in the world of infinite shelf space — and platforms to fill them — the limiting reagent for Amazon’s growth would not be its website traffic, or its ability to fulfill orders, or the number of SKUs available to sell; it would be its own bureaucracy.  As Walt Kelly put it, “we have met the enemy, and it is us.”  In order to thrive at ‘internet scale,’ Amazon would need to open itself up at every facet to outside feedback loops.  At all costs, Amazon would have to become just one of many customers for each of its internal services.

And so, as told by former Amazon engineer Steve Yegge, Jeff Bezos issued an edict: 1) All teams will henceforth expose their data and functionality through interfaces, 2) teams must communicate with each other through these interfaces, 3) all interfaces, without exception, must be designed from the ground up to be exposed to developers in the outside world, and 4) anyone who doesn’t do this will be fired.

This principle, this practice, this pattern, would enable Amazon to become the sprawling maze of complexity that it would eventually become without collapsing under its own weight, effectively future-proofing itself from the bloat and bureaucracy that inevitably dragged down any massive company’s growth…

With an established pattern for solving the practical and bureaucratic issues that arose from infinite shelf space, Amazon began systematically removing bottlenecks to growth.  It found that Marketplace sellers were not particularly adept at shipping directly to Amazon’s customers, causing a poor experience for customers and a frustrating experience for the sellers themselves.  At the rate that new sellers were signing up, Amazon could not feasibly convert all of these sellers to its vendor program, nor did it have sufficient capital to carry all of the inventory on its balance sheet; instead, Fulfillment By Amazon (FBA) allowed sellers to ship their inventory to Amazon’s fulfillment centers, thereby giving Amazon complete control over the customer experience.  It carried with it the tremendous added benefit of honing Amazon’s own rapidly-expanding fulfillment network, which were certainly at risk of bloat and slop, with the sharp edge of seller expectations; just as with AWS, Amazon became just one of many “customers” for its own fulfillment centers.

Platforms became Amazon’s answer to every growth obstacle it encountered.  Platforms became part of the algorithm.  Sellers are limited by access to capital?  Launch Amazon Lending. Customers can only buy things when they are in front of their computer or phone?  Build Echo. UPS and FedEx can only deliver within 24 hours?  Launch Amazon Flex and Amazon Logistics.

Amazon assembled a massive machine to deploy its algorithm over and over, and the momentum was unstoppable.  Every barrier in its path was solved with a platform — until one of these platforms led Amazon to a catastrophic mistake…

More broadly, though, Amazon’s misstep is symptomatic of the weirdness that eventually happens when an unbound search runs across such a massive problem space.  In building Marketplace and removing itself as the constraint for vendor onboarding, Amazon has opened itself up to inevitable ‘gaming’ by sellers.  Another way of saying this: as soon as a system’s rules are understood, it will be gamed according to the rules that have been created.

With infinite shelves that are constantly expanding and filling without constraint, Amazon cannot possibly police the ever-growing universe that it has created.  Another way of thinking about this is that while Amazon’s catalog of SKUs is constantly growing, the number of top slots for popular categories is not; no matter how many thousands of water bottles get added to Amazon’s catalog, there are a fixed number of page-one results.  But the growing horde of competition puts enormous pressure on the entire system; companies are always trying to find ways to knock the current king off the hill.  Bad-actor tactics inevitably surface, and Amazon is in a constant war to keep its own platform consistent with its customer-centric mission.  This is a war that Amazon will never be able to “win”; the best it can hope for is to try to keep up with the evolving bad-actor tactics, which seems like a tall order given that the total set of sellers is constantly increasing (one might call this “the IRS problem”)…

With its marketplace platform, Amazon has created a Wild West for sellers  — with all of the tremendous benefits, and pitfalls, that come with it.  It continues to accumulate marketplace sellers at an incredible pace.  Amazon’s next decade (in retail) will be consumed with capitalizing on this opportunity, along with mitigating all of the extraordinary challenges that come with trying to maintain order in a marketplace at a literally inhuman scale.

It is easy to look at Amazon’s exploding marketplace numbers and see insurmountable success, but we are only beginning to see what it has signed up for by building a world of infinite shelf space and opening the door to anyone who wants to set up shop.  Amazon’s herculean challenge will be retaining its crown as “earth’s most customer-centric company” given the marketplace dynamics that it has created.

Referenced In This Post