From a portrait drawer and musician to being the CTO at Shopify, Jean-Michel Lemieux (JML for short) unlikely story is fascinating. In this podcast, Tobi talks to Jean-Michel about his nerd journey to where he is today. They also converse about his fight against crazy sneaker-sniping bots, what it takes to run the most extensive Ruby on Rails monolith on the planet, how he keeps his army of 2000 engineers at Shopify aligned, among other things.

When he was in Grade 12, their music class was doing a musical for the Phantom of the Opera. Jean-Michel asked his music teacher whether he could sequence the entire musical using the Atari ST computer that his dad had just bought him, allowing the teacher to fire the band. He went ahead and did a couple of songs, prerecorded and sequenced them. This was his first encounter with computers and the possibilities they present, which he says he felt like they gave him superpowers.

The teacher didn’t quite understand what JML was up to, but Jean-Michel says he pulled it off during rehearsals. However, that wasn’t big enough a sign for him to realize he could pursue a career in computers-- not until his guidance counselor suggested it. And so his programming journey began. More than anything, as an artist, computers made him fall in love with the power to create things.

“I found that I could create things with (the computer),” he says, “And I just went from painting to programming as a very natural evolution.”

JML’s career in software development began in earnest when the transition from analog to digital began. It was the pre-internet era. They were digitizing the telephone network.

“That's literally where the first part of my career was,” JML reveals, “What was fascinating is it was a precursor to I'd say cloud computing. The telephone network was a cloud of telephony, and we were doing things like fraud detection, we were running super-high bandwidth ‘cause the phone network had a lot of traffic.”

“One of the biggest days of the year was Mother's Day because Mother's Day is when everyone phones their mom. So, I went through building a lot of real-time systems. You're doing like TCP/IP Nonce… it literally is cloud computing of like 20 years ago.”

JML was a founding member of the Eclipse platform. Around this time, he was quite passionate about creating tools that made coding easier, a journey that took him to work on Eclipse for a decade. Ultimately and by accident, he found Shopify.

Black Friday at Shopify

This podcast interview happened two days before Black Friday. Tobi wanted to know how JML could spare some time for this chat so close to Shopify’s busiest day.

“We watch every metric, and our goal is: Black Fridays should be boring. We spend the whole year getting ready for it. We're done. Deploys are locked. We're just twiddling our thumbs at this point.”

A lot of Shopify’s efforts in recent years have been spent on creating testing environments that simulate what's in production.

JML explains.

“It's like, ‘how do I create a version of Shopify that has the exact same traffic, but it's not the real one that I can go in and practice’? That's really hard, right? You're thinking about network topology. How do you mirror the exact same traffic and a lot of the super high-scale projects around the globe?”

“What we've done at Shopify over the last couple of years is, to the best of our abilities, create a bit of like a training version of Shopify that we can load-test on; we can try things on way before they happen in production with our live customers.”

“We have to simulate Black Friday before it happens almost every day, so we know what our bottlenecks are. We have to understand what the cascading failures, what our tripwires are. So we've been spending the last three, four years at Shopify building that out: our simulation environments. So that today I can be a bit calm: because we've seen everything that's probably going to happen.”

While it’s possible to customize many things when using Shopify, JML says they’ve limited merchants’ ability to do a lot of checkout customization. This engineering decision is informed by the need to avoid third-party code that could slow down or interfere with the exchange of goods and money.

“You adopt Shopify; you get our battle-hardened checkout. We’ve spent billions and billions of dollars. We kind of know what conversion looks like. You can mess around with how your online store and your clothing site looks like, but once you get to checkout and you get into Shopify, we do take over control of that, and we don't let you extend it. 

“I think some other platforms have probably allowed a bit too much flexibility there. Once money starts exchanging, then we take control. And we test that everyone's getting the default fastest, quickest checkout, and battle-hardened.”

The Gaming Culture at Shopify

At Shopify, gaming has influenced the company culture. JML intimates that Shopify CEO Tobias “Tobi” Lütke and his kids ‘game all the time’ and that Tobi met his wife gaming. They see gaming as very similar to the complex environment in a company where you’ve got to make short-term, medium-term, and long-term decisions by understanding the mechanics at play.

“For us, gaming is possibly the best training experiments for building a company. We believe that if we make a good set of bets that are balanced for the long-term and short-term, immediate term well that we could possibly be 10% better than all the other companies out there.”

‘The Biggest Monolith on Rails’

Though Shopify is almost entirely built on Ruby, Jean-Michel says that he had not written a single line of Ruby before joining Shopify. Yet, he is quick to say that he believes that Ruby has played a crucial role in Shopify’s success.

“I really like Ruby. Ruby's extremely expressive. It’s easy to read. From an engineering culture, you've got to favor the fact that you'll be spending more time reading code than writing code. And you've got to make sure that you're in an environment, in a language that's enjoyable to read (and) easy to understand. And I think Ruby's done a lot of that.”

That said, JML says they’ve had some struggles with Rails but saw those struggles as an opportunity to contribute back. Their biggest struggle has been operating, perhaps, the biggest monolith on Rails. To deal with the challenge, they added the concept of modularity into Rails.

“So, a lot of the teams that are working on large monolith Ruby on Rails projects have come to us (how to) start structuring large monoliths with a bit more of a mindset of like they’re operating systems. So, one of the things we have to do outside of Ruby being a great programming language is how do we have a software architecture of a large monolith? And that's some of the stuff that I think we've had to invent.”

Shopify’s “Sneaker Mode”

Tobi asks JML about the “sneaker mode” header he had put up on his Twitter account. JML reveals that the mode is his attempt to troll the sneaker community online that he says has been causing him a lot of pain.

Because of the huge resell value for sneakers, many sneakerheads have created millions of bots to try and get sneakers from Shopify at relatively low prices and sell them twice or even thrice the price.

“They've caused me to stay up many nights trying to keep Shopify up cause they DDoS’ed us. They're doing a coordinated DDoS attack on the platform to try and get (sneakers) when that one sale drops. So we've done a lot of sneaker bot protection and throttling.”

In the podcast, JML goes into how these sneakerheads do it and how Shopify is dealing with this problem. He also talks about the challenge of continually improving Shopify while it’s still in use.

“Shopify is running. People run their business on it. So, we had no pause button. We couldn't go away for three years and come up with a better version and come back and show it off. We have to do it in-flight. But that's what we've done over the last 15 years.”

Shopify Is Recruiting 2,021 Engineers in 2021

Shopify chose early on not to turn to microservices. JML says doing so would have amounted to adding more complexity to an already complex system. They resisted that, instead choosing to retain Shopify as some kind of monolithic operating systems like macOS or Linux.

“It's easier to refactor when it's a monolith when you've made some mistakes than it is when you've got microservices.”

Shopify has around 2000 engineers. It is currently in a hiring frenzy aiming to recruit 2,021 engineers in 2021. Tobi asks JML about David Heinemeier Hansson’s opposition to big teams and his assertion that he would quit if he were the CTO of a company with 1500+ engineers.

“That's DHH’s role in our community, isn't it?” JML responds amid chuckles, “To be the bit of the challenger of everything.”

“My philosophy is, we're going to have to get good at building bigger engineering teams so we can do big things. It's intimidating. It's hard. It's not for everyone. But I think the world's been impacted by some really big engineering (teams).”