Shape Up for Data teams
Only if you are a shaper
I am a very organized person, borderline OCD, so one of the aspects of my role that I put a lot of effort into is how to run the team and plan the work. This part of the job is heavily influenced by the team you work with, the environment and the company culture. There is no point in trying to implement the heavily bureaucratic processes of a big bank in a startup, or pretending that a boomer company can all of a sudden work like a young AI-first tech company.
More often than not I end up with a process that has no name and it is a bit of a minimum common denominator of all frameworks. We plan a quarter at very high level just to surface the major “swim lanes”, we plan each month in a quarter in a more granular way, we have a weekly standup on Monday to discuss what happened last week, what’s the plan for this week, and flag any blocker, and for the rest we collaborate async.
However, I always wanted to adopt a framework in a more rigorous way, and the one that caught my eye is Shape Up.
Shape Up is Basecamp’s product development method, written by Ryan Singer and published as a free book in 2019. The foreword by Jason Fried, 37signals cofounder, is like a manifesto of the company philosophy: no sprints, no daily standups, no backlog, no velocity tracking. Fried waves off the whole vocabulary of anything remotely tied to a metaphor that includes sprinting and being tired at the end.
Most traditional planning asks how long something will take, gets an estimate, and then watches the estimate miss, because, as Singer points out, “estimates don’t show uncertainty”. Shape Up inverts the question. Instead of asking how long a thing will take, you start with an appetite, the amount of time the thing is worth, and then cut scope to fit it. Shape Up calls this fixed time, variable scope. The appetite comes in two sizes, a small batch of one or two weeks, or a big batch of a full six-week cycle. It sounds like a small move, but it flips who is in charge, putting the calendar ahead of the wishlist.
Work happens in six-week cycles, chosen because six weeks is “long enough to finish something meaningful and short enough to feel the deadline from the beginning”. After each cycle comes a two-week cool-down, unscheduled time to fix bugs, breathe, and decide what to do next.
While the builders build in those six weeks, uninterrupted, the senior people shape what might get built next. One of the central pieces of the framework is one meeting: the betting table. During the two-week cool-down period, a small group of senior people looks at a handful of shaped pitches collected in the previous six weeks, and decides which ones get the next six-week bet.
If a project doesn’t ship inside its six-week cycle, it doesn’t get extended automatically. If someone wants to keep going, they have to make a fresh case for it at the next betting table, competing against every other idea that has been shaped during the six weeks.
Status of a project is kept on a hill chart. The idea behind it is that every piece of work moves through two phases:
First comes the uphill climb, figuring out the approach, resolving the unknowns, deciding how the thing will actually work.
Then the top of the hill, the moment you can see the whole solution with no surprises left.
Then it’s all downhill, execution, the part where you already know what to do and just have to do it.
In Basecamp each scope of work gets drawn as a dot on a literal hill, and the team says where their dots sit. A dot still climbing means the work is uncertain. A dot over the crest means it’s basically de-risked. This can be represented in other project management tools with a fixed set of labels, like “Not started”, “Initial exploration”, “Scoped”, “In Progress”, “Wrapping up”, “Done”. If you see that status doesn’t change for a long time, a manager or team lead can see the risk without having to, in Singer’s words, “interrupt the team with the awkward status question”.
One of the most unconventional parts of the framework is this: no backlog. The reasoning is half operational and half therapeutic. Backlogs have a tendency to become a graveyard of old, outdated ideas. Grooming a giant list of old tickets is a waste of time, and “the growing pile gives us a feeling like we’re always behind even though we’re not”. If an idea doesn’t get bet on, you let it go. In theory, good ideas should come back on their own merit, but you are not obligated to carry every thought anyone ever had.
Shape Up is a framework intended for teams that are not too big and have limited dependencies, so that they are in total control of their roadmap. Senior leads have six weeks of time to jam together and shape the next idea. This is harder to apply to Data teams.
While the data platform team is more autonomous and in charge of its destiny, the “frontline” data teams (analytics engineering, data science) are more a support role for the business, so they need to be an integral part of *their* Shape Up cycles. Data leads must be co-shapers with the other leads in the engineering team or the business function supported by the Data org. They must be empowered to bring their perspective and constraints. If engineering wants to ship a feature but there is no good way to measure its impact, either we ship and don’t expect any analytics measurement, or we shape in the groundwork needed to enable analytics. If you are not a shaper, you are a taker. The bets have already been placed by other people, and you take whatever they decided, usually as “can you also help with this”.
The clearest case I have of why a data team can’t adopt Shape Up on its own was a few years ago. We supported four growth PMs with an experimentation capability. Requests came in as issues thrown into a repo, with no prioritization, no discussion, and no venue to decide which ones were worth doing. Priorities changed by the week, sometimes by the day. Attention spans were goldfish-like.
Running an experiment properly is a full program, and it fits Shape Up well: discovery, telemetry instrumentation, power analysis, then post-hoc analysis, or ideally an experimentation platform to automate the measurement. Shaping an experiment means sitting down with the PMs before anything gets built, to agree on the question, the instrumentation, and the appetite. We were never at that table, because they worked in isolation and expected us to absorb whatever came out of them.
If you run a Data team and want to adopt Shape Up, most likely you cannot do that in isolation. You need all your stakeholders to work in the same way and dance at the same tune. Data must be at the shaping table, not only at the building table after betting.
Do you like this post? Of course you do. Share it on Twitter/X, LinkedIn and HackerNews