How to best integrate ML into your software process
Here is to why starting a “Machine Learning team” is a big mistake.
Over the years, as ML became an integral part of product development, I have witnessed the journeys of many companies/startups to integrate Machine Learning (ML) into their software development process. Compared to non-ML software, ML requires a different calibre, operates on a different cadence/process, and involves a much higher degree of uncertainty, which presents a challenge.
Team structures in the Tech industry
Let’s think back to how team structures evolved in the tech industry. We started out with technical teams such as front-end teams and back-end teams, and those of us old enough will remember the time when ‘mobile teams’ were added to our companies. However, this structure involved having to talk to too many people to execute on a project that required multiple skill sets. And hand-offs were a nightmare.
In came agile software development, and the new standard became self-organizing cross-functional teams. Spotify took that to the next level when it formalized the concept and practices around it, and gave us handy terms to use such as “Squads” for self-organizing cross-functional teams.
“Squads” became the new industry standard. According to Marty Cagan, the FAANG norm is “Empowered” cross functional teams that are given business context and goals, then allowed to innovate and iterate over problems.
ML teams
Because ML is such a specialized discipline and one that is less understood and more difficult to manage, the default arrangement for Machine Learning teams is to bring ML folks together in one team that is managed by a technical (ML) manager.
This setup of separating out ML teams has multiple advantages:
Creates a sense of community for ML folks and provides a good setup for ML knowledge sharing including tooling.
Because the ML work is gated by a technical ML manager, this should mean the work coming to ML folks has already been vetted by an ML person who is also a manager. When ML folks are part of a cross functional team in an org that doesn’t understand ML very well, they are sometimes asked to work on non-ML things, which they hate, and is a global loss of efficiency for the company.
However, the disadvantages are that you lose out on the advantages of cross-functional teams:
A lot more overhead, loss of speed, and loss of autonomy for individual contributors.
More importantly, you lose out on innovation. When ML teams are separated out, ‘requests’ have to go to them through a technical manager who doesn’t have the full business context. Exaggerating just to make the point, this setup is like a service desk that is quite removed from the business/user problem.
Cross-functional ML teams
I strongly believe that integrating ML folks in cross-functional teams is the best org structure for the following reasons:
In a dynamic ever-evolving tech scene, prioritizing the structure that optimizes for innovation and access to business context increases your chances of success.
While splitting out ML folks into different teams might lead to a loss of a sense of community, this can be compensated for rather easily. Technical folks love to build technical communities, and the technology is the same if people change companies/orgs. However, it is extremely difficult to imbibe business context that’s not provided by your immediate org structure. Business/product goals/context vary company over company and org over org.
ML folks are sometimes worried that being part of a cross-functional team will lead to them working on non-ML things. If true, this concern is by nature temporary, and should go away with time/education as the company gets more familiar with ML.
If you are worried that ML folks need an ML direct manager, you can adopt Spotify’s matrix structure.
Tiger teams
If your ML team is quite small because you’re in a startup or just starting out an ML journey, you might have ML folks serving different teams by necessity, and it makes sense to have them grouped into an ML team.
One way to get the best of both worlds that I’ve used in the past is Tiger teams. You can maintain a technical ML team, but clearly assign ML folks to a product team, or part of it, for a fixed duration (such as a quarter or say 3-sprints) to solve a certain problem. Create a temporary Tiger team comprised of ML and non-ML folks, give it a name if you have team names, create a separate Slack channel for it, and have it conduct its own Agile rituals. Do whatever it takes for that team to really feel like one team.
Where do I start
No org structure is perfect; each one has pros and cons. My best advice is to be aware of the inherent cons of the structure you have and compensate for them. For example, a technical ML team achieves a certain degree of success if its manager has good eye for business and strives to “integrate” his team members into the business/product problem.
If you are not familiar with Spotify’s Engineering culture, I highly recommend taking a look. Marty Cagan has a lot of advice on achieving an Empowered product team structure, and both of these resources are great, even if we set aside the ML question.
Team structures is a topic I’m passionate about, and I’m always happy to chat and help out people on their journeys to unlock the potential of their teams. So send me a message!