Guest post by Bob Schatz of Agile Infusion, LLC
The interest in expanding agile development practices is growing at an increasingly faster rate. The good news is that this indicates that companies are seeing positive results in their initial experiences on projects and are looking to further the expansion, even beyond product development. The challenge is recognizing that these early successes required overcoming many obstacles and shifting the mindset of a number of people. And that these things get exponentially harder as you introduce change across the enterprise.
The pressure is also put on consultants, trainers, authors, speakers, agile software tool vendors, and coaches to codify and simplify models to reduce the strain from employing agility at scale or “super-sized agile). When success happens, everyone wants to share the experience, but they often focus on showing the model that worked for them in a specific case. Since the structures are simple by design, this creates a feeling that it must be relatively painless to just replicate the experience in other parts of the company, and then to other companies.
This is similar to what we’ve seen in the past with the basic Scrum process, Lean/Six Sigma, the Toyota Production System and just about any process improvement strategy that has a significant success story to tell. As a result, there has been a flood in the community of scaled agile methods that are all too similar to be called different. Arguments ensue on a daily basis; thought leaders pit their methods against the others trying to prove why they have the right answer. A slew of marketing professionals try to position each method as the better answer in hope of attracting a “big fish” company that will tout how successful the method made them. Right now, we have no less than six scaled agile methods actively being marketed, not to mention the hybrids that companies are presenting in conferences, articles and such. And the methods just keep coming! Ultimately, success will require:
- Identifying the problem you are trying to solve
- Understanding your current organization and its ability to satisfy consumers
- Identifying the gaps
- Designing an organization and process
- Growing your leaders at all levels
- Execution and relentless continuous improvement
In order to bring us back to the ground, I would like to tell a couple of my own stories which might help you and your organization focus on keeping things simple and overcoming the struggles that all organizations will eventually face in order to instill a culture of continuous improvement and process innovation. These will be critical to serving your customers better than anyone else can.
Back to the Future
My first story is from what seems not too long ago, but I guess that’s what all of us “old-timers” say. I came into the software profession as a programmer in 1984. I was hired by GE in Valley Forge, PA in the start of my senior year of college when defense contractors were building up at an incredible pace. I was really looking forward to seeing how the little projects we were doing at school would apply to working on a massive Department of Defense project. And just to be honest, a lot of my school projects involved punch cards. So, the first scaling problem I had was keeping a huge deck of cards in the right order!
The first project I was assigned to at GE was a massive system involving hardware, space vehicles, communications, ground stations, and a host of other components and challenges that made my head spin. The system involved thousands of people across multiple contractors and government agencies all working together to solve a huge problem. It was even more complicated by the fact that this was a classified system where we had to deal with compartmentalization and people with different need-to-know levels that restricted with whom you could collaborate. The project management was mostly visual techniques and we relied heavily on interface control documents, which let us know the inputs and outputs for our piece of the puzzle. (In reality, those documents were not entirely accurate, coordinated, or followed, which caused more than a few defects later in the project).
We all relied heavily on each other to communicate from one component or process to the next. System integrators tried to keep an eye on all of the parts to ensure a smooth system-wide deployment. These projects went on for years. We employed what we now know as the waterfall development model, but at the time we just referred to it as MIL-STD version 2167. Now, I’m not claiming that these were smooth projects. They were terrible and required massive amounts of overtime (which luckily we were paid for), rework, and heroics of some of us to pull it off. These were much different times, but looking back, we did what always worked, and used common sense and a will not to fail. We were motivated to serve our nation’s best interests and we were young, enjoying cashing in those big overtime checks. Organizing ourselves was less of an issue, knowing that we had to solve important problems and everyone was focused on success.
For my 12 years there, the projects got bigger and more complex. We kept using the same organization techniques because they worked for us in those circumstances. Every project required a level of heroics at the end, which nobody would tolerate today. These were different times. Today’s software development environment requires faster speed, much higher quality, and the ability to adapt to a much higher rate of change. My point here is that we didn’t need a model to tell us what to do and, to my knowledge, there was never a Scaled Waterfall Model.
A Burning Platform
Fast-forward through a number of great experiences, including a very successful start-up where I really learned the value of agility in its purest sense before “agile development” was a term. At the end of 2001, I wound up as the vice president of development at Primavera Systems. Now part of Oracle, Primavera makes enterprise portfolio, program, and project management software. This was where we used Scrum staring in early 2003 in an organization with almost 150 people — that became one of the first, and certainly well-publicized agile success stories.
We started off with the goal to improve our lives, our products, and our relationships with other groups in the company. We had serious issues with our product and process. More importantly, our people were suffering. Nothing I tried in all of my leadership experience to date had worked to my satisfaction in my first year there, so it was time to radically re-tool. Scrum was just a leap-of-faith decision to attempt to save us.
Our challenges began with figuring out how to create Scrum teams. We wound up with about 20 of them all working on a single, large enterprise product. We ran into a lot of issues with builds and people stepping on each other, so we innovated. We set our focus on stopping the branch/merge environment and forced ourselves into a single trunk, so everyone was now responsible. We had discussions about how to coordinate dependencies, coordinate development of feature sets involving multiple teams, and how to deal with obstacles that affect many teams. As with everything we didn’t have an answer for, we designed experiments and tried them. This led us to create the idea of having a representative from each team form another Daily Scrum after each team conducted their own to coordinate and deal with cross-team topics. This eventually became known as the Scrum of Scrums. It also led to solving the issue of having 10 product owners learning to coordinate their decision-making by establishing a chief product owner role.
We created the idea of an integration Scrum team focused on testing integrations with other products in our portfolio, as well as third-party integrations. They also did continuous performance testing. We carved out fixed days/times for functional meetings so that directors of development, testing, documentation, etc. could spend two to three hours developing their people and their disciplines. We had teams who organized around certain product areas, architecture, and database development, helping to keep everything coordinated. We kept working on the process to improve and expand it.
Once we had a good handle on our own work, we had all other organizations participate in Sprint Reviews so they could get access to the product being developed. We brought in end users from the very start and just kept getting more and more of them to participate, which really helped us to build a better product. They became increasingly engaged in helping us manage our portfolio and product backlogs. As we moved forward, we dealt with other challenges like having multiple chief product owners, all with their own products wanting to use the same Scrum teams, and learning to manage the flow of work. Later, we encountered the challenges of distributed development and forming Scrum teams in India and the Ukraine.
Through it all, we just kept doing the hard work of solving problems and not looking for the silver bullet as Ken Schwaber had taught us so well. The message is clear; agile development practices are simple by design; implementing them is hard. It requires people who want to improve their circumstances. They want to serve their customers better than everyone else. They are willing to use their smarts to design experiments that will help them get there. And they are willing to fail sometimes, but learn quickly.
So, Now What Do We Do?
The answers to large-scale agility do not lie in a book, or a set of slides, or a poster. Every project that has more than one team is a project at scale, and every environment is different. The answers will emerge when people are willing to put in the work. There are no shortcuts; seeking shortcuts will only result in change initiatives that will not survive the test of time. Stories like these can help spark ideas, but they are not to be followed like a recipe. Doing so only sets us up to repeat history. So, the first question is not which model to follow, its asking yourselves “What problem are we trying to solve?”