Elements of Agile in Hardware and Physical Product Development

Building cyber-physical products, hardware, machines, modules and components with agile teams is challenging and often needs contextual customisation to be effective. Dr. René Hussong and Ali Hajou are talking about concepts, common practices and learnings from the field.

In this KEGON blog article Ali Hajou and Dr. René Hussong continue the valuable discussion started in Ali’s webinar on “Agile for Physical Product Development”. Missed the session? 

Listen to the recording!

Ali, during your webinar on “Agile for Physical Product Development”, you mentioned the importance to organize teams around the product. To be more precise: to organize around the architecture of the product. Could you elaborate a little bit more on that?

I have been very much involved in the notion of stable teams which are organized in a form and shape where the engineers can keep on working together, far longer the lifespan of a project. The habit of pulling teams apart has been a painful side-effect of what we call “resource management”. Resource Management combined with the continuous tyranny of the urgent degrades smart and capable engineers to interchangeable tools to get a job done. While this is not great for the engineers themselves, the eternal hunt for capacity is definitely not great for your project development. The question therefore is, what is the alternative? Well, the alternative is: Stable Teams.

How “Stable Teams” look like in ‘your’ scenario highly depends on your context. Your context might require thousands of engineers to develop a complex product, or you might be in a context where your total team consists of 10 experts running various projects. Highly different contexts with different nuances.

A nuance that is generalizable is the notion of organizing your teams around a long living technological construct: a product family, a product, or a module. The company you work for has been building and will be building the product family, product, or module for a long time, and will maintain the installed base. Hence, since that is a constant factor in the lifetime of ‘your’ company, why not organize around that?

How is this in line with the statement “organize around value”? It seems that for complex cyber-physical systems we cannot only focus on the final customer value, not even on the value that might be produced for an internal customer, like another team or another Agile Release Train (ART). Should we focus more on the job(s)-to-be-done that single parts of the system fulfill?

A large and complex system might be designed, industrialized, and manufactured by thousands of people. Anyone mildly knowledgeable in Agile Frameworks will know that Scrum describes the Scrum team a to be most effective when it contains 10 or less people. The Scaled Agile Framework describes the Agile Release Train to be a virtual organization which operates best if it is smaller than ~125 people. But what about a thousand? A system consists of various interlinked parts, which, depending on the system, are individually sellable parts. Think of modules of an aircraft, car parts, or modules of a conveyor belt. The parts are kept on stock and are sold separately, even though we know the product to be “the aircraft”, “the car”, or “the conveyor belt”.

I have experienced that the term “value” does not resonate with engineers very well. Engineers work on a small component of a larger module or system which they do not have full control over. Hence, what delivers “value”? Rather than falling into the trap of using too much Agile terminology (somebody I admire called it “Agile Mumbo Jumbo”), I prefer the ability to communicate in language engineers understand: ‘product functions’ or ‘product architecture’. A products’ architecture contains various modules which contribute to a systems’ function. It is the function which brings value to the customer. Hence, ideally, we organize around the functions of the product. This ensures the inability to say “but my component works…”. Constraint by the competences you have in our team, you might have to slightly deviate from product function orientation.

Is that not in contradiction to our general idea that Agile Release Trains should be formed mostly by stream-aligned teams and not component teams?

I have noticed that the concepts as described by Skelton and Pais (book: Team Topologies) are valid, but might need a rebranding amongst engineers. Hence, it is not a contradiction, but rather a rebrand. Given the situation (and the products we develop), we need a blend of mono-competence teams and multi-competence teams. Some of the teams are focused on commonality and platforming (e.g. re-use of technologies), and others are focused on customer facing product capabilities (e.g. housing, packaging, HMI, etc).

By having so many teams involved, all of them working on complex system(s), the use of the double V model or “W” model seems to be appropriate. Could you tell us a bit more about the “Dual Vee” model and the ideas behind that?

The Dual Vee (from Forsberg and Mooz, 2006) adds a dimension to a V-model. While a large project simply goes through certain phases, it does not mean that engineers have to dogmatically stick to the phases. Ideally, teams own “the left side of the V and the right side of the V”, indicating that each team has the ability to define (detailed) specs, build something, and perform validation and verification on their own level. This allows an iterative way of working on each level. While the project navigates downwards through the V, different types of more specialized (and often more mono-competence) teams start working on the product in, you guessed it, iterations. The Dual Vee model blends the ability to work in iterations while maintaining (budgetary and quality) control on the level of the project.

Hence, the Dual Vee model fosters a shift-left approach, where we try to bring tests and verification closer to objects and artifacts to be tested, correct?

As a matter of fact, the Dual Vee model emphasizes early tests, integrations, and verifications. The ability to iterate yourself through the difficulties of product development only enhances the ability to manage project risks. It is painful to work this way, as it sometimes is easy to write a specification document and throw it over the wall to another team, only to figure out months later that the specs have been interpreted incorrectly. Would you like to have one large surprise at the end of the project (and handle the headaches of escalations, fire-fighting, and panic), or many small surprises at the end of every 2 weeks (and handle small headaches, manageable escalations, manageable campfires, and frequent moments of nervousness)? I will choose the latter over the former any day.

Great Ali, thanks for these valuable insights. One final question: Latest since Covid-19, we have all heard about the so-called rolling review procedure, where regulatory requirements have been fulfilled and verified in an iterative and agile manner. Can you elaborate a little more on that? How do you deal in practice with intensive regulatory requirements like ISO 26262 for functional safety e.g.?

Rolling Reviews have been described by Sutherland and Schwaber in 1994 as the “Sprint Review” and as “Evolutionary Delivery” by Tom Gilb in Evolutionary Project Management (1960s). Covid-19 forced us to become more involved with each other (design/development and quality control/verification). Everybody must agree that joining a meeting every 2 weeks is way more taxing (and requires way more involvement and dedication) than a dedicated month at the end of a 1-yr development cycle. But, “Nood Breekt Wet” as we say in Dutch (paraphrased as “Necessity Knows No Law”). If we reaaaaly need to get things done, we somehow organize war rooms, with a team having multiple competences. We give them the support and protection they need and we trust them to get the job done.

The uncomfortable message here is that, in order to achieve true agility, we need to collaborate amongst all the competences required to build qualified systems. That includes the close collaboration with safety managers and their delegates. If the plan-do-check-act cycle is reduced to two weeks, then activities to remain compliant (f.i. Functional Safety) should be part of those two weeks. There is so much value in building your deliverables incrementally, and being semi-compliant all the time. And on top of that… it feels good to be in control!