Patterns of Effective Delivery - Challenge Your Understanding Of Agile (RootsConf 2011)
Highlights from Dan North's excellent, inspiring, and insightful talk Patterns of Effective Delivery at RootConf 2011. North has a unique take on what agile development is, going beyond the established (and rather limitied and rigid) views. I really recommend this talk to learn more about effective teams, about North's "shocking," beyond-agile experience, and for great ideas on improving your team.
The talk challenges the absolutism of some widely accepted principles of "right" software development such as TDD, naming, the evilness of copy&paste. However the challenge is in a positive way: it makes us think in which contexts these principles really help (in many) and when it might be more effective to (temporarily) postpone them. The result is a much more balanced view and better undestanding of their value. A lot of it is inspired by the theory (and practice) of Real Options.
What are Patterns of Effective Delivery?
Some of the patterns take years to master and require investment to learn and start getting the benefits. You might need to try (and fail) a few times before getting them right.
Disclaimer: These are notes that make sense to me. They will likely make only limited or none sense to people that haven't heard the talk. It would be best to go and listen to it instead.
Ex. of spike-and-stabilize regarding test naming: originally named blah - don't know what it should do yet, experimenting, when the code evolves into st. meaningful, name it properly, according to that.
Ginger Cake - copy and paste code, rip unrelevant things out until the only important things left, then write tests around; may end up with code that is similar *but not in the ways expected* => if started with abstracting, it would be the wrong abstraction. It says: "We know and respect DRY but are not slaves to it."
Short SW Half-Life: 1) We don't care about the SW but the utility it gives us; if writing it gives us better ideas, we can delete it and do the better thing; 2) how would you write the code if 1/2 of it - but you don't know which half - would be gone in a few weeks? => start simple (see Spike & St.); extract commonalities, improve quality etc. for code that has already been around for a while and has proven itself useful; Some architecture styles lend themselves better to such quick evolution - small, focused services (popularly known as micro services (slides, esp. p.42+)). "Look at the code as it evolves and decide what to invest in." (The investment includes thinking about the design.) All code is not equal.
Create Urgency - to change a paradigm, the way of thinking, people must be desperate, have no more options, and have the knowledge what to do. => apply when learning st. new - do it on st. real, under self-inflicted pressure. Ex.: Commit to do an app, with a crazy deadline, using the new tech => urgency, no more options. Forces to learn only the parts you really need x diff. than what tutorials teach.
Socratic Testing (coaching style) - don't tell the team what's wrong with their code, which is threatening and thus hard to accept. Pair with them on writing test and to support the test, make "helper" classes etc. that you'd like to see in the prod code. If they really are useful, they will spot it and decide to pull them into the prod code. Make them the hero, don't tell but ask.
Fits In My Head - we need code that we can understand and reason about (x big classes, methods, complex models, ...). Keep the code simple, optimize for understandability, readability, obviousness, .... Build Shared Idioms in the team - so that the team members would, given the same context, arive to the same decisions/design. Something should only differ from the usual way of doing it when there is a good reason for i; thus a difference provides a hint, difference is data. F.ex.: all communication over ZeroMQ, only at one place through shared memory - this indicates there is some, most likely performance, reason for it; communication means shouldn't be picked randomly, ad-hoc.
TDD - a pattern that, in a *particular context*, may make you much more effective
Explict tips from the talk:
But: NO SILVER BULLETS - This stuff is hard - Versioning, Integration, Testing, Deployment; eventual consistency hard for people to wrap head around; ... .
Note: Comoyo.com, powered by a number of ex-googlers and other smart people, does the same thing. So does, I believe, Netflix.
The talk challenges the absolutism of some widely accepted principles of "right" software development such as TDD, naming, the evilness of copy&paste. However the challenge is in a positive way: it makes us think in which contexts these principles really help (in many) and when it might be more effective to (temporarily) postpone them. The result is a much more balanced view and better undestanding of their value. A lot of it is inspired by the theory (and practice) of Real Options.
What are Patterns of Effective Delivery?
- Patterns - strategies that work in a particular context - and not in another (too often we forget the context and to consider the context where a strategy doesn't work / is contra-productive); beware: a part of the context is the experience of the developer; for unexperienced devs it might be better to just stick to a process and applying TDD etc. all the time than trying to guess when they are appropriate and when not without having the experience to decide it right
- Effective - optimize for something: volume of SW produced? time to market? learning/discovery? certanity? user experience?
- Delivery - get stuff that is useful out of the door; software is not important, the utility it provides is; know why you write the SW to be able to get better at it
Some of the patterns take years to master and require investment to learn and start getting the benefits. You might need to try (and fail) a few times before getting them right.
Disclaimer: These are notes that make sense to me. They will likely make only limited or none sense to people that haven't heard the talk. It would be best to go and listen to it instead.
Selected patterns
Spike and Stabilize (or throw away): traditionally we decide whether we are writing production-grade code (with high rigour such as TDD) or just a throw-away spike before we start coding - i.e. at the moment when we know the least about it. We should rather not decide this uprofnt but "exercise the option of investing into the quality" later, based on experience. Start as a spike and if the code proves valuable, stabilize it, refactor, test etc. Evolve the code based on experience (good naming, quality), defer the commitment to the quality of the code => optimize for learningEx. of spike-and-stabilize regarding test naming: originally named blah - don't know what it should do yet, experimenting, when the code evolves into st. meaningful, name it properly, according to that.
Ginger Cake - copy and paste code, rip unrelevant things out until the only important things left, then write tests around; may end up with code that is similar *but not in the ways expected* => if started with abstracting, it would be the wrong abstraction. It says: "We know and respect DRY but are not slaves to it."
Short SW Half-Life: 1) We don't care about the SW but the utility it gives us; if writing it gives us better ideas, we can delete it and do the better thing; 2) how would you write the code if 1/2 of it - but you don't know which half - would be gone in a few weeks? => start simple (see Spike & St.); extract commonalities, improve quality etc. for code that has already been around for a while and has proven itself useful; Some architecture styles lend themselves better to such quick evolution - small, focused services (popularly known as micro services (slides, esp. p.42+)). "Look at the code as it evolves and decide what to invest in." (The investment includes thinking about the design.) All code is not equal.
Create Urgency - to change a paradigm, the way of thinking, people must be desperate, have no more options, and have the knowledge what to do. => apply when learning st. new - do it on st. real, under self-inflicted pressure. Ex.: Commit to do an app, with a crazy deadline, using the new tech => urgency, no more options. Forces to learn only the parts you really need x diff. than what tutorials teach.
Socratic Testing (coaching style) - don't tell the team what's wrong with their code, which is threatening and thus hard to accept. Pair with them on writing test and to support the test, make "helper" classes etc. that you'd like to see in the prod code. If they really are useful, they will spot it and decide to pull them into the prod code. Make them the hero, don't tell but ask.
Fits In My Head - we need code that we can understand and reason about (x big classes, methods, complex models, ...). Keep the code simple, optimize for understandability, readability, obviousness, .... Build Shared Idioms in the team - so that the team members would, given the same context, arive to the same decisions/design. Something should only differ from the usual way of doing it when there is a good reason for i; thus a difference provides a hint, difference is data. F.ex.: all communication over ZeroMQ, only at one place through shared memory - this indicates there is some, most likely performance, reason for it; communication means shouldn't be picked randomly, ad-hoc.
TDD - a pattern that, in a *particular context*, may make you much more effective
Bonus: Micro Services
From James Lewis's talk Micro Services: Java, the Unix Way (2013) - especially slides 42+:- Use web, do not bypass it - REST, JSON; standardised application protocols and message semantics
- Small with a single responsibility (does one thing, fits into one's head, small enough to rewrite and throw away rather than maintain)
- Containerless and installed as well behaved Unix services (executable jar with embedded Jetty + rc.d start scripts and config files)
- Avoid unnecessary coupling - Domains in different bounded contexts should be distinct – and its ok to have duplication, physical separation to enforce it ; there will be common code, but it should be library and infrastructure code; leverage Conway's Law to support decoupling
- Provisioned automatically: "The way to manage the complexity of many small applications is declarative provisioning" (including instance count, scaling, load balancing)
- Status aware and auto-scaling - in-app status pages, monitored => autoscaling
- Each service is entirely decoupled from it’s clients, scalable, testable and deployable individually
Explict tips from the talk:
- Divide and conquer Start on the outside and model business capabili3es
- Use Conway’s Law to structure teams (and enforce decoupling)
- The Last Responsible Moment - Don’t decide everything at the point you know least
- Be of the web, not behind the web
- If something is important, make it an explicit part of your design (reify) - ex.: inst. of services creating users by posting to /user, they post a user creation request and get response immediatelly, the user created eventually (reminds me of futures)
- Favour service choreography over orchestration
- Use hypermedia controls to decouple services
But: NO SILVER BULLETS - This stuff is hard - Versioning, Integration, Testing, Deployment; eventual consistency hard for people to wrap head around; ... .
Note: Comoyo.com, powered by a number of ex-googlers and other smart people, does the same thing. So does, I believe, Netflix.