August 2007 - Posts

• Enterprise Agile Generations

  Recent events in my professional life have placed me in an Agile evangelistic position that can help permeate the adoption of Agile development throughout my companies global presence.  I am really excited about this opportunity but at the same time apprehensive.  I have spent the last 4 years learning and refining my understanding of many of the Agile development methodologies.  I know of its shortcomings and I know of its many strengths, which leads me to my quandary.   “The many flavors of Agile development, while powerful are very sensitive to some degree.  The Adoption of these processes and practices require cultural changes from management all the way down to development and testing.  The failure at any one of these points can lead to catastrophic break down in the alleged implemented Agile methodology that leaves adopters with a rather unsettling perspective of Agile all together.”   In order to deal with simply having organizations give up, I was thinking of this evolutionary scale for Enterprise Agile adoption.  I will refer to these evolutionary tracts as Generations.  This is not an attempt to rate Agile maturity as some organization may exist fine in Gen 1 for a period of time and have great success.  This is merely my perception on the possibilities of growth in Agile adoption to a certain state of development nirvana.  (Do you hear the flowers singing…)  J.  (If there is already a term for this out there, please let me know.  )     Overview Management Perception Business Perception Developer Perception Tester Perception Gen 1 This organization is adopting Agile for this first time.  Their management team has agreed upon some sort of methodology that promotes an iterative incremental approach to developing software with daily feedback back meetings and release retrospectives.   Prominence is placed on self organizing organic team structures. Gen 1 organizations emphasis is on the process and has not adopted software engineering practices such as unit testing, pair programming, continuous integration etc. Gen 1 organizations have yet to embark on a project and are in the proof of concept phases of the process. Skeptical at best.  Very few managers actually see Agile as a viable solution to software development.  In the past Agile was viewed as more of a passing fad then a proven discipline to developing software.  Culturally this is changing but some managers will want to stick to what they know best.  After all they have been completing projects in a traditional fashion for the last 30 years, why should they change.  Pay close attention as management is looking for any reason on why they shouldn’t use this process. It’s not that management is trying to stop Agile, it is just that they don’t understand it and have no idea where they fit in this equation.  Emphasis is on education and patience.   A bitter perception similar to management’s.  You are bringing in Agile for a reason and more than likely that reason is because you haven’t been delivering software on time and with features that the business has requested.  Agile is looked at as another attempt for IT to justify spending more money with the intention of delivering quality software.  They view IT as a used car salesmen.   Over promised and never delivers! Emphasis again is on education.  In the past I have held product owner training just for the business.  Remember they are going to be your partner in all of this, not just some stakeholder that has a deliverable to you.   Some of your developers will be excited at the fact that the term Agile is even being considered.  Others will seem less enthused as it is management trying to change the machine again.  “If they would just listen to me and do it my way, I can solve our problems” mentality. Since Gen 1 focuses more on process than software engineering practices, the development team’s focus is on adjusting to time boxed deliverables.  This concept in general takes time for teams to adjust to. Your focus is to become a leader and coach.  You must learn to enable you development team to become one cohesive unit that works well with the business and quality groups.   Similar to the development perception but they become more and more frustrated as the project continues as they are expected to deal with evolving requirements. From their perspective this process goes against the culture that they have become accustomed towards.  They are use to working with completed requirements. Careful attention must be made to insure that the Testing group plays an active role in the adoption of Agile.  Testers tend to feel that Agile is a development only approach to designing software that isolates them from the process.  This cannot be further from the truth as their input is greatly needed in order to insure successful completion of business features.   Gen 2 This organization has successfully implemented Gen 1 projects roughly on time and on budget.  The emphasis is still on the refinement of the process and communication.  The team has now realized the value of time boxed goals and rapid feedback of working software to the business. Trust of team members forces the team to focus on producing working software. Maintenance cycles are now taken into account as coding quality becomes a necessity in order to decrease maintenance cost.  At latter stages of the project embracement to change is still frowned upon as complexity of architecture and the defect ripple effect to change is viewed as too high of a risk. Managers now have a greater appreciation for this thing called Agile.  Scrutiny is still given at certain points in the process such as planning but they understand that issues that would have been uncovered late in the game are now discovered in a timely matter and dealt with immediately through backlog prioritization.   They start to appreciate the fact that they are now having to deal with important issues such as impediment removal and keeping the team focused on the goals of each sprint.  Their roles have changed from task masters to mentoring and growing a team.   Leadership is paramount. They like that they get to go home at normal hours. Management is surprisingly comfortable in Gen 2 as it feels familiar to traditional development methodologies they have followed in the past.  They see it more as micro Water Fall. They are happy, not ecstatic but happy.  They learned that Agile development allowed them to be a vital contributor to developing software and not a document monkey. They appreciate the rapid feedback and watching the software evolve throughout the entire project. They are still a little apprehensive at the fact that some of their enhancements were not welcomed so late in the game.  But are ok with it since they were still in control of the backlog throughout the entire project. They are apprehensive about if Agile is capable of handling a maintenance line as well. Developers appreciate the fact that they are developing working software early on in the process.  The term “done” has a whole new meaning encompassing value and completeness. The elective structure of work assignments is viewed as a huge benefit to the team.  Each member has learned parts of the system that they would have never had the opportunity to learn in other environments. Team concepts of trust, cooperation and continuous improvement have a higher degree of value. The development team is eager to explore more Agile software engineering practices. There may be some developers who are still reluctant to change and still want to go back to more traditional methods. They love that they work normal hours now.   Unfortunately the testing group’s perception remains unchanged.  They do not appreciate that every sprint they have to retest the entire application over again.  At this time this is still a manual effort. While their trust of development has increased, Iterative changes to requirements are placing a drain on test, retesting efforts. They see Agile as still focused on development but has little focus on more traditional QA approaches to software testing. The reason that the testing group is having such a hard time is they have not been given the training or the tools on how to automate acceptance test incrementally.  Time must be taken to insure that the testing team is brought up to speed on automation. Gen 2.5 In order to address the issues of code quality and adapting to change rapidly this organization has started to adopt software engineering practices such as unit testing, automated acceptance testing, paired programming and continuous integration in addition to their already proven Agile process. More metrics are introduced in an attempt to gauge quality and velocity accurately. Architectural impediments are starting to become apparent on existing systems when embracing new enhancements that are based on business rules that weren’t completely understood early on in the project. (Note: Judgment call must be made  if the Gen 2.5 practices  can be introduced into Gen 2)   Managers have a hard time dealing with the perceived increase in time spent on development task due to the fact that the development team is now creating test first before even creating application code!  Not to mention that they are witnessing a doubling of effort on task that use to take a single developer less time! There is a tendency from management to want to forego these engineering practices and step back to Gen 2  after all it was working and what’s a little more time spent on QA toward the latter stages of the project.  We have managed to launch software in the past with defects. This is a critical milestone concerning the Agile evolutionary tract.  Overcoming this mind set of inefficiency will lead to greater rewards and increased confidence of development team going forward.  “What do you mean that new form I asked for, that is similar to an existing form, is going to take twice as long as before?” Quality; isn’t that what the QA group is for? Their concerns are similar to management since tasks are taking longer than before to complete.  It isn’t till latter releases in the project that they begin to realize that the defect rate is significantly lower. They are amazed that their ideas and changes are welcomed during latter stages in project. Care must be taken similar to the management tier as this is a critical milestone that must be overcome to witness true agility. Developers are very apprehensive about XDD development.  The self discipline that XDD requires seems tedious and unneeded to some developers.  Senior developers who do not see any benefit will immediately withdraw. Paired programming is not as big a deal since the team has been working together through Gen1 and 2.  You will have some developers who refuse to pair with others. The development team reverts back to the storming phase of group development until all these new practices become culturally instilled in their day to day activities. Typically these software engineering practices take around 1 to 2 releases before the team starts to see immediate benefits to productivity and confidence in their architecture. Training by an experience Agile development coach is critical! Testers must become accustomed to writing their test cases in a verity of tools.  Each case is unique to the development environment but all have a steep learning curve. The testing team usually starts to see the benefit of automated acceptance testing after the 3rd to 4th Sprint.  You want to see big smiles just look at a QA technician after he knows he can rely on his automated test to do the work for him. By incrementally growing the acceptance test after every sprint the QA teams works in tandem with the business to insure that the acceptance test are in line with their expectations. Gen 3 After becoming proficient at software engineering practices of Gen 2.5 the team starts to adopt Domain Driven Design to address the architectural issues that are becoming apparent.  (Note: DDD can be introduced in Gen 2.5 if the team is not overwhelmed with learning the engineering practices in Gen 2.5) The team forms a ubiquitous language around common object contexts where the software becomes an expression of the physical model that represents the problem domain. Distributed Agile concepts begin to emerge as the need for large scale enterprise software projects are needed.  The success of the organization on previous projects becomes a logical candidate for exploring Distributed Agile. Managers are your best friend now.  “You want to try something new, Go for it!” Adaptation becomes instilled in the management culture. Because they have been allowed to grow with you in your experiences, they trust the group and are open to new ideas. They start to become evangelistic in their senior management meetings and begin to communicate principles and practices to their peers. Others groups become increasingly interested in what this organization is doing. Distributed Agile is viewed as a simply an extension of an already proven process.  While this is true to some extent, great care must be taken to insure that the feedback loops are incorporating feedback from all fronts. The business becomes apprehensive again since another concept is being introduced.  DDD is questioned as to its validity since IT has been delivering software that they have requested.  Emphasis must be made that DDD is trying to solve the business problem domain and form a model that every stakeholder can talk to.  This will help to address gaining greater elucidation into business rules to allow for the architecture to become even more malleable than before. Distributed Agile is something that the business has to play an active part in since they are playing a role on usually several fronts.  Conceptual Contours of the architecture help to focus several business teams on problems domains. Developers feel pressured about learning new terms for concepts that they are already familiar with.  Arguments ensue within the team about what is an Entity and what is a Value object. Lower level developers will want to explain the model simply as patterns and not see them from a domain perspective.   If they lack soft skills such as PATIENCE and high level communication they will become very discouraged with this process. DDD will probably be the most challenging discipline to master.  DDD requires that the developers have a high level of soft skills in order to talk to the Domain Model. As the Business, Developers and Testers begin to talk towards the model.  The problem domain evolves much more quickly and nuances that would cause havoc at later stages of the project are found early on.   Tester really appreciate DDD as the ubiquitous language coupled with modeling exercises help them to capture nuances in their acceptance testing that would have gone unnoticed before. There is a higher degree of business rule coverage from all aspects of testing. Besides being tester DDD is allowing all members of team to gain a greater understanding of business.  Gen 4 This organization is fluent in many of the Agile methodologies.  They are not dogmatic in their approach to Agile development and see ALL Agile methodologies as a toolbox of practices that can be aligned to create high quality working software that is beneficial to their business customers. They are looked at as innovators and trendsetters within their organization and the Agile community worldwide. They understand that Gen 4 is merely another step on the evolutionary tract to Gen 5 as process, practices and mind sets can constantly improve. (flowers are singing) J You make them look awesome and they love you for getting them all these bonuses! Now would be a good time to ask for that raise if you haven’t done so already. Is there anything you can’t do?  The business is still trying to find ways to squeeze out as much as possible out of IT for the money. The main point here is that there is complete trust that the team as a whole can get the job done with any project. At this point the development team is feeling 10 feet tall and ready to concur a small country. This team is fully aware of their limitations and strengths. Their understanding of Agile software engineering principle is sound and they take every opportunity to constantly refine their craft. Some developers may consider becoming coaches to help other development team come up to speed on Agile Development. About the same as the developers perception, without the, concur the small country feeling.         The graph below depicts the maturation rate of small, medium and large enterprises.  Based on yours truly experience..LOL     As you can see it should take on average approximately 3.5 years for a large enterprise of over 1000 people in IT to adapt Agile to a Gen 4 state.   OK I am tired let me know what you all think.  I am looking forward to your comments. Posted Aug 30 2007, 10:57 PM by Joe Ocampo with 5 comment(s) Filed under: Agile Project Management, Agile Project Coaching & Management

• Behave# and NSpec Nirvana, Getting Close

  Jimmy hasn't been sleeping lately and has really put allot into the latest trunk of Behave#.  His latest post shows you how to combine Behave# with NSpec syntax to create very fluent, readable code. He also answers one of the most common questions we get asked, "When to use Behave# and when to use, well other existing testing frameworks". Also in the works is combining the NBehave project with Behave#!  More to come on that later. Happy coding! Posted Aug 27 2007, 03:55 PM by Joe Ocampo with no comments

• Behave# in Action

  Nelson Montalvo has been gracious enough to post his experiences working with Behave#.  Please make sure to check out his post.  His point about the discovery of the his API through the story definition is very encouraging. Posted Aug 16 2007, 04:00 PM by Joe Ocampo with no comments Filed under: BDD (Behavior Driven Development), Domain Driven Design (DDD), Behavior Driven Design (BDD)

• Great Dane, Golden Retriever, Chihuahua == Developer Estimates

  I just read Ayende's and Udi's post on estimation.  I was intrigued because over the last three year we have been using "story units" to estimate our work load for all our projects to great success. Story Units (points, Bellware calls them fibs) are not for the faint at heart as there accuracy tends to increase as a team develops more software together.  This is due in part to the central limit theorem. Mike Cohn says it best: The Central Limit Theorem tells us that the sum of a number of independent samples from any distribution is approximately normally distributed. For our purposes, this means that a team’s story-unit estimates can be skewed way towards underestimation, way towards overestimation, or distributed in any other way. But when we grab an iteration’s worth of stories from any of those distributions, the stories we grab will be normally distributed. This means that we can use the measured velocity of one iteration to predict the velocity of future iterations. Naturally, the velocity of one iteration is not a perfect predictor. For example, an iteration containing one ten unit story rather than ten one-unit stories would be a less accurate predictor. Similarly, velocity may change as a team learns new technology, a new domain, or becomes accustomed to new team members or new ways of working. My own personal experiences proves that this theorem is factual.  When the development team was forming, our story estimates were all over the map.  One developers 1 unit (12 ideal hours) story would turn into an 8 unit story by the time the team completed it.  Another developers 3 unit story would take 0.5 units. Over the course of each iteration our accuracy increased and the velocity became consistent. For those of you that have never heard of story point estimating let me break it down for you.  Agile project terms differ from process to process but they have the same basic similarities. A basic project looks like this: As you can see there is a basic inception phase that the project team uses to estimate the duration and cost of the project.  The product owner gives a high level scope of what the application will accomplish.  From this high level overview a vision statement is created by the entire team.  During this phase the product owner outlines the high level features of how the application will function.  Try to not confuse this with a work break down structure exercise it is similar but doesn't go into that level of detail.  So lets say the team came up with the following list of features. Welcome Screen Login Screen Create Customer Look up customer Create lead Admin: Create Campaign etc.... From these list of features the development team gets together and plays planning poker with each feature. When the team comes to a general consensus on the size of the feature, a unit value is assigned to it. Here is where my dog analogy comes into play. Lets say the first feature is valued as a Golden Retriever.  You simply look at the next feature in relation to the first and ask yourself one simple question, "Is it bigger or smaller than the Golden Retriever?" If it is smaller how smaller? Chihuahua or Poodle?  Well I am sure you get the picture, the point here is not about accuracy as more as it is volume. After you have gone through all the features you simply add the units up. Welcome Screen - 1 Units Login Screen - 3 Units Create Customer - 8 Units Look up customer - 6 Units Create lead - 6 Units Admin: Create Campaign - 8 Units Total: 32 Units Now you have your units for all the features.  This is where I am going to diverge from traditional agile practices simply because I am in a enterprise environment and I answer to a PMO so I have to come up with a target completion date and estimated cost.  To do this I am going to have to rely on iteration velocity. Within each release there are a certain number of development iterations. Most teams typical iteration length is 2 weeks, I prefer 1 week iterations as it accelerates the feedback loop. Each iteration is given a development velocity.  Velocity is a measure of how much work the team can accomplish in an iteration.  As an example lets say you have a team of 4 developers and they have been working together for sometime.  Their velocity is 8 units per week.  On the other hand lets say you have a new development team of 4 developers.  Their velocity would probably be half of the seasoned teams velocity.  It would probably (hope) take around 6 iterations by the time this new team is able to match the velocity of the seasoned team.  There is a huge theorem I use at work to predict this but I don't want to bore you here.  The main point is velocity is not a constant it is simply a measurement of estimated performance. A typical release might look like this. So you have the following phases in a release: Storming Planning Development Iterations A (8 Units) Development Iterations B (8 Units) SIT UAT Production Deployment Velocity for the release is 16 units and the release is approximately 3 weeks in duration.  So in 3 weeks you can have 16 units worth of working software ready to go into production. Lets go back to our Feature list example. You know the total count is 32 units, based on how awesome your team performs the entire projects is going to take approximately 2 releases to complete.  Because you are a seasoned project manger you plan for one additional release in reserve.  The total project estimate is 9 weeks.  From there you can figure out person hours on your own. Release Planning Estimation: Planning Poker Part Duex Great now you are into your first release but what are you going to work on.  DO NOT make the fatal mistake of using the high level estimates you did during the inception phase to plan the actual stories for the release, those where just estimates! For the sake of time lets assume that your project owner has given you a list of features they would like accomplished for this release.  What you do now is sit down with your product owner and go over each feature that has been prioritized for this release utilizing DDD modeling you break the feature into stories.  Once each story has been defined.  The developers take some time to break the story into task and estimate them at a finer of level of course using the planning poker scheme as before. Story: Blah blah blah - Original Estimate (3 units) Task: create Blah entity  -  0.25 units Task: create blah database table - 0.25 units Task: create UI mapper for Blah - 2 units Task: create web page elements for blah -  2 units Task: create blah validation service - 0.5 units New Estimate after task analysis - 5 Units Story: foo foo foo - Original Estimate (5 units) Task: create Foo entity  -  0.25 units Task: create Foo database table - 0.25 units Task: create UI mapper for Foo - 1 units Task: create web page elements for Foo -  1 units Task: create Foo validation service - 0.5 units New Estimate after task analysis - 3 Units You do this with each story that the product owner has assigned to the release.  The product owner simply evaluates how many points they have to work with and assigns the stories until the iteration velocity budget is exhausted. Iteration A - Velocity 8 Units Story A: 1 Unit Story B: 2 Units Story C: 0.5 Units Story D: 5 Units   NOTE:  It is very important to track the target vs actual of each story as I tend to see in my team that they are very accurate with 0-3.5 unit stories anything above that and there accuracy starts to diminish exponentially.  To minimize the this risk we ask the developers to determine if a 4 unit story and above is an epic and can it be broken into smaller stories? Of course we ok this with the product owner before simply breaking out even more stories. Another important aspect is plan to the horizon.  Meaning you can plan for iteration B but understand that it will more than likely change. "Embrace Change"  Sometimes this change is for the good and sometimes not so good the point is reflect after every iteration and determine if the stories in the back log are still what you estimated them to be.  Some may have gotten smaller based on frameworks and patterns you may have introduced during iteration A.  Some may have increased because of an issue you came across in iteration A. What I love most about Agile is you can't hide!  It brings everything to the forefront for you to deal with now, not later but now! I hope I haven't confused anyone with I have outlined.  I am use to it since I deal with a PMO on a regular basis. :-) Posted Aug 08 2007, 11:36 PM by Joe Ocampo with 2 comment(s) Filed under: Agile Project Management, Agile Teams, Agile Project Coaching & Management

• Attempting to Demystify Behavior Driven Development

  After receiving several emails and reading Roy Osherove's post on Behave#, I wanted to give more incite and answer some questions that were asked about Behave# and BDD in general. What is BDD? I am really going to do one hell of a hack job on Behavior-Driven.org but my intentions are to draw you to the issue at hand. Behavior Driven Development’s (BDD) primary intent is to bring software development's focus back to the delivery of business value.  It accomplishes this by bridging the dichotomy between business requirements and working software into one cohesive common vocabulary called the ubiquitous language. Now it is very important to note that BDD is simply the evolution of the existing practice of Test Driven Development (TDD). A majority if not all of the principles of TDD are still applicable. BDD merely focuses on the vocabulary on which these principles are conveyed that is meaningful both to the business advocates as well as the development and testing teams. Experienced TDD practitioners, particularly those that have been involved in helping other developers learn the practice are accustomed to the following epiphanies: The developer starts writing unit tests around their code using a test framework like JUnit or NUnit. As the body of tests increases the developer begins to enjoy a strongly increased sense of confidence in their work. At some point the developer has the insight (or are shown) that writing the tests before writing the code, helps them focus on only writing the code that they need. The developer also notices that when they return to some code that they haven’t seen for a while, the tests serve to document how the code works. A point of revelation occurs when the developer realizes that writing tests in this way helps them to “discover” the API to their code. TDD has now become a design process. Expertise in TDD begins to dawn at the point where the developer realizes that TDD is not about testing, it is about defining behavior. Behavior is about the interactions between components of the system and so the use of mocking is a fundamental to advanced TDD. We have observed this progression in many developers, but unfortunately while most, with a little help, find their way to step 4, many miss the big wins found at steps 5, 6 and 7. OK so why are steps 5, 6,