I have started many agile engagements and I always get asked where is your project plan? Can you tell me the project milestones and target dates? Now, mind you this is before I have anything resembling a backlog or attended any kind of discovery workshops with the customer and core team members. I try to explain that while we have a Statement Of Work (SOW), it is high level and purposely vague and it will take some time before we understand the requirements and have a backlog. This means whatever plan I have produced is subject to change, in fact it will change.
After all, I am not a Soothsayer, and what I understand early on during a project is but a very high-level understanding of what will be delivered. Agile frameworks like Scrum lend themselves to adapting to change, but it never seems to satisfy the need for certainty that sponsors and other key stakeholders seek. So the team will press on gaining a greater understanding of what needs to be accomplished and a solution evolves, but even after gaining insight things change. Even simple things like setting up cloud accounts with the services and access the team needs can take weeks instead of days in some organizations, throwing your plans out the window. A user story that is targeted to take a week, drags on for several sprints as you are dependent on other teams who have their own challenges to deal with. An initial design using a particular technology after further consideration becomes unfeasible, and then it is back to the drawing board.
Agile projects tend to evolve as deliverables change, and the cold fact is your project plan is constantly changing. This is why agile frameworks like Scrum were created because the waterfall methodologies that were steeped in planning generally failed to deliver to the plan. If you can’t coach your sponsor and other senior executives that change is inevitable, then you will likely be subject to a mountain of criticism. This is where the ScrumMaster and Product Owner must seek to provide reality checks and be persuasive to stave off the blind adherence to a plan.
So that is why the authors of the Agile Manifesto stated: “We value responding to change more than following a plan”.
The third statement of what we value from the Agile Manifesto states:
Customer collaboration over contract negotiation
This is an interesting statement for me as I am a consultant and the basis for my engagements with customers starts with a contract, in my case a Statement of Work (SOW). While an SOW is the starting point it is often left fairly vague in some ways to allow for adaptation during the engagement. We do this purposely so that we can work with the customer to uncover the requirements that fit into the broader framework of deliverables.
It is clear to me that the authors of the Agile Manifesto felt that clearly collaborating with your customer was superior to using a contract as some kind of checklist. The project may become highly adversarial in the event that we spend our time fighting over items in the contract instead of making sure we are collaborating with our customers. There is nothing less agile than simply following a contract instead of being focused on delivering the right business outcomes.
I have been as guilty as the next person is thinking that the customer understood the contract in detail, and when we delivered to the letter of the contract we actually had issues. This became my fault by not collaborating with certain stakeholders to the degree I should have. In fact, I know a fair amount of customers who never even read the SOW, so thinking you can use that later to negotiate delivery is somewhat meaningless. On the other hand, I have worked with customers where they analyze every sentence, clearly using the contract as the source of truth for all deliverables. This is both painful for the project team and really not in the customer’s best interest.
So what’s the best thing to do during an engagement if you are to adhere to the agile tenant where collaboration is preferred over contract negotiation? My advice is to do the following:
The contract is a starting point for understanding high-level deliverables, maybe these deliverables become Epics in your backlog.
Let collaboration guide your team with the focus on creating business outcomes, instead of blindly producing what the contract outlines.
Instead of spending your time talking about what is in the contract, it is better to collaborate with the customer to show them that you will over-deliver and have their best interests at heart.
While this seems like such a simple statement that we value collaboration over contract negotiation, it speaks loudly to the real intent. In almost every engagement I’ve been a part of, requirements change as we learn more, and an agile team knows this and will use it to their advantage to produce something better than originally scoped out. Letting a contract dictate the total scope of the engagement and ultimately what is delivered in the end is not agile, but instead waterfall.
I would love to read your feedback on this subject.
Just for the record documentation has its place, especially early on during say version 1 of an application, but it’s no substitute for software that actually works and thus fulfills the requirements. The reason I say documentation has its place in the early stages of an application, is that is when it is most accurate, and over time it becomes less and less accurate as developers make changes and the documentation remains the same. I can hear the screaming now; the developers are supposed to update the documentation as they make changes aren’t they? Sure, in an ideal world, but you and I know this just isn’t true.
The authors of the Agile Manifesto were just trying to emphasize that the most important thing when developing software is that it actually works. I have led many projects that required extensive documentation, only to find out later that no one was reading it or could even find it. In fact most of the time it was the high level architectural diagrams that had the most value and could stand the test of time, at least time that is measured in months. Being agile in software development means being able to frequently release new functionality; with the idea being that delays in releasing working software only postpone value being delivered. If you wait for extensive documentation to be developed before releasing software you will always be late to the party.
I often see the conflict come into play when an organization has three separate entities involved in the creation and release of software. These organizations are Q/A Testing, Developers, and Business Analysts, with the most conflict occurring between the Testing organization and the Developers. The more deep rooted problem is a lack of agility and lack of automated testing tools. You probably encountered some of this in the form of “You need to finish all the functionality before promoting it to the Q/A or Staging environment, so I don’t have to retest everything”. Let’s be real with each other, a lot of organizations think they are doing agile development, when all they are doing is wrapping Scrum around their waterfall processes.
This is all about what is most valuable, and the point of software development is to create working software that meets the needs of the organization.
The first statement in the Agile Manifesto is “Individuals and interactions over processes and tools“.
Why is it more important to value individuals and interactions over processes and tools?
You can have the most wonderful process in the world, but if you don’t have the talent on your team; you are in big trouble. A rigorous process won’t save you if you don’t focus on individuals making high quality contributions.
Take interactions in the area of collaboration. Just because you have a daily scrum doesn’t mean you have high quality collaboration occurring. What if you have all the processes around communication in place, but the individuals on your team are constantly arguing with each other or worse undermining each other.
Agile processes require individuals to do the right thing and adapt to changing requirements and situations, versus just blindly follow some elaborate process. If you don’t invest in the individuals on your team, you may end up with a lot of disengaged team members and your burndown begins to flat line.
What I have seen in the past few years is an increasing focus on agile frameworks that have many processes, artifacts for everything, and one tool after another added to the stack. Let’s be honest a lot of the agile frameworks I see being used reflect a prescriptive a process as if they were lifted right from the Project Management Body Of Knowledge (PMBOK). In many organizations the amount of latitude given to the team is almost zero, and there is no such thing as a self managed team. I see adherence to process dominating what some people call agile frameworks; the hell with the results, instead it’s all about how many of the best practices you check off during a project review. Surely if you are following the best practices and using the most comprehensive tools you must be doing it right, well that’s the thinking behind it. Forget about the people, just follow the process and all will be well.
Listen, I’m not against using a good framework like Scrum or Kanban, or tools like Jira or Azure DevOps. In fact it is essential to have a framework that has some proven processes or ceremonies to utilize, but when you keep adding on additional processes and tools you begin to constrain the choices your team has at its disposal. You trade a self managed team , for adherence to a rigorous process. Ultimately you chip away at innovation, choice, and instead of focusing on the core work, you are focused on supporting some corporate metrics. Last but not least you begin to make all work equal as the administrative burden on the team increases and job satisfaction plummets.
The next installment of the Agile Manifesto will focus on “Working software over comprehensive documentation”. I would love to hear your comments, either supportive or not.
Anyone who has been involved with agile software development or project management would have likely at one time read the Agile Manifesto. I would like to dissect these somewhat simple yet powerful precepts and discuss whether we have lost sight of these principals, since it was written way back in February 2001, more than 19 years ago. In this post I will focus more on the historical perspective of the Agile Manifesto; where in future posts we will delve into each of the principals and our adherence or lack of adherence today. The other question we will explore is the Agile Manifesto still relevant?
The agile manifesto states:
“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.”
The authors of the Agile Manifesto include
Kent Beck
Mike Beedle
Arie van Bennekum
Alistair Cockburn
Ward Cunningham
Martin Fowler
James Grenning
Jim Highsmith
Andrew Hunt
Ron Jeffries
Jon Kern
Brian Marick
Robert C. Martin
Steve Mellor
Ken Schwaber
Jeff Sutherland
Dave Thomas
This group of collaborators was actually the who’s who of software development at the time representing various frameworks including Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming, and several others. When you think about it, this was somewhat amazing that they could agree on a set of principles that would become what we consider agile today. In addition to the Agile Manifesto the group created 12 Principles for Agile Software development:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity–the art of maximizing the amount of work not done–is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
I hope this was a good refresher on the Agile Manifesto. In the past I would start with this when teaching a class on Scrum to set the stage for what is possible. My next post will go into detail on the principles advocated by the Agile Manifesto.
I have been in the IT consulting business for the past 5 years or so and more recently working for one of the largest Cloud providers as an Engagement Manager. Prior to that I was involved in managing projects and programs within the corporate environment, where most of my stakeholders were fellow employees. When I started with my current employer two years ago they had a pretty much do whatever you want approach to managing engagements. In the past year the consulting organization has been going through an agile transformation to Scrum with some customization’s that make it unique to our company. Having used Scrum in both a consulting and non consulting capacity I wanted to outline some of the challenges you face using an agile approach to delivery and specifically Scrum when you are a consultant.
Challenge #1 Startup
Those of who have a consulting background can relate to this. The statement of work (SOW) is signed and the customer says can we start next week on Monday? What about agile tools? What about time to on board the team? What about an internal kickoff meeting? Anyways this is typically a pretty chaotic situation. As a consultant you either bring your own agile tools or you use the customer’s, in either case you need to setup Jira, Rally, VSTS, or whatever tool of your choice. If the customer wants you to use their agile tool and you have not used it in the past then you have a learning curve to deal with. In almost all cases there is no backlog so some form of planning meeting needs to occur to define the goals of the engagement and get an initial backlog setup. You will also need to identify your stakeholders and figure out who the core project team members are. The real challenge here is that you will need to do all of this in the first week and then the expectation is that Sprint 1 starts the next week. You are likely to have issues as you on board the team because they need access to email, calendars, wireless network, and various other systems or databases. From an agile perspective the backlog you walk into Sprint 1 with will most likely be lacking in many ways. Your user stories may at best a vague description of what needs to be accomplished without acceptance criteria, estimates, owners, and tasks. So what can you do about this compressed timeline for initiating your engagement:
Try to buy some time by getting some of the planning done before you go onsite. Start your stakeholder identification and determine who will constitute your core team. Make sure you understand who your Project Sponsor is and figure out who would be the Product Owner.
If you have done a similar project try to re-use some of that backlog on the new project.
Find out who is responsible for on boarding your team at the customer site and make those requests.
Determine ahead of time what agile tools you will be using and determine if you need to bring your own or preferably use what the customer already has.
Challenge #2 Relationships
I’ll use the term stakeholders to mean anyone affected by the project. The funny thing about consulting at least where I work is, you often don’t know your own team members prior to the start of an engagement and most of the time you don’t know any of the customers. So you need to figure out very quickly who will be working on various work streams and start building a relationship with your team members from your company and team members from the customer, not to mention the product owner and project sponsor. I’m not saying this never exists in a non consulting situation where you are an employee and doing the project for your organization, but if you have been there very long you are a bit more likely to know at least some of the project stakeholders. The real challenge here is almost everyone on the engagement is working together for the first time, so you better understand how to quickly move through the stages of forming a team and be pretty good at building trust with your team and the customer.
Challenge #3 Maturity
I’m referring to the customer’s maturity with agile and Scrum. As a consultant you are likely to experience the gambit when it comes to a customer’s experience using frameworks like Scrum. The same lack of experience not only exists with the customer, but also with your own team. This isn’t so much unique to consulting, but you will need to assess the customer’s capability early on in the process, so you can determine how much coaching will need to take place, and to what degree they are capable of fully embracing all of the ceremonies. An example of this might be the retrospective. If they have never conducted retrospectives they may not understand the purpose and approach the meeting with in a guarded fashion versus one of transparency. Another issue I run into all the time are customers who have not performed story point estimating and want to equate these estimates to hours or days. In either case you will need to do a lot of teaching and coaching to get these teams up to speed quickly so you can take advantage of what these ceremonies have to offer. Most of the consulting engagements that I am involved in range anywhere from 3 to 6 months and require you to take your team from forming, storming, norming, and performing very rapidly as customer expectations are that you will be producing at maximum efficiency in a short period of time.
Challenge #4 Level of Engagement
I have encountered this a number of times and it can be difficult to overcome depending on who is your Sponsor or Product Owner in the event you have one. Some organizations think well I’ve hired these consultants so they often don’t understand why they need to be so involved. I mean I’ve outsourced this project haven’t I? They say things like I want you to tell me what to do, be prescriptive, that’s what I’m paying you for. Let’s be real, projects don’t go well without stakeholder involvement and ultimately if the customer shirks this responsibility they will come back to you later on with various complaints about how you missed requirement A or B. You will need to understand how to escalate to the right people or this thing will come off the rails. I’ve worked in organizations where the management team would not come to the Sprint Review and we spent the time doing demos for the core project team. I’m not saying none of this happens with internal projects, but it can often be an issue during a consulting engagement. You will need to identify your stakeholders from the start and set expectations with them, and if you don’t get the level of participation you need then escalate, escalate, and escalate some more!
Challenge #5 Money
One of your good friends in a consulting sales role crafts an SOW with either a lot of vague statements or a list of deliverables a mile long and a budget that supports about 1/2 of what is promised. In most corporate settings you are not held to some ridiculous budget that was negotiated by upper management and procurement. At most you will go through an internal change control process where more internal funds will be allocated, because after all they don’t consider it real money. This is bullshit of course but for some reasons salaries don’t seem to count in most corporations. On the other hand the cost of a consulting engagement will be scrutinized and negotiated by a number of people. Of course consulting companies put language in the contracts that say something like this: “We will assist the customer to deliver X or Y based on the number of hours or days in the SOW”. The problem is that doesn’t change your customers expectations and won’t help you when they call your boss and say you haven’t delivered what they expect. From a budget perspective you will nearly always be having too little budget to accomplish all the scope. Scrum will help you if you have a well estimated and prioritized backlog and begin talking about minimal viable product (MVP) from the beginning of the engagement, but no matter how skilled you are at this you will always be have a supply and demand equation that is not equal. Coaxing another $500,000 out of a customer who only budgeted $400,000 to begin with will be nearly impossible. I’m not talking about some lack of project management budgeting and tracking here. The reality is you will be asked to execute engagements that are flat out underestimated and not adequately funded. Welcome to consulting!
Challenge #6 ScrumMaster vs. Project Manager
In a corporate setting that has adopted Scrum your role as a ScrumMaster can at least lean towards actually being a ScrumMaster, on the other hand when you are a consultant you are often pulled between the ScrumMaster role and that of a Project Manager. Here are a few situations that make you feel the push and pull between the ScrumMaster role and that of a more traditional Project Manager:
The customer early on in the engagement wants a schedule showing when the deliverables are to be completed and when the project is complete. Without even a decent backlog you are scrambling to come up with some kind of high level timeline and you go from ScrumMaster to Project Manager in a heart beat.
Instead of relying on the daily scrums, sprint reviews, and retrospectives the customer wants a weekly status report where you have to re-iterate everything that was shown in the sprint review or discussed at the daily scrum. Welcome back to 1990 and you end up spending 3 or 4 hours cranking out another status report week after week.
As funding is finite on a consulting engagement you will be encouraged to drive your team to achieve the goals of the project with a do whatever it takes type of guidance. Some of the agile values like team autonomy, fostering experimentation, and taking risks can go right out the window under the pressure to just drive the team and get it done. You go from being a team builder and facilitator to a hard ass dictator (Project Manager).
In my experience customers want you to say you are using agile methods and frameworks like Scrum but in reality their actions send a much different message. Face it you will end up playing a hybrid role somewhere between a ScrumMaster and Project Manager. It’s up to you to justify why you need the right people to attend the Sprint Review, having a backlog that drives higher level estimates, and why your team needs to be empowered to make their own decisions. Often only your ability to communicate effectively and establish relationships stand between your engagement being agile or waterfall.
Summary
None of this probably comes to a surprise to any of you who have been consulting for more than a couple of years. Some of these issues can come up with a waterfall project and can have equally devastating results. However with that said using an agile framework like Scrum requires a level of understanding and participation that is unparalleled in many ways. To a large degree your success will be based on your expertise and ability to adapt to one new situation after another.
I would like to know what challenges you have faced using Scrum or another agile framework on a consulting assignment.
AgileMethods.blog 