Challenges with agile methodology

Having used Agile methodology in various projects, I wanted to share my thoughts on the opportunities and challenges presented by this methodology.

What is Agile?

As per www.dictionary.com, word agile means ‘Quick and well-coordinated in movement’.

Applying this definition to IT development process, it would mean that the process should be –

• Quick to deliver results
• Efficiently coordinated to deliver best results

Let me further investigate ‘quick’ and ‘efficient’. What does ‘Quick’ means for IT development?  It is purely defined by the needs and nature of the business goals and aspirations. It can be 1 year for a research project Or 1 week for a vanilla LMS implementation.

The success of any IT development process is dependent on the seamless coordination among all stakeholders. So, a process being ‘efficient’ goes by default. The challenge is to measure the process success as being ‘efficient’ or ‘inefficient’ and the ability to introduce changes in the ongoing process to keep improving it to the Nth degree.

Agile methodology for software development attempts to achieve the ‘Quick’ and ‘Efficient’ traits by mandating the following key principles:

1. Break the project into number of small deliverables
2. Regular deliveries at predefined short time intervals
3. Regular feedback from key stakeholders on the deliveries
4. Keep adding to the list of deliverables as they are discovered during the development process
5. Re-plan as and when changes arrive or requested for
6. Ensure the availability of all important decision making stakeholders to make a confident ‘real time’ decision
7. Cut the communication time by conducting face to face conversations
8. Less focus on documentation/recording and more focus on ultimate deliverable
9. Always keep making progress despite ‘roadblocks’ by shifting to alternate work for the time when ‘roadblocks’ are dealt by ‘specialists’

To summarize, process should be flexible to adapt changes, iterative with N number of development cycles and minimize the communication related delays.

Benefits of Agile

These key principles immediately throw some excellent benefits over traditional approaches, such as:

• All stakeholders are apprised of project’s status at all times
• Helps visualize the final deliverable in early stages of development
• Gives a quick estimation of ‘How far are we from final product’
• Keeps the team on its toes and hence, competitive
• Reduces the ‘idle’ time to minimum and thus, increases resource utilization

Challenges of Agile Methodology

I do not doubt the realization of any of the ‘Agile’ benefits. However, after having worked in a complex and big development & implementation project using agile methodology, I have some concerns on ‘Where Agile can be best utilized’.

Here are some of the practical challenges which I feel constraints the use of Agile methodology.

1.    Quality, Cost, Time and Scope

As per PMI (Project Management Institute), changes to any of these constraints bring a change in one or more remaining ones.

Agile being flexible allows frequent changes in scope which means that Cost, Quality or Time has to change. In practice, majority of projects have fixed budget and a mandatory deadline (say, LMS deployment with custom data-flow and reporting mechanism for regulatory compliance).

Hence, even though a project follows agile methodology, at ‘certain point’ the scope changes have to cease.

The realization of this ‘certain time’ is the most difficult task. Do it any sooner than required and business looses the benefits of being agile and on the contrary, do it any later than required, and projects runs over budget or time.

2.    ‘Ready to use’ product

One of major benefits for agile is that ‘ready to use’ product with limited features is always available throughout the development cycle. Hence, if the project has to cut short due to any external constraints, business have a fall back product.

Again, from my personal experience, I do not believe that this benefit can be realized in practise for most of the development projects.

Majority of projects need performance testing and tuning. Basic principle of performance testing warrants a stable functional application to be tested on. An application under agile development process is always evolving and may have functional defects. Hence, performance testing can only be performed after substantial number of deliveries.

Hence, the deliverable is never in ‘ready to use’ form during the development stage unless it is performance tested. Like performance testing, there are many other work items which may continue well after development stage, such as:

• Data Migration
• Setting users in live system
• Security/Anti Hacking testing

And basically, anything to ‘implement’ the deliverables in Live Environment.

3.    Inability to ‘design’ for future requirements

Irrespective of best design models and most experienced design personnel on a project team, it is very hard to design a system on the basis of unseen requirements. This often leads to ‘rework’ at various stages in development and testing.

There was a major mandatory design change due to elaborated requirements during one of the development cycles of the project I worked with. The impact of this change was so big that 2 development cycles had to be used to fix the system as per new design.

4.    External and Internal Dependencies

Majority of projects have external dependencies which are out of control of core project team. External teams will not always follow the agile methodology and hence, will deliver at a certain point in time. Any dependent work can only be undertaken once external deliveries are made.

The problem arises when these dependencies are only discovered during the development process.

There can also be internal dependencies between the different agile teams working on same agile project. In one project we had 3 agile development teams. The selection of work items in an iteration was heavily dependent on the deliverables from other teams. This made planning each iteration very tough, and it needed huge coordination effort.

5.    Need of experts in agile team

Since, each iteration is for a small duration; it is must that each member of team must be an expert in their respective field. This is rarely a case in practise. Hence, often there is a danger to overrun the estimated time for work items.

Conclusion

• Though, agile offers several benefits to Business but at the same time it remains a very challenging task to be able to deliver an agile project successfully.
• The nature of project must be considered before the selection of methodology. In my experience it works well for small to medium projects, or implementations that require less product customizations.
• Some of the challenges I have mentioned can be addressed by better planning and experience.