Bank of America is one of the world’s largest financial institutions, with approximately 200,000 employees, serving individual consumers, small and middle market businesses and large corporations with a full range of banking, investing, asset management and other financial products and services.
While the bank’s retail division provides banking facilities to 59 million US consumers, the company’s Global Corporate and Investment Banking (GCIB) group offers services to businesses, institutional investors, financial institutions and government agencies. GCIB provides services in M&A, equity and debt capital raising, lending, trading, risk management, treasury management and research.
Given the breadth of those activities – and the current climate in the financial services sector – its software development teams are under intense pressure to deliver high value to the business. Agile methods – which promote the delivery of iterations of working parts of the application under development to users in short ‘sprints’; open collaboration between developers and the business stakeholders; and process adaptability throughout the life cycle of the project – have been instrumental in providing software that is on time, on budget and, most importantly, what the business needs.
In an in-depth interview, Sean Cody, process architect at the bank’s Global Markets Architecture team, offers insight into how Agile reduces risk, the shift away from classic waterfall approaches, the return on investment that Agile has delivered and the challenges of applying Agile in distributed/offshore teams and large projects.
Information Age (IA): What is the compelling argument for adopting Agile approaches?
Sean Cody (SC): The big sales pitch for Agile techniques is the delivery of working software, on time, to customer specification, with as few defects as is possible, in as dynamic a business environment as is necessary. This is obviously a large claim to make, and is contingent on the effective execution
of a range of technical and project management practices that have varying degrees of difficulty – it’s only when the team has a degree of competence in these practices that Agile’s benefits are truly realised.
For this reason, I feel that the capacity for risk mitigation present in Agile practices is the more compelling argument than the grandiose claims made above, if for no other reason than the simple claim that risk management serves to moderate some of the hype that surrounds Agile.
Regardless of the level of Agile practices that the team has proficiency in – even if the team is only running per-iteration retrospectives – there is a level of risk mitigation associated with every practice in the Agile arsenal. It’s true that only employing one or two Agile practices instead of the full suite opens the team up to other risks, but, operating on the assumption that the team takes a disciplined approach to Agile adoption and that a holistic employment of the practices is the desired end state,
then it can certainly be argued that comprehensive risk management is the single most compelling argument for Agile – especially as it underlies the claims put forward above.
This has been one of the main reasons for adoption of Agile within Bank of America (BofA) – large-scale projects that aim to implement critical business functionality in a transparent manner and smaller projects that aim to incrementally deliver to business partners have both found that the transparency, control and flexibility of Agile (its inherent risk management features) have been of critical importance to both successful delivery and early highlighting of problems.
IA: Does its adoption require a different set of skills from developers – both tech skills and business relationship skills?
SC: Regardless of the specific flavour of Agile being employed – Scrum, XP, hybrids, etc – there are five core practices that are fundamental to any Agile process: continuous integration; test-driven development (TDD); an iterative and incremental approach to development; story-based development; and multi-function teams that engage all stakeholders in the project.
Having given this list, however, I think it’s important to note that, at core, Agile practices are simply common sense, and indeed many developers will have employed them in one form or another (on an admittedly ad hoc basis) at some point in their careers. This combination of technical and project management practices can be adopted in a structured and incremental manner, and the feedback loop between them, when correctly identified, can be gradually leveraged to ensure fully integrated operation within a reasonably short period of time.
IA: What’s the issue then?
SC: Perhaps the key issue is the discipline that is required to effectively employ these practices such that they become second nature, and thereby deliver their promised benefits. A team can receive training in TDD, for example, but when pressure for delivery really hits, the team must be disciplined enough to not throw out the practices they’ve signed up for as part of Agile, but rather to stick with their approach and, if necessary, convince the other stakeholders in the project that these practices are truly worth it.
Certainly this has been one of the challenges that teams within the bank have experienced, but generally speaking the level of adherence to practices has been very high.
Apart from discipline, the other fundamental skill that the developers must be able to take on is a willingness to embrace change – probably the hardest thing for a team to get in place. Part of Agile’s risk management arsenal means a willingness to accept change when necessary, whether it be as a side effect of adoption of Agile, during the adoption process or in the project itself; and people being people, there are often issues with making sometimes large-scale changes to things that people have already invested time, effort and emotional energy into.
Fortunately, the level of necessary change has been quite manageable pretty much across the board, and even if some teams may perhaps have been reluctant to make the changes necessary, there has almost always been a recognition on the part of the team that, once implemented, the changes have brought benefits, especially if the teams are provided with the opportunity to influence their own outcomes. Encouraging and empowering teams to act as their own agents of change, rather than passively receiving direction, has hence been a critical factor in effective process adoption.
IA: What level of buy-in is required from the business?
SC: It varies. Obviously, a purist interpretation of Agile practices would state that the business must fully commit to the process if Agile is to succeed, but the reality of the business environment we operate in is that we’re not always going to get that complete level of buy-in. In other words, sometimes we get a good level of participation from the business, other times there is more of a “go away and build it” approach.
The former is obviously the desired end state, while the latter represents a significant challenge in ensuring that what the customer wants is actually delivered by the team – if the customer isn’t engaged throughout the process, how does he or she validate the delivered stories at the end of each iteration?
IA: Given that business buy-in is necessary, how can it be reached?
SC: We have often found that the necessary first step is for the development team to become competent in Agile practices before taking them to the customer – it does little to inspire confidence if the team is trying to engage the customer but is changing its approach on a per-iteration basis due to a lack of familiarity with the best practices for their given environment.
Once the team is proficient in Agile the customer can be incrementally engaged, and we’ve generally found that the benefits that they begin to see – rapid access to skeletal functionality, early, incremental delivery and the ability to change priorities based on changing business needs – mean that they often become eager to embrace Agile principles.
IA: What role does software configuration management play?
SC: Software configuration management (SCM) certainly plays a fundamental role in the Agile world [BofA uses one of the market-leading SCM systems, Perforce], but there is also a larger question of tools in general. One of the things we realised very early on was that a standardised set of tools would help teams adopt the process, so we built out a standard set of tools for Agile teams that encompassed SCM, iteration and story management, and collaboration, all of which were linked together via some internally developed integration modules that served to concretise as much as possible those Agile process features that could be embedded in a set of tools.
The end result is an “Agile toolchain” that covers Story creation, elaboration, estimation, scheduling and prioritisation; provides collaboration functions and revision history information for Stories; links source control artefacts and build records to Stories; and provides published information to unregistered users via a wiki.
So, while tools take a lesser role in the Agile manifesto, it’s important to note that they play a very important role for any organisation, let alone a financial organisation the size of Bank of America.
The suite of tools we have chosen are designed to provide an automated paper trail and system of record for virtually all communications and changes related to the Agile project, thereby meeting audit and regulatory requirements by ensuring that there is enough traceability to determine what was done when and why.
The use of tools, however, is a double-edged sword; there is always a risk that teams focus on the tools and how to use them, rather than concentrating on the more important part – getting the process right. In extreme situations, teams have started using the tools and made the assumption that they were ‘Agile’, without realising that tools can provide only a framework on which to build other practices.
In the SCM space, for example, efficient and common sense policies for branch management are almost more important than the tool being used – of course, some tools will be better than others,
but the rational application of proven approaches to code management is vital to successful use of any tool. The necessary precursor to any use of tools, therefore, is ensuring that the teams understand what is required from them as part of the Agile process itself.
IA: Is Agile right for all projects or are there some projects that are better served by other methodologies?
SC: This is a hard question to answer, as there is a large subjective element to it. Perhaps the best answer is to say that we have both a standardised Agile process and a standardised waterfall process within the bank – some teams will be more comfortable with a traditional waterfall approach, while others will prefer the Agile route, and no amount of persuasion will convince them otherwise.
In an objective sense, there is evidence to suggest that Agile approaches aren’t suitable for distributed teams, or large projects, but the experiences we’ve had within the bank suggest that while there are certainly some challenges associated with these types of projects taking the Agile path, the benefits for doing so, [see the earlier response above] make for a compelling argument to try and overcome the challenges.
IA: Can Agile be used when the development is outsourced or even offshored – i.e. when the target users of the application within the business are separated from the development team?
SC: We’ve had some experience with our offshore partners using Agile for their delivery, and as with our onshore teams, the success or otherwise is dependent
on the quality of the team more than anything else, and their ability to master change and employ the practices in a disciplined manner.
The separation of the users from the development team is certainly an issue, but not an insurmountable one – it is, after all, an issue of team dynamics
and communication, and as long as appropriate steps are put in place to ameliorate the downside of geographical distribution, there’s no reason to eschew Agile in favour of waterfall.
Having said that, however, it should be acknowledged that there is more work associated with offshore Agile, especially if an offshore team is delivering part of a larger system. Approaches such as ‘hand-off pairing’ and similar approaches have also been employed to try and improve the interaction with and delivery of the offshore teams, but we’ve also found that setting up offshore teams as standalone entities that are decoupled from the main project and are run as self-driven entities is also an effective strategy – as I mentioned earlier, it really depends on the teams in question, and there is not really any such thing as a single authoritative answer on the subject.
IA: Have you seen the desired benefits – faster delivery, meeting requirements, delivering on budget, avoiding scope creep?
SC: Yes, as long as the necessary people are appropriately hooked into the project.
Agile isn’t a panacea, nor a silver bullet – it’s simply a set of practices that reduce risk, but the benefits to be derived from this risk reduction are still contingent on human decision-making. So yes, we have seen the desired benefits, but only once people take those steps that are necessary given the conditions encountered in the project.
IA: Is there demonstrable ROI?
SC: Most definitely, but the degree is of course dependent on the metrics used.
If you want to refer to the number of defects found in production, or the number of Stories delivered defect-free to quality assurance, the numbers are quite straightforward, and easily translate to money saved. Perhaps the bigger impact, however, is in terms of satisfaction for the business, but this is by far the more difficult benefit to measure objectively. In general terms – and given the past year’s market conditions in the financial sector – the flexibility made possible by the use of Agile as a standard practice within the bank has proven to be a valuable resource that teams have been able to draw on to help their business partners focus on their core businesses.
Furthermore, the sanctioning of Agile by the senior technology leadership in the bank has proven to be a prescient decision that has resulted in significant commercial and technical advantages. Have these advantages translated into real dollars saved? I would have to say yes.
IA: How widely is it applied within the development group at Bank of America. And are there plans to use it right across the group?
SC: As implied by its size, Bank of America has many development groups spread across all lines of business, so a blanket adoption of Agile practices would at best be a huge undertaking.
Rather than take this approach, and in the knowledge that a standardised approach to development was necessary given the scale of the organisation, the regulatory requirements and the commercial realities, the bank’s senior technology leadership decided to make Agile one of the two methodologies available for development teams.
We now have over 200 Agile projects and approximately 2,000 users configured in our standardised suite of tools, and while this isn’t a complete picture of Agile adoption across the bank, it certainly represents the bulk of the community – based on the experiences of the past 12 months, we can see this number continue to grow for the foreseeable future.
IA: Are there other significant challenges of using Agile?
SC: The challenges are varied. In addition to those already mentioned, we have encountered a range of issues in Agile adoption, including: how to achieve non-disruptive interaction and/or more formal integration with enterprise change control procedures; providing consistent delivery mechanisms for supplying code to enterprise-wide testing centres; ensuring that standard methodology on-boarding processes are followed; guaranteeing appropriate artefact delivery and retention for audit purposes; enabling TDD as a standard technology for teams new to the process; and ensuring that teams stay Agile.
Any of these issues in isolation would be a challenge for any organisation – Bank of America has, perhaps due to its size, experienced all of them at one time or another, but I’m happy to say that we’ve managed to develop fairly effective strategies for coping with them, and continue to meet the challenges as they arise. I think the approach of organic growth of process utilisation is a key factor here – providing Agile as one of two options the team has for developing software ensures that those who are committed to an Agile approach can gradually join the process at an agreed pace rather than have it forced on them, as might happen with a blanket adoption approach. The end result is a community of motivated individuals within the bank that work together to support each other across business lines and try to provide solutions to problems that have been experienced before or are new variants on old issues.
In general terms, I think Agile adoption is more an issue of sociology than technology. The technology and practices are proven to work, so it’s not a question of whether the activities, correctly employed, will garner benefits – clearly they will. Rather, the main challenges lie in the following areas: instituting an acceptance of change as a necessary precondition to effective process adoption; ensuring the disciplined application of the methodology’s practices given business pressures, especially given the current market environment; overcoming ingrained practices that have the potential to act as adoption derailers; and providing a collaborative environment that can cut across traditional divisions in an aim to ensure successful project delivery.
Fundamentally, all stakeholders need to be willing to embark on the journey that is Agile adoption, so attempting to impose any universal usage is counter-productive. We believe that the BofA approach of providing Agile as an option that can be taken up by a development team, in a mannernthat meets a standardised set of requirements, has resulted in considerable benefits for technology and the business, and will continue to do so.
Further reading
The Agile revolution – How a radical rethink of development processes has aligned software with the business
The difficulties of Agile development –
The fluid nature of Agile projects can make them difficult to manage, says Dr Peter Merrick
Nexus of innovation? – The financial services industry is struggling under the weight of three decades of technology build-up.
