“SAFe provided the agility, visibility, and transparency needed to ensure we could integrate with numerous other efforts, get predictable in our delivery, and ensure timelines are met.”
–David McMunn, Director of Fannie Mae’s Agile COE
Challenge:
Within three years, the organization would need to stand-up an entirely new business model that would change the way securities are issued to the market—and do so within aggressive timelines.
Industry:
Financial Services, Government
Results:
- Releases now happen every month, instead of once or twice a year.
- They integrate reliably every two weeks.
- Fannie Mae reduced delivery risks.
- The organization reduced the defect rate substantially.
- Teams now deliver more than 30 attributes per sprint compared to 2-5 before.
- Velocity increased from 10 story points to more than 30.
Best Practices:
- Sync cadence – Establishing a common cadence was critical to success. Engineering practices must evolve in order to comply with bimodal governance.
- Work on database modeling upfront – For any data-heavy effort, perform advance work on database modeling to avoid the impact of changes identified later in the sprint.
- Develop a playbook – Such guidance reduces rework for multiple teams working in parallel.
Introduction
Fannie Mae is the leading provider of mortgage financing in the United States. Operating under a congressional charter, Fannie Mae—and its sibling organization Freddie Mac—play an important role in the nation’s housing finance system; they provide liquidity, stability, and affordability to the mortgage market.
Coming out of the housing crisis in 2013, Fannie Mae recognized that the lending environment it was moving into required it to be even more responsive to meet rapidly changing customer needs. Further, Fannie Mae recognized that agility was critical to achieving this objective—not just in technology, but across the organization.
In January 2015, Fannie Mae was preparing to align with guidance provided by the Federal Housing Finance Agency (FHFA) and Congress, under a new joint venture named Common Securitization Solutions (CSS). As part of this effort, Fannie Mae undertook an initiative to transform some of their key internal business processes to align with CSS to build a universal securitization platform for the issuance and management of mortgage-backed securities.
Within three years, Fannie Mae planned to develop an entirely new business model that would change the way securities are issued to the market—and do so within aggressive timelines. More than 20 development teams, encompassing over 300 individuals, were needed to integrate development and testing efforts across 30 assets. As Fannie Mae prepared to implement this change, the organization encountered several challenges as the new model was being defined based on continuously evolving requirements.
“When you’re doing a large-scale integration with a lot of data, the number-one factor for success is early integration and early testing,” says Atif Salam, Director of Enterprise Data at Fannie Mae. “The federal mandate required us to mitigate risk from the get-go, and we realized early on it would not be possible following a waterfall approach. There was no better way for us to mitigate that risk than to adopt Agile.”
Overcoming Initial Roadblocks
Enterprise Data’s efforts to adopt Agile uncovered several challenges, both internal and external:
Challenge #1: No Agile capability evident for the initial two teams at the outset of the Enterprise Data initiative.
The first Enterprise Data teams were brand new to Agile, the Scrum methodology and, having been formed specifically for this initiative, working with each other.
Prior to adopting SAFe, Enterprise Data developed a standard on-boarding approach and entrance criteria for standing up new teams. Additionally, external Agile subject matter expertise was brought in to train and work with the teams, and an Agile Mature Model (AMM) was created to baseline behaviors and practices, as well as identify areas for optimization.
Thereafter, once the decision had been made to adopt SAFe, the program began to work through the SAFe Readiness Checklist. The AMM was used to set target benchmarks that all program teams were required to meet in order to ensure there was sufficient capability in place from which to scale.
Challenge #2: At the outset of the Enterprise Data initiative, a Scrum team could only complete a single user story due to inflexible architecture, end-to-end testing challenges, and numerous build constraints. Further, it was typical for the work to be gated by subject matter expertise between developers who viewed data attributes as a data point, comprised of both sourcing and vending complexities, that could only be implemented sequentially.
In response, technical leads focused on eliminating constraints, reducing complexity, and optimizing workflow. Specifically, Technical Leads worked with the teams to leverage cross-functional team/paired programming constructs to augment technical expertise. As a result, the teams began to view data attributes not as a data point, comprised of both sourcing and vending complexities, but rather as having two distinct pieces of business value, specifically sourcing and vending.
Additionally, they made the effort to move system integration testing (SIT), as well as user acceptance testing (UAT), left into the Scrum team. As a result, and over time, each team began to complete multiple user stories within a given sprint. Additionally, the organization adopted an emergent design mindset, formed cross-functional Agile feature teams, and aligned to a common cadence that synchronized their activities (e.g. sprint planning, Scrum-of-Scrums, sprint reviews).
Challenge #3: At the outset of Enterprise Data’s journey, complexity was further complicated by the fact that teams were required to develop and integrate their code in the same mainline, thereby replacing branching as an accepted technical practice. Additionally, Fannie Mae required new release traceability management that would satisfy corporate and federal governance requirements.
To address these challenges, technical leads and shared services focused on building a continuous integration capability, across all teams, using the same codebase. The organization had always had application lifecycle management (ALM), however, it needed to rethink continuous integration to realize true efficiencies. Over the course of 10 months, the organization focused on leveraging automation to reduce the time to implement builds from once every six months to multiple times a day.
Additionally, Enterprise Data adopted behavior-driven development engineering practices for traceability, automated testing, and prototyping.
Challenge #4: Upstream technical dependencies specific to architecture, database design/modeling, and test data provisioning prevented the teams from completing a single user story within the two-week sprint cadence.
In addition to the technical challenges the teams were facing, there were also multiple upstream dependencies on architecture, data modeling, and test data management that they had to resolve before a User Story could be implemented by a team working in a two-week cadence.
Initially, working ahead of the teams, a group of business analysts were assembled and assigned to groom the program backlog sufficiently so that User Stories met, or exceeded, 80% of the sprint team’s Definition of Ready. Despite this focus, however, there was barely enough ready work in the program backlog for the teams to bring into their respective sprint planning. This was due to the lead times required to resolve upstream dependencies as well as the need to respond to continually changing requirements.
In preparation for scaling, Enterprise Data worked with their business stakeholders to create a roadmap of features spanning one business quarter. Simultaneously, they focused on optimizing backlog health, sufficient in depth to support the Agile teams, for at least two consecutive sprints. Additionally, adopting a system perspective, the entire value stream was analyzed to better anticipate, and mitigate for, internal/external technical dependencies.
Challenge #5: The organization’s culture was accustomed to working within a traditional implementation methodology.
At the outset, Fannie Mae had a traditional command and control culture, supported by a broader ecosystem of corporate functions that had to change to support Agile. Those leading the change made a significant effort to work with leadership and management to pivot from the traditional role of directing delivery to becoming Lean-Agile leaders and critical change agents, both supporting the teams as well as modeling the values and principles of the Agile Manifesto.
As already noted, leadership and management changed their focus to clearing impediments impacting the teams. Additionally, they influenced corporate functions to align in support of Agile, get the business integrated and involved, as well as to put the pieces in place to create an environment focused on continuous learning. “Historically we would have seen challenges as failures in requirements or development rather than opportunities to fail fast and learn, and improve,” Salam explains.
While still new to their roles, the Lean-Agile leaders infused a sense of purpose in the teams and gave them autonomy to implement the work while decentralizing decision-making and minimizing constraints.
SAFe: Agility. Visibility. Transparency.
Although Fannie Mae had pockets of Agile capability up to this point, leadership understood that a scaled Agile methodology was required to achieve their objectives. Fortunately, individuals within the company had prior success with large-scale Agile deployments using the Scaled Agile Framework® (SAFe®).
Fannie Mae teamed up with an external Scaled Agile Gold partner to develop and mature its Scrum capability and then deploy SAFe. As the first to make the SAFe transition, the Enterprise Data division became the torch bearer.
“We had multiple waterfall efforts, third-party integration, and a hard regulatory mandate that made coordination and execution exceptionally difficult,” explains David McMunn, the Director of Fannie Mae’s Agile Center of Excellence (COE). “SAFe provided the agility, visibility, and transparency needed to ensure we could integrate with the numerous other efforts, get predictable in our delivery, and ensure timelines are met.”
Fannie Mae applied a dogmatic approach to ensure the organization was developing a consistent set of practices across multiple teams at the outset. External coaches delivered Agile, Scrum Master, Product Owner, Leading SAFe (SA), and SAFe for Teams (SP) training. The SAFe training was then mandatory for every new team joining the effort.
Fannie Mae launched its first Agile Release Train (ART) encompassing six programs, across 12 teams, with more than 130 people, in June of 2015. Admittedly, that first Program Increment (PI) offered some learning experiences.
“In spite of all the preparation that went into the backlog, setting expectations, confirming attendance from stakeholders, and the training prior to planning, the first PI was somewhat of a chaotic experience,” says Scott Richardson, Chief Data Officer at Fannie Mae.
Context setting provided by the business, product, and architecture leads took time away from team break-out sessions and, as a result, the teams struggled to resolve all of the open requirements and scope questions to complete their plans.
“But by the end of the second day,” Richardson continues, “we started to see progress.” The teams had mapped out their dependencies on the program board, resolved, owned, accepted, or mitigated (ROAM) all of the known risks in the PI and achieved a Fist of Five confidence score of 3.
“The session offered the very first opportunity for all stakeholders to work together on this multi-million dollar program.” Richardson adds. “A new way of managing large-scale integration efforts at Fannie Mae was emerging that would spread across the technology enterprise.”
Over the next few PIs, the organization knew more clearly how to prepare for the PI planning meeting and confidence scores began averaging 4 and higher.
Modeling Confidence in the New Methodology
During cross-team planning in an early PI, it became clear that several teams were not on track to deliver important capabilities within the targeted timeline. “Some of my best new Agile team leaders offered to throw more people at the problem ‘just this once,’ and crash the schedule like they did in the old days,” Richardson says. “It’s in those moments that you need to model confidence in the Agile method, to be the calm in the eye of the storm.”
Instead, the Agile team leaders were encouraged to go back to the Product Owners regarding the change in priorities and empower them to devise a new minimum viable product. “Within a couple of hours, everything was back on track with planning, and ultimately all the teams delivered, and the external customer delivery was on-time,” Richardson says. “Now they carry this story with them, and are empowered to solve problems and make decisions in truly productive ways. It’s part of the culture.”
Gains across the Board
Today, Fannie Mae has come a long way. The Enterprise Data division delivered an integrated solution on time and with much higher quality than was expected for an effort of this size. From a broader perspective, the transformation to SAFe revolutionized how the organization plans for the delivery of large-scale programs.
Fannie Mae has seen improvements on multiple fronts:
- Reduced risk – Fannie Mae reduced delivery risks through the relentless focus on innovation and automation to ship “production ready” code with higher and higher frequency. They significantly mitigated the risk inherent in complex integration between legacy and new architectures/applications, as well as between internal and external systems.
- Faster feedback cycles – Enterprise Data delivers system demos and integrated code every two weeks. Releases now happen every month, instead of once or twice a year, for the largest application across the enterprise, with millions of lines of code.
- Improved predictability – Teams, within the program and across the enterprise, integrate reliably every two weeks.
- Boosted quality – The organization reduced the defect rate substantially.
- Increased business value – Teams now deliver more than 30 attributes per sprint compared to 2-5 attributes when Agile was first adopted within Enterprise Data.
- Better team progress – Teams undergo regular AHR (Agility Health Reviews) cycles and have matured to higher Agile Maturity Model levels.
- Greater efficiency – Fannie Mae realizes significant efficiency through a reduction in technical debt.
After the initial deployment, the division rolled out SAFe to the rest of the organization, training up to 600 people on Leading SAFe, SAFe Advanced Scrum Master, SAFe Scrum Master, SAFe Product Manager/Product Owner, and SAFe for Teams, depending on roles. Several employees went on to achieve their SPC certification.
Currently, Fannie Mae runs three ARTs. The Enterprise Data ART recently completed its 13th PI. Additionally, there are more than 200 Lean-Agile teams across Enterprise IT, encompassing over 3,000 people. Functional and business portfolios are adopting lightweight Lean-Agile values and practices as part of their day-to-day activities.
“This way of working has spread across the organization,” Salam says. “It’s changing the way we deliver for the customer, the way we hire and do our budgeting, and is continuously extending further and further into the business.”
Share:
Back to: All Case Studies
Suggested Case Study:
Dutch Tax and Customs Administration
