Mobile Modernisation in the Trenches: Technical Wins, Political Lessons

PISR: Problem, Impact, Solution, Result

Problem: Commonwealth Bank of Australia, an Enterprise organisation in the Retail Banking sector, faced critical challenges across their mobile development infrastructure. Their legacy 8-year-old Appium testing framework, combined with severe Team City infrastructure limitations, created a perfect storm of developer productivity issues. The 9GB monorepo with 150+ developers was running 25,000 unit tests for every single PR (even one-line JSON changes), whilst testing dependencies were handed off to offshore teams in India.
Business Impact: This resulted in build queue times of 5-7 hours (2 hours wait + 2 hours build + additional agent wait time), test execution times of 7 minutes per test case, and a completely broken CI/CD pipeline with no automated feedback. Developers working close to the 4-week release cycles faced even worse queue times, with some builds taking overnight to complete. The estimated cost of our AWS solution would be up to £5M annually, but this was competing against existing £2M+ on-premises infrastructure investments.
Our Solution: Over 9 months, ClearRoute's team tackled two parallel workstreams: mobile test automation modernisation and CI/CD pipeline transformation. We implemented a native testing strategy using Swift UITest and Espresso with comprehensive mocking, and developed a GitHub Actions + AWS EC2 solution that could eliminate queue times entirely by spinning up dedicated Mac EC2 instances for builds.
Tangible Result: The testing transformation achieved a 97% reduction in test execution time (7 minutes to 15 seconds for mocked tests), whilst the pipeline solution demonstrated complete elimination of queue times through on-demand AWS infrastructure. However, both solutions encountered significant organisational resistance due to competing team investments, territorial concerns, and complex stakeholder politics, preventing production deployment despite proven technical superiority.

The Challenge

Business & Client Context

Primary Business Goal: Accelerate Cards Modernisation programme delivery whilst improving overall mobile development velocity for 150+ developers working on one of Australia's largest banking applications.
Pressures: Cards Modernisation programme serving as "guinea pig" for broader transformation, developer frustration with 5-7 hour build queues getting worse near release cycles, and pressure to reduce dependencies on offshore testing teams and legacy infrastructure.
Technology Maturity: Monorepo structure (9GB) with mini-apps architecture, legacy Team City infrastructure with limited runners, 8-year-old Appium testing framework with broken CI/CD integration, and complex iOS certificate provisioning requiring specific team memberships.

Current State Assessment: Key Pain Points

Severe Infrastructure Bottlenecks: Team City runners created 5-7 hour total build times (wait time + agent availability + 2-hour execution), with queue times worsening significantly near 4-week release cycles as more teams competed for limited infrastructure.
Inefficient Testing Strategy: Running 25,000 unit tests for every PR regardless of change scope (even single-line JSON modifications), with 7-minute Appium test execution times and thousands of total test cases creating massive feedback delays.
Broken Automation Pipeline: Completely broken CI/CD integration meant developers relied on manual email reports from India testing team, with no automated visibility into regression testing status or failures.
Complex Repository Governance: Mobile Foundations team acting as "God reviewers" with mandatory approval requirements, fork-based development model creating additional complexity, and resistance to code reuse across existing modules.
Offshore Coordination Challenges: Hand-off to India team created multi-day feedback loops, communication gaps, and developer disconnection from testing results, with some team members concerned about role elimination.

Baseline Metrics (Where Available)

Metric Category	Baseline	Notes
Build Queue Time	5-7 hours total	Wait + agent availability + execution
Build Execution Time	2 hours	For full test suite
Test Execution (Appium)	7 minutes per test	Team City measurement
Test Execution (T2 Environment)	1-2 minutes per test	Real environment testing
Repository Size	9GB	Monorepo with mini-apps
Total Unit Tests	25,000	Run for every PR
Developer Team Size	150+ mobile developers	Native iOS/Android
Release Cycle	4 weeks	Queue times worsen near releases

Solution Overview

Engagement Strategy & Phases

Phase 1: Discovery & Problem Identification: Conducted route-to-live mapping for Cards Modernisation programme, identified testing framework brittleness and infrastructure bottlenecks, analysed Team City logs to understand true build queue impact.
Phase 2: Parallel Solution Development: Developed native testing POC using existing developer tools whilst simultaneously creating GitHub Actions + AWS EC2 pipeline solution, leveraging existing Team City build commands and dependencies.
Time to First Value: Delivered self-service native testing demonstrating 97% speed improvement and functional GitHub Actions pipeline eliminating queue times, proving both technical approaches viable.
Phase 3: Integration & Stakeholder Navigation: Attempted repository integration for testing framework, navigated Mobile Foundations "God reviewer" approval processes, discovered competing £2M infrastructure investment creating territorial conflicts.
Phase 4: Political Resolution Attempts: Addressed stakeholder exclusion from meetings, managed competing team concerns about investment displacement, documented technical solutions and organisational learnings despite non-production deployment.

QCE Disciplines Applied

Quality Engineering: Delivered comprehensive three-tier testing strategy leveraging existing Swift UITest and Espresso tools, implemented aggressive mocking for 15-second feedback loops, and established sustainable testing patterns that developers could maintain and extend.
Cloud Platform: Architected AWS EC2-based pipeline solution using Mac instances for iOS builds, developed automated certificate provisioning and simulator management, created cost-effective on-demand infrastructure replacing fixed Team City runners.
Developer Experience: Eliminated 5-7 hour queue times through self-service infrastructure, provided immediate local testing feedback, and created familiar tool-based workflows that required minimal learning curve for 150+ developers.

The Results: Measurable & Stakeholder-Centric Impact

Headline Success Metrics

Metric	Before Engagement	After Engagement	Improvement
Build Queue Time	5-7 hours	0 minutes (on-demand)	-100%
Test Execution (Mocked)	7 minutes	15 seconds	-97%
Test Execution (E2E)	7 minutes	1-2 minutes	-71% to -86%
Infrastructure Cost	£2M+ on-premises	£5M estimated AWS	Higher cost but eliminates queues
Developer Feedback Loop	Days (via India)	Seconds (local)	Near real-time
CI/CD Reliability	Broken	Functional (POC)	100% improvement

Note: Production deployment was not achieved due to organisational challenges, limiting business impact measurement.

Value Delivered by Stakeholder

For the 150+ Mobile Developers:
- Queue Elimination: Removed 5-7 hour build waits that were destroying productivity, especially near release cycles.
- Immediate Testing Feedback: Transformed testing from days (via India team) to seconds (local execution) using familiar tools.
- Self-Service Capability: Eliminated dependencies on both offshore teams and shared infrastructure limitations.
- Tool Familiarity: Leveraged existing Swift UITest and Espresso knowledge rather than requiring new tool adoption.
For Cards Modernisation Programme:
- Delivery Acceleration: Provided infrastructure foundation to support rapid feature development across 40+ card functions.
- Risk Mitigation: Demonstrated scalable testing approach that could handle programme expansion beyond initial 5% coverage.
- Technical Foundation: Delivered replicable patterns for other transformation programmes across the organisation.
For Engineering Leadership:
- Cost-Benefit Clarity: Provided concrete comparison between £5M AWS solution and hidden costs of developer productivity loss from queue times.
- Strategic Options: Delivered proven technical alternatives to £2M+ on-premises infrastructure investments.
- Organisational Insights: Revealed true impact of territorial politics and competing team investments on transformation success.
For Infrastructure Teams:
- Modernisation Path: Demonstrated migration approach from Team City to GitHub Actions with detailed implementation guide.
- Scalability Solution: Proved on-demand infrastructure could eliminate fixed runner limitations.
- Cost Transparency: Exposed hidden costs of queue times versus explicit cloud infrastructure expenses.

Organisational Learnings

Competing Investments Impact: £2M+ existing on-premises investments created significant political resistance to cloud-based solutions, regardless of technical superiority.
Territorial Dynamics: Multiple teams (Mobile Foundations, Infrastructure, India Testing) had overlapping responsibilities creating defensive responses to external solutions.
Process Compliance Criticality: Failure to raise formal dependencies in first quarter created mandate questions that undermined technical credibility later.
Stakeholder Exclusion Patterns: Being excluded from meetings about our own work indicated relationship breakdown requiring immediate attention and escalation.

Lessons, Patterns & Future State

What Worked Well: Both technical solutions delivered dramatic improvements - 97% test speed increase and complete queue elimination. Native testing approach leveraging existing developer tools proved optimal, and AWS EC2 solution successfully replicated all Team City functionality. The parallel workstream approach allowed us to solve multiple interconnected problems simultaneously.
Political Navigation Skills: Developed understanding that defensive stakeholder responses often indicate territorial concerns rather than technical disagreements, requiring different engagement strategies.
Challenges Overcome: Successfully navigated complex iOS certificate provisioning, created functional GitHub Actions pipelines for both iOS and Android, developed cost-effective Mac EC2 management, and proved technical feasibility despite organisational constraints. Jack's detailed implementation diagrams and technical documentation were particularly valuable for stakeholder communication.
Key Takeaway for Similar Engagements: When competing against multi-million pound existing investments, technical excellence alone is insufficient. Early stakeholder relationship building, formal process compliance, and explicit mandate establishment are prerequisites for delivery success. Political navigation skills are as critical as technical capabilities in large enterprise environments.
Replicable Assets Created: Native mobile testing framework with mocking patterns, GitHub Actions pipeline templates for iOS/Android, AWS EC2 Mac instance management scripts, certificate provisioning automation, and stakeholder engagement frameworks for politically complex environments.
Client's Future State / Next Steps: CBA retained comprehensive technical documentation and proof-of-concept implementations for both solutions. The Android pipeline approach was identified as potentially viable given lower infrastructure requirements (no Mac EC2 costs). The testing framework remains applicable when organisational readiness aligns with technical capability. Infrastructure team expressed interest in hybrid approach combining on-premises foundation with cloud overflow capacity.

Cost-Benefit Analysis Learnings

Hidden Queue Costs: Organisations often underestimate true cost of developer productivity loss from infrastructure bottlenecks - 150 developers waiting 5-7 hours represents massive hidden expense.
Explicit vs Hidden Costs: £5M annual AWS cost appears expensive but becomes reasonable when compared to hidden costs of queue times, delayed releases, and developer frustration.
Investment Displacement Sensitivity: Existing £2M+ infrastructure investments create political sensitivity requiring careful positioning of new solutions as enhancement rather than replacement.

Implementation Strategy Refinements

Stakeholder Mapping First: Begin all complex enterprise engagements with comprehensive stakeholder mapping including territorial concerns and existing investments.
Formal Process Compliance: Establish formal dependencies and approval processes immediately, even when they seem bureaucratic or unnecessary.
Gradual Displacement: Position new solutions as complementary or pilot approaches rather than direct replacements for existing team investments.