Shared Environments, Separate Boundaries: CommSec's Testing Innovation

PISR: Problem, Impact, Solution, Result

Problem: Commonwealth Bank's CommSec platform was unable to achieve reliable end-to-end test automation for its core Equities Trade Lifecycle (ETL) services after years of attempts. The architecture, a distributed monolith, was tightly coupled to shared external test exchanges, leading to non-deterministic, flaky tests that provided no reliable signal on deployment safety.
Business Impact: This resulted in 0% reliable E2E automation coverage despite years of attempts, forcing continued reliance on slow, expensive, and high-risk manual testing. The inability to automate validation for a platform processing billions in daily trading volume was a major blocker to adopting Continuous Delivery and reducing time-to-market.
Our Solution: Over 6 months, our 3-person team performed deep architectural analysis. The solution wasn't a new framework, but new infrastructure: a declarative Test Support API to manage complex data dependencies, header-based request routing to dynamically isolate the ETL services, and an application-boundary mock of the ASX exchange using WireMock, circumventing an intractable legacy integration.
Tangible Result: The transformation yielded the first-ever reliable automation for this domain. Key results include a ~92% reduction in test execution time (from >1 hour manually to <5 minutes), comprehensive E2E coverage for the entire trade lifecycle, and a deterministic test reliability rate of >99.9%. This established the technical foundation for safe, high-velocity deployments.

The Challenge

Business & Client Context

Primary Business Goal: Implement reliable, automated testing for the core Equities Trade Lifecycle (place, amend, cancel) to de-risk deployments and enable adoption of Continuous Delivery practices.
Pressures: High operational risk on a critical financial platform, intense competitive pressure to increase deployment velocity, and the need to overcome years of failed automation initiatives that had eroded confidence and wasted investment.
Technology Maturity: The platform was a partially-decomposed monolith with insufficient service contract definition, leading to high coupling. The core ASX exchange integration code was legacy and considered "impossible" to modify or mock at a low level. Previous test automation attempts had failed because they focused only on new test frameworks without addressing these deep architectural impediments.

ETL Service Context & Order Flow

The Equities Trade Lifecycle represents the core business function of CommSec's trading platform, encompassing the complete journey of a customer's trade order through a complex distributed system. Understanding this flow was key to understanding the testing complexity we faced.

Key Business Functions:

Order Placement: Customer orders received through various channels require complex validation, risk checks, and routing to the appropriate exchange.
Order Management: Real-time amendments, cancellations, and status updates must be processed immediately and reflected to the customer.
Exchange Integration: Direct, protocol-compliant connectivity to the ASX and other exchanges for order execution and real-time response handling.
Trade Settlement: Post-execution processing, where external systems perform callbacks into settlement systems, which in turn inform the Order Management API of the final trade status.

Current State Assessment: Key Pain Points

External Dependencies: Test automation was dependent on a shared ASX test exchange used by multiple financial institutions. Tests would frequently fail due to external factors, such as another company's test manipulating the same stock, making deterministic outcomes impossible.
Complex Test Data Orchestration: In a 300+ person engineering organisation, setting up the prerequisite state for a single trade was a monumental task. It required manual coordination across service domains to create users, configure accounts, set portfolio positions, and manage risk limits, taking upwards of 20 minutes per test.
Architecture: The tight coupling of the ETL services to the legacy exchange connector meant there were no clean seams for isolation. It was impossible to introduce test doubles or mocks where they were needed most, forcing all tests to run against the entire, unreliable, integrated system.

Baseline Metrics (Where Available)

Metric Category	Baseline	Notes
Reliable E2E Test Automation Coverage	0%	All previous automation suites were decommissioned due to extreme flakiness
Test Execution Reliability	Highly Unreliable	Failures were predominantly caused by external dependencies, not application bugs
Manual Test Data Setup Time	20+ minutes per test	Required navigating multiple UIs and potential cross-team coordination
Full E2E Test Cycle Time	> 1 hour (Manual)	No viable automated alternative existed for the complete lifecycle

Solution Overview

Engagement Strategy & Phases

Phase 1: Discovery & Baseline (Month 1): We conducted comprehensive evaluation, confirming the infeasibility of mocking the legacy ASX connector directly. This led to a critical pivot: instead of low-level mocking, we would implement mocking at the application boundary, controlled by intelligent routing.
Phase 2: Foundational Infrastructure (Months 2-3): Applying TDD, we built the core infrastructure. This included the initial Test Support API (starting with in-memory storage to prove the model) and the Equity Order Mock using WireMock. We focused on getting a single, simple 'place order' test running deterministically in CI to demonstrate immediate value.
Time to First Value: Delivered working end-to-end automation for a basic order flow within 4 weeks, proving the architectural approach before investing in comprehensive coverage.
Phase 3: Comprehensive Implementation (Months 4-5): With the foundation in place, we developed the comprehensive Playwright test suite for all ETL functions. The Test Support API was hardened during this phase, migrating its backend to Redis to support more complex, shareable test states.
Phase 4: Integration & DX Optimisation (Month 6): We built a React user interface for the Test Support API, enabling manual QA teams to visually create, save, and replay complex test states. This gave them the power of the automation infrastructure, saving over 20 minutes per test execution and bridging the gap between manual and automated testing.

Test-Driven Development Approach: Value-First Delivery

Our success was fundamentally rooted in focusing on getting value first, then working back to implement and improve the solution. Rather than building a comprehensive solution upfront, we leveraged Test-Driven Development (TDD) principles to drive implementation and ensure we didn't over-engineer.

Architectural Overview

The key insight was that we didn't need separate environments - we needed intelligent routing within a shared environment.

After State: Deterministic, Isolated Testing Within Shared Environment

Technical Deep Dive: The Declarative Test Support API

To solve the complex test data orchestration problem, we built a reusable Test Support API that became a valuable asset for all teams.

Core Concepts:

Test State: A collection of all resources and their configurations required for a specific test scenario. It represents an isolated, versioned test context.
Transaction: A single operation to create or modify a resource (e.g., CreateUser, AssignHoldings, SetMarketState). Each transaction specification includes its type, inputs, and dependencies on the outputs of other transactions.
Dependency Management: The API uses a declarative syntax (${alias.outputField}) allowing transactions to reference outputs from others, creating a directed acyclic graph (DAG) of dependencies.

Example API Request:

{
  "testId": "payment-flow-test-001",
  "description": "Create resources for payment flow test",
  "transactions": [
    {
      "type": "CreateUser",
      "alias": "primaryUser",
      "inputs": {
        "firstName": "John",
        "lastName": "Doe",
        "emailTemplate": "john.doe@example.com"
      }
    },
    {
      "type": "CreateAccount",
      "alias": "tradingAccount",
      "dependencies": ["primaryUser"],
      "inputs": {
        "accountType": "Trading",
        "userId": "${primaryUser.userId}",
        "initialBalance": 10000.00,
        "currency": "USD"
      }
    },
    {
      "type": "AssignHoldings",
      "alias": "appleStockHolding",
      "dependencies": ["tradingAccount"],
      "inputs": {
        "accountId": "${tradingAccount.accountId}",
        "symbol": "AAPL",
        "quantity": 100
      }
    }
  ]
}

Architecture and Execution Flow:

A test initiates by sending a single POST request to /api/v1/states containing the graph of transaction specifications.

The DependencyResolver component performs a topological sort on the transaction graph to determine the correct execution sequence.
It then iterates through the sorted list, invoking the appropriate TransactionHandler for each type (e.g., UserHandler, StockHandler, CashHandler).
Each handler is responsible for communicating with the underlying service to create or modify the resource.
The TestStateManager tracks all created resources against a unique test session ID. A corresponding DELETE /api/v1/states/{stateId} call tears down all resources in reverse dependency order, ensuring a clean environment after every test run.

State Management & Isolation:

The Test Support API implements comprehensive state management to ensure test isolation and prevent resource conflicts:

Resource Locking: The API creates locks to ensure only one test can use specific accounts or resources at a time, preventing race conditions in the shared development environment.
Automatic Teardown: When a test completes, the DELETE /api/v1/states/{stateId} endpoint automatically tears down all created resources in reverse dependency order, ensuring no test artifacts remain.
Test Isolation: Each test receives a unique state ID, creating complete isolation between concurrent test executions while allowing them to share the same underlying infrastructure.

QCE Disciplines Applied

Quality Engineering: Implemented modern test automation architecture that established clear test boundaries and eliminated flaky external dependencies. The combination of Test Support API with intelligent dependency management, request-based routing, and WireMock-based Equity Order Mock enabled deterministic test outcomes for critical trading functions whilst maintaining realistic integration test coverage across CommSec's multi-team environment.
Platform Engineering: Delivered re-purposed development environment infrastructure that maximised existing resource utilisation whilst enabling coexistent manual and automated testing. The header-based routing approach eliminated the need for environment duplication whilst providing comprehensive integration testing capabilities through intelligent request boundary management.
Developer Experience: Created streamlined test development and execution processes through Playwright framework integration and intelligent request routing. The re-purposed development environment approach eliminated environment provisioning delays whilst enabling both manual testers and automation to work simultaneously, providing immediate feedback without disrupting existing testing workflows.

The Results: Measurable & Stakeholder-Centric Impact

Headline Success Metrics

Metric	Before Engagement	After Engagement	Improvement
Reliable E2E Test Automation	0%	Comprehensive	First-time achievement
Test Execution Time	> 1 hour (Manual)	< 5 minutes	~92% Reduction
Test Reliability	Highly Unreliable	>99.9% Deterministic	Paradigm Shift
Manual Test Setup Time	20+ minutes	Near-instantaneous	>95% Reduction

Value Delivered by Stakeholder

For the CTO / CIO:
- Dramatically mitigated deployment risk on a critical, high-volume financial trading platform. (risk_mitigation: "Automated safety net for core trading")
- Unblocked the company's strategic Continuous Delivery initiative after years of being stalled. (business_agility: "CD enablement")
- Maximised ROI by re-purposing existing infrastructure, avoiding all new environment costs. (operational_efficiency: "Zero additional infra cost")
For the VP/Director of Engineering:
- Solved a high-visibility, multi-year technical challenge, boosting engineering morale and demonstrating a path forward for other legacy systems. (capability_building: "Solved intractable testing problem")
- Enabled teams to reliably test complete order lifecycles with deterministic outcomes, leading to higher deployment confidence and velocity. (team_productivity: "Reliable automation after years of failures")
For the DevOps / Platform Manager:
- Delivered the Test Support API as a reusable, platform-level asset for test data orchestration, benefiting multiple teams. (developer_experience: "Automated cross-service test data orchestration")
- Eliminated a major source of environment contention by allowing automated and manual testing to coexist safely. (infrastructure_efficiency: "Dual-purpose testing environment")

Client Testimonials

Client testimonials are pending as this represents an internal learning document focused on technical delivery insights rather than external marketing material.

Lessons, Patterns & Future State

What Worked Well

Architectural-First Analysis: By correctly diagnosing the problem as architectural rather than tooling-related, we avoided repeating past failures. This took significant time upfront but was absolutely critical.
Value-First TDD Approach: The agile delivery methodology that prioritised getting end-to-end order tests running in CI quickly, then iteratively improving the underlying implementation, avoided over-engineering whilst demonstrating value within weeks.
Application Boundary Mocking: The "impossible" legacy integration was solved not by changing it, but by building a seam around it. This approach provided necessary isolation without costly and risky re-architecture.

Challenges Overcome

Legacy System Constraints: Rather than fight the legacy ASX connector, we worked around it by implementing intelligent routing at the application boundary. This required accepting that we couldn't achieve perfect unit-level isolation but could achieve excellent functional isolation.
Multi-Team Coordination: The Test Support API became the key to enabling cross-team testing without requiring constant coordination. By making test data setup declarative and automated, we eliminated the human bottleneck.
Environment Resource Constraints: The dual-purpose environment approach meant we couldn't have pristine test environments, but the benefits of shared resources and reduced infrastructure costs outweighed this limitation.

What We'd Do Differently

Earlier Stakeholder Alignment: We should have involved the manual QA teams in the design process from day one. Adding the React UI in the final phase was valuable but came late - earlier involvement would have improved adoption.
Performance Testing from Start: The Test Support API became a bottleneck as test suites grew. We should have implemented performance monitoring and optimisation strategies earlier in the development cycle.
Documentation Strategy: We focused heavily on making the system self-explanatory through good APIs, but should have invested more in comprehensive documentation for onboarding new teams to the patterns we established.

Key Takeaway for Similar Engagements

For any complex distributed system, a declarative, dependency-aware Test Support API is not a luxury; it is a prerequisite for achieving scalable and deterministic test automation. However, the key insight is that you don't always need separate environments - intelligent routing within shared environments can provide the isolation you need whilst maximising resource efficiency.

Replicable Assets Created

Re-purposed Development Environment Architecture: An intelligent routing pattern to enable coexistent manual and automated testing without environment duplication.
Test Support API with Intelligent Dependency Management: A proven architecture for a reusable service that orchestrates complex test data setup across service domains using topological sorting.
Application-Boundary Mocking Strategy: A pragmatic pattern for testing services with intractable external dependencies that can't be modified or mocked at the code level.
Header-Based Request Routing Pattern: A lightweight approach to achieve test isolation within shared environments without infrastructure duplication.

Client's Future State / Next Steps

With reliable end-to-end test automation established for core ETL functions using the re-purposed development environment approach, CommSec is positioned to extend automated testing across their broader trading platform and implement Continuous Delivery practices. The coexistent testing infrastructure provides a foundation for scaling automation to additional trading functions and markets without requiring additional environment investments.

The established patterns and tooling enable teams to independently develop and maintain automated tests whilst preserving existing manual testing workflows, supporting ongoing platform evolution whilst maintaining deployment confidence for critical trading operations. The Test Support API has become a foundational platform service that other teams are now adopting for their own testing challenges.

Action Items for Practice

Pattern Documentation: We should formalise the dual-purpose environment pattern as a standard QCE practice for clients with resource constraints or complex legacy systems.
Test Support API Template: The declarative test data orchestration approach should be packaged as a reusable accelerator for future financial services engagements.
Legacy Integration Playbook: Document the application-boundary mocking strategy as an alternative when traditional test doubles aren't feasible.
Training Materials: Develop internal training on header-based routing patterns and when they're preferable to environment duplication.

This engagement demonstrated that some of our most impactful work comes not from implementing new technologies, but from applying creative architectural thinking to seemingly intractable problems. The CommSec success has become a reference architecture for other teams facing similar legacy testing challenges.