Financial Exchange Infrastructure Modernization

Executive Summary

A leading financial exchange initiated a complete modernization of its public digital platform. The objective extended beyond redesigning the website; it also required building a modern infrastructure capable of supporting continuous availability, disaster recovery, deployment automation, and operational visibility.

The existing environment consisted of a traditional two-tier architecture with one web server and one database server in both Production and Disaster Recovery environments. Although functional, the infrastructure introduced operational risks, manual deployment processes, limited monitoring, and longer recovery times during failures.

To address these challenges, I designed and implemented a modern Active-Passive on-premises architecture with dedicated frontend and backend application servers, centralized MySQL and MinIO storage services, self-hosted GitLab for CI/CD, and a complete observability stack using Prometheus, Grafana, and Loki.

The implementation significantly improved platform resilience, deployment consistency, operational visibility, and disaster recovery readiness while creating a scalable foundation for future growth.

Client Background

The client is one of India's established financial exchange organizations operating critical market-facing digital services.

As part of a larger digital transformation initiative, the organization decided to completely redesign its public website using modern technologies while simultaneously modernizing the supporting infrastructure.

The project demanded an infrastructure capable of supporting enterprise-grade availability, operational resilience, disaster recovery, and future scalability while remaining entirely on-premises.

Project Overview

The project involved much more than deploying applications. It required designing an infrastructure that could reliably host multiple production workloads while reducing operational complexity and improving service availability.

The implementation covered:

1. Production Environment: Multi-server Active-Passive setup
2. Disaster Recovery Environment: Geographically separated standby infrastructure
3. Frontend Application: High-performance Next.js rendering layer
4. Backend APIs: Node.js Market Data & CMS microservices
5. CMS Layer: Headless content management service
6. Object Storage: MinIO distributed asset repository
7. Database Layer: Centralized MySQL database with replication
8. Deployment Automation: Self-hosted GitLab & GitLab Runners
9. Monitoring Platform: Prometheus, Grafana, and Loki log aggregator

Every component was designed with long-term maintainability and operational resilience in mind.

Legacy Architecture (Old)

The original infrastructure followed a simple two-tier deployment model. Both the Production and Disaster Recovery environments contained:

1. One Application Server
2. One Database Server

Although straightforward to manage, the infrastructure presented several operational limitations. Application services shared the same server, deployments were largely manual, monitoring capabilities were limited, and application availability depended heavily on a small number of systems.

As business requirements evolved, the existing design was no longer sufficient to support a mission-critical financial platform.

Business Challenges

The modernization project addressed both application-level and infrastructure-level challenges. The existing website was built on an older technology stack, making further enhancements increasingly difficult.

Operational & Infrastructure Challenges

1. Heavy dependence on only two application servers.
2. No application redundancy within the production environment.
3. Manual deployment process across environments.
4. No centralized source control or automated CI/CD pipeline.
5. Limited infrastructure monitoring and alerting.
6. Limited application observability and log traceability.
7. Higher operational effort and risk during deployments.
8. Longer recovery time during application or server failures.
9. Infrastructure not designed for future scalability.

Key Project Objectives

To overcome these bottlenecks, the project was designed around clear strategic objectives:

1. Improve application availability and eliminate single points of failure.
2. Reduce downtime during hardware or software failures.
3. Introduce application redundancy with Active-Passive server pairing.
4. Improve Disaster Recovery readiness with live data replication.
5. Standardize application deployments via automated pipelines.
6. Centralize infrastructure monitoring with real-time dashboards.
7. Improve operational visibility with aggregated log management.
8. Reduce manual operational effort across engineering teams.
9. Create a scalable infrastructure foundation for future financial services.

Solution Architecture (New)

To address these challenges, I proposed and implemented a completely redesigned Active-Passive architecture. The new infrastructure introduced dedicated responsibilities for each service rather than combining multiple workloads on the same systems.

The Production Environment included:

1. Active Frontend Server & Passive Frontend Server
2. Active Backend Server & Passive Backend Server
3. Dedicated Centralized MySQL Database
4. Dedicated MinIO Object Storage Cluster
5. Self-hosted GitLab & CI/CD Runners
6. Isolated UAT Environment

A geographically separated Disaster Recovery Environment mirrored the production architecture using asynchronous MySQL replication and MinIO site replication.

The architecture was designed to ensure service continuity, simplified deployments, and improved operational resilience while maintaining a clear separation between application, storage, deployment, and monitoring layers.

Implementation Strategy

To minimize deployment risk, the migration was completed in multiple controlled phases. This phased implementation allowed each component to be independently validated prior to public cutover.

Roles & Responsibilities

I led the technical implementation of the infrastructure as the Cloud Engineer & Infrastructure Architect.

My responsibilities included:

1. Designing the overall Active-Passive infrastructure architecture.
2. Planning server sizing, capacity planning, and resource allocation.
3. Recommending network segmentation, VLAN isolation, and deployment topology.
4. Configuring and hardening all Enterprise Linux (RHEL 9) servers.
5. Installing and configuring runtime application dependencies (Node.js, Redis).
6. Deploying frontend, backend, and CMS applications.
7. Configuring Nginx reverse proxy routing and automatic failover rules.
8. Deploying and tuning Redis in-memory cache layer.
9. Deploying MySQL centralized database and setting up DR replication.
10. Deploying MinIO object storage and configuring site replication.
11. Installing and configuring self-hosted GitLab server and CI/CD Runners.
12. Implementing GitLab CI/CD multi-server deployment pipelines.
13. Deploying Prometheus, Grafana, and Loki observability stack.
14. Validating production readiness and conducting failover tests.
15. Supporting production go-live and post-deployment monitoring.

Technical Implementation Details

The implementation covered multiple interconnected infrastructure layers to ensure maximum performance, security, and maintainability:

Linux Infrastructure

Provisioned and hardened Red Hat Enterprise Linux (RHEL 9) servers according to enterprise security baselines. System tuning included optimizing kernel parameters, file descriptor limits, and SSH hardening.

Application Layer

Configured Next.js frontend, Node.js Market Data API, CMS API, and supporting backend services. Redis was integrated as a high-speed caching tier to reduce database workload during market traffic spikes.

Reverse Proxy & Traffic Routing

Implemented Nginx reverse proxy to route incoming HTTPS traffic, manage SSL termination, and support the Active-Passive failover design. Upstream server blocks were configured with health checks to automatically route traffic away from unhealthy application nodes.

Database Layer

Configured MySQL as the centralized production database with asynchronous binary log replication to the Disaster Recovery environment, ensuring minimal Recovery Point Objective (RPO).

Object Storage

Implemented MinIO object storage with site replication to synchronize application media, documents, and market data assets in real-time between Production and Disaster Recovery buckets.

DevOps & CI/CD Automation

Built a complete deployment workflow using self-hosted GitLab and GitLab Runners. Pipelines were built to run automated linting, build checks, and zero-downtime deployments to both Active and Passive server nodes in parallel.

Observability Stack

Designed and deployed a centralized monitoring platform consisting of:

1. Prometheus: Infrastructure metric collection (CPU, Memory, Disk, Network, Process health) via Node Exporters.
2. Grafana: Custom visual dashboards for real-time system monitoring and threshold alerting.
3. Loki & Promtail: Centralized log aggregation, capturing Nginx access logs, application errors, and system logs without requiring heavy database indexing.

Challenges Faced During Implementation

One of the primary challenges involved enterprise network connectivity and security compliance.

Because the infrastructure operated inside a secured financial enterprise network, multiple servers existed across isolated network segments (VLANs) with restricted outbound communication and strict firewall rules.

This required careful coordination with the networking and security teams to establish:

1. Explicit domain and URL whitelisting for internal API communication.
2. Firewall port approvals between DMZ, application, and database zones.
3. Cross service communication between Active and Passive nodes.
4. Secure database accessibility for applications while blocking external access.
5. MinIO site replication port access across geographic boundaries.

Resolving these network dependencies early through comprehensive matrix mapping prevented production blockers during final cutover.

Validation & Testing

Every infrastructure component underwent comprehensive validation before production deployment. Testing encompassed:

1. Network connectivity validation.
2. Firewall rule verification across all ports.
3. URL accessibility and SSL certificate validation.
4. Application-to-backend API communication latency checks.
5. Database connection pooling and query performance validation.
6. MinIO asset upload, download, and cross-site sync verification.
7. GitLab CI/CD deployment pipeline execution across Active/Passive nodes.
8. Prometheus metric scraping and Grafana dashboard verification.
9. Centralized log delivery to Loki.
10. Simulated application and server failover testing.

Production Go-Live & Verification

After successful validation, all production applications were made publicly available using the organization's production DNS and public IP configuration.

Deployments were automated through self-hosted GitLab, enabling simultaneous deployments to both Active and Passive application environments without manual SSH intervention.

Following production deployment, extensive post-deployment verification confirmed:

1. Full application availability and crisp response times under live traffic.
2. Deployment consistency across Active and Passive application nodes.
3. Infrastructure stability across database and storage layers.
4. Active metric collection and operational log streaming in Grafana/Loki.
5. Automatic failover readiness in the event of hardware or node failure.

Business Impact

The redesigned infrastructure significantly improved operational resilience and transformed platform management:

Key Improvements Achieved

1. Reduced downtime during application failures via automated Nginx failover.
2. Standardized application deployments through self-hosted GitLab CI/CD workflows.
3. Centralized infrastructure monitoring with instant alerting on metrics.
4. Improved visibility into application health, errors, and system resource usage.
5. Faster incident detection and rapid troubleshooting through Loki centralized logs.
6. Improved Disaster Recovery preparedness with verified live data synchronization.
7. Reduced manual operational effort for engineering and deployment teams.
8. Improved operational confidence during production releases.
9. Scalable architecture foundation capable of supporting future platform expansion.

Key Learnings

This project significantly expanded my experience in designing and operating enterprise-scale on-premises infrastructure. Some of the most valuable learnings included:

1. Designing for business continuity — Infrastructure must be built around continuous availability rather than simply deploying code.
2. Phased production migrations — Breaking complex migrations into controlled phases mitigates release risk.
3. Navigating enterprise network requirements — Early alignment with network and security teams on firewall rules and isolated VLANs is critical.
4. Operating multi-server environments — Maintaining configuration parity across Active and Passive nodes is essential for seamless failover.
5. On-premises DevOps & Observability — Building enterprise-grade CI/CD and monitoring stack on bare-metal/virtualized Linux without relying on cloud-native SaaS requires deliberate architectural planning.

Technologies Used

Project Outcome & Conclusion

The project successfully delivered a modern, production-ready on-premises platform that met the organization's objectives for availability, resilience, deployment automation, and operational visibility.

By introducing dedicated infrastructure layers, automated CI/CD, centralized monitoring, and Disaster Recovery replication, the platform became significantly easier to operate, more resilient to failures, and better prepared for future growth. The phased implementation also ensured a smooth transition with minimal disruption to production services.

"Modern infrastructure is not defined by the number of servers it uses but by how well it supports business continuity, operational reliability, and future growth."

This project demonstrates how a carefully planned Active-Passive architecture, combined with deployment automation and centralized observability, can transform a traditional on-premises environment into a resilient enterprise platform.