Installation

Amazon EKS

OvalEdge supports the multi-pod deployment architecture provided by AWS EKS.

Overview

OvalEdge is a n-Tier web application that can be deployed on multiple infrastructure platforms like:

  1. Cloud based containerized infrastructure. AWS EKS, AWS Fargate, Azure AKS
  2. Cloud based VM environments. AWS EC2, Azure IaaS
  3. On-premise environments. Linux, Windows

On the AWS cloud, OvalEdge can be deployed on any of the following environments:

  • Elastic Kubernetes Services (EKS)
  • Fargate
  • EC2

The scope of this article is the deployment of OvalEdge on AWS EKS containerized environment.

OvalEdge Version

Version Deployment Platform Editions

Release 5.1.x

Release 5.0.x

AWS Community, Enterprise, Premium, Standard

Pre-requisites

  1. AWS IAM user account is required.
  2. Access permissions for EKS, RDS, Secrets Manager.

Solution Architecture Overview

OvalEdge can be deployed as a single instance or as a cluster that hosts multiple pods of OvalEdge platform, which perform the required activities to extract metadata of data and reporting entities from the configured data sources and process it to provide the stated functionality. The multi-pod architecture is targeted for high usage environments to provide horizontal scalability.

Data connectors on the system are configured on the system by the users to connect to the data sources and once done, the connectors are used to extract metadata on a periodic schedule in the form of jobs. Access to these data systems should be enabled on the network, both internal and external. The data jobs are distributed across the pods setup to balance the load on the system. The number of pods can be varied based on the job load observed. Each pod is independent of the others and pods can be scaled up or down based on the load.

The metadata processed data from the same, user defined queries for data governance, data quality are stored in the OvalEdge database. Sensitive information like passwords are stored in non-human readable encrypted form. All the pods in a cluster connect to one database instance.

Users have to access the system through the web portal. The user requests are distributed across the pods in the setup through a load balancer, to provide optimal response time. User authentication is single sign-on enabled through integration with the Active Directory services and other supported options.

The following diagram illustrates the logical architecture described.

Integration Architecture

Infrastructure Deployment Architecture

Hardware Specifications

The below are the Solution Components and the required Sizing details based on the above NFR inputs.

   SI. Component Deployment Model  Sizing Required Remarks
   1

AWS Elastic Kubernetes Services (EKS)

PaaS

t2.xlarge 2core 16GB RAM nodes x 730 Hours; Pay as you go; 2 General Purpose SSD (gp2) Volumes - 100GB, 0 clusters

Basic specification

   2

AWS RDS MySQL

PaaS

Single Server Deployment, General Purpose Tier, 4 vCore x 730 Hours, 100 GB Storage, 0 GB Additional Backup storage - LRS redundancy

 

   3

AWS Firewall

PaaS

0 VM nodes x 730 Hours, 0 App Service nodes x 730 Hours, 0 SQL Database servers x 730 Hours, 0 Storage transactions, 0 IoT Devices, 0 IoT Message transactions, 64 Kubernetes vCores x 730 Hours

Indicative specification

   4

AWS Secrets Manager

PaaS

10,000 operations, 10,000 advanced operations, 0 renewals, 0 protected keys, 0 advanced protected keys

Indicative specification

   5

AWS Cloud Watch

PaaS

8 VMs monitored, 5 GB average log size, 90 additional days of data retention

Indicative specification

Note : The specification for Fargate and EC2 options will be similar to the EKS configurations, for multi-pod deployments.

Deployment Automation

The various deployment activities are detailed in the table below:

Deployment Activity Artifacts

First Time Deployment

OvalEdge binary WAR file.

SQL DDL scripts file for the database.

Deployment YAML file.

Helmchart deployment file (Optional).

Upgrade

Docker image name

Pod scale up / down

Docker Image name

Automated pod scaling

This is part of the roadmap.

Scalability

Vertical scalability for OvalEdge instance can be achieved by increasing the H/W configuration of the node(s).

For horizontal scalability, OvalEdge supports the multi-pod deployment architecture provided by AWS EKS. The number of pods in the deployment can be scaled up or down, through EKS, to match the load. There are two pod configurations:

  • User Pod: This pod configuration is for servicing user requests and can be scaled up or down based on the number of concurrent user load. There should be a minimum of 1 pod.
  • Job Pod: These pods are configured for executing the background jobs for data catalog building, data governance management and other processing intensive processes. There should be a minimum of 1 job pod. 

High Availability

The multi-pod architecture ensures high availability through its ability to be deployed on different hardware across different regions. 

In addition, to handle disruptive scenarios, a disaster recovery deployment is recommended, as depicted in the above section - 'Infrastructure Deployment Architecture'. AWS Backup service will be used for automated backup of OvalEdge application data in MySQL DB.

Monitoring & Alerting

The OvalEdge deployment can be monitored using:

  • OvalEdge health check monitor, which is an in-built feature.
  • AWS Cloud Watch
  • Prometheus exporter, which exports the monitoring data to a Prometheus server. This is integrated with the OvalEdge platform.