American Security is one of the largest providers of connected safes to both consumers and small businesses. Their CashWizard online portal enables their customers to modify safe configurations and view a wide variety of reports to track and audit user activity with a safe.
“Protagona has been great to work with. They quickly understood our technical needs, resources capabilities and adapted to them. Throughout the engagement communication and transparency has been key in keeping the project moving forward. I particularly appreciate their ability to help bridge the understanding of the non-technical individuals.” – Mara Ghisani
Challenge
American Security is a provider of smart safe products. Its CashWizard product line has been in the market for well over a decade, and gives its customers unparalleled visibility into daily cash transactions.
As American Security’s customer base grew, the volume of data CashWizard was required to capture and process outgrew its relational data model. The volume of data to be aggregated started outpacing the original extraction, transformation, and loading (ETL) processes that were in place, applying additional pressure on data lifecycle jobs that quickly became part of the critical path for Cash Wizard to stay performant.
Quickly approaching the limits of their relational database cluster, American Security recognized the need to pivot, and sought Protagona’s expertise to propose and build a platform for the future –one that could be operated and extended using cloud-native technologies.
Solution
Protagona began by analyzing the current volume of data and the payloads for each request. We then explored what the end state of the data should look like to end users as they generate audit reports and other analytics. Taking into account the customer’s existing operational and engineering capabilities, as well as the expected customer growth, Protagona designed a multi-stage data lake to accommodate both current and future pain points related to data ingestion and report generation.
Understanding that this was new territory for American Security, Protagona ensured that every layer of the new platform was instrumented. Opting to stay AWS-native, CloudWatch provided optimal visibility, giving the customer a clear view of the platform’s operation and health. This empowered AmSec to stay ahead of future demands from added data volume, and increased processing capacity to maintain optimal performance.
As the structure of the data lake was rolled out, AWS Database Migration Service was used to migrate data from SQL Server into the data lake. The solution used API Gateway to provide a REST interface for IoT-enabled safes, allowing Protagona to implement a Strangler-pattern to incrementally route traffic to the new ingestion platform, while leveraging the legacy .NET monolithic application for all other API calls.
DynamoDB, Lambda, SQS, and S3 provided the foundation of the data lake and ETL pipelines. The pipeline was instrumented through CloudWatch+X-Ray, to further extend the ability to see end to end traces and integrated metrics. While ElastiCache provided much needed relief to the existing relational database from frequent read requests.
To ensure that the solution could be easily maintained, Protagona wrapped platform deployment automation using CodeSuite, giving American Security an intuitive, modern GitOps workflow.
This approach ensured that American Security could now move away from expensive relational database engines and traditional storage and offload data, data analytics, and any future insights derived from their data ingestion to cloud-native technology.
It also provided AmSec with the capability to monitor and observe CashWizard traffic through a new lens courtesy of API Gateway + CloudWatch.
Tech Stack
- AWS VPC
- AWS EC2
- AWS Lambda
- AWS S3
- AWS SQS
- AWS DynamoDB
- AWS API Gateway
- AWS CodePipeline
- AWS CodeCommit
- AWS Elastic Cache
- Terraform
Outcome
Performance
By moving away from batch processes and embracing an event-driven architecture in AWS, the average processing time for event data used by customer reports was reduced by more than 80%.
Observability
By consolidating logging, metrics, tracing, and business KPI’s under a common platform, American Security can now monitor, react, and stay ahead of the operational demands of their CashWizard deployment base.
Future Insights
Leveraging Amazon S3 as the primary datastore eliminated the need to continually run complex, single-threaded data archival jobs. It also gave American Security a structured data lake they can tap into to extract business and operational insights not yet uncovered.
Cost
Shifting away from a relational data model to Amazon S3 reduced the need for secondary reporting databases and the infrastructure required to support execution of background aggregation and archival jobs. Not only did this increase data retention limits, but these efficiencies have also led to cost savings of more than 70%.