From Data Silos to Unified Publishing Intelligence

This organization curates and delivers book collections for school and public libraries across the United States. Editorial and curation teams depend on rich, accurate title metadata — sourced from publishers, industry databases, and proprietary systems — to build collections and serve library readers. As product lines expanded, so did the number of systems holding fragments of that metadata: an ERP, a proprietary title management platform, a vendor submission portal, external sources like Library of Congress and BISAC, and span planning documents living only in spreadsheets. The goal was a single, authoritative source of truth.

Protagona was engaged to validate the architecture and data ingestion patterns needed to consolidate a fragmented title and metadata landscape into a unified data lake. Success meant demonstrating that data from two core internal systems — Myriad and Jedi — alongside external sources could be normalized, linked by ISBN, and made queryable, establishing the foundation for automated metadata enrichment and cross-team intelligence.

The core complexity was not moving data — it was reconciling data never designed to interoperate. Multiple systems held overlapping but inconsistent representations of the same titles, with no shared key reliably connecting them. ISBN management alone was a significant architectural challenge: a single title can carry multiple ISBNs across formats and editions, and different business units care about different ones. Any unification strategy had to preserve that multiplicity. The two systems in scope for the POC — Myriad and Jedi — sat at very different points in their lifecycle, with inconsistent versioning practices, varying retention requirements spanning up to seven years, and external sources ranging from well-structured APIs to no programmatic access at all.

Protagona designed a cloud-native data lake on AWS, using Amazon S3 as the central storage layer organized to accommodate varying retention policies and data formats. An ingestion framework pulled from both internal databases and external metadata sources, applying normalization logic to reconcile inconsistent publisher-supplied data — particularly around ISBN variants and imprint hierarchies. AWS Glue cataloged and transformed incoming data; Amazon Athena enabled querying without requiring a separate analytical database.

‍

Title unification by ISBN was handled at the Amazon QuickSight layer, with Amazon Q Chat used to associate alternate ISBNs across library, trade, and audio editions to a parent title record. This preserved each business unit's distinct view of the same title while surfacing unified intelligence through system-specific dashboards — eliminating the need to log into multiple systems for cross-title queries.