Data Domain Integration using Event Driven Architecture

Domain Data integration is a very common use case but often it’s wrongly designed. In this blog, I’ll discuss a couple of common integration approaches and their pitfalls. I’ll then explain a better integration approach to enhance maintainability.

Terminologies

The following terminologies have been used throughout the blog:

Producer Domain: A domain or solution responsible for creating data products.

Consumer Domain: A domain or solution consumes data produced by other domain(s).

Data Products: Data-as-a-product, the concept is based on Data Mesh principles.

Domain Events: Events produced by a domain e.g. onCreate, onUpdate, onDelete events.

References:

• Data Mesh Principles and Logical Architecture
• Data Domains — Where do I start?

Approach 1: Scheduled consumption

Context

Producer domain ingests data from various sources, cleanse, augments, transforms, aggregates and produces…