Most introductions to lakehouse architecture stop at the basic layer stack – storage, table format, catalog, compute, consumption. That’s necessary but not sufficient. Within that stack, there are several established architectural patterns for how data actually flows, gets organized, and gets owned, and picking one that doesn’t fit your organization creates friction that shows up months later, usually as either an ungoverned mess or a rigid bottleneck.
This post catalogs the architecture patterns worth knowing before you commit to one – what each looks like, where it came from, and what situation it actually fits. For the underlying layer stack these patterns get built on top of, our companion piece on the core layers of a modern lakehouse architecture is a useful starting point.
If you’re choosing a pattern for your own architecture right now, our Data Lake Services team at Algoscale works through exactly this decision with clients regularly.
Six Lakehouse Architecture Patterns
Medallion Architecture
The most widely adopted pattern, medallion architecture organizes data into progressive quality tiers: a bronze layer holding raw data exactly as ingested, a silver layer holding cleansed and validated data, and a gold layer holding business-ready, aggregated data for direct consumption. Its appeal is simplicity – every team and every pipeline shares the same mental model of “how refined is this data,” which makes onboarding new engineers and auditing data quality considerably easier. Medallion architecture works well as a default pattern for most organizations, particularly ones without a strong need for decentralized ownership across many business domains.
Data Mesh / Domain-Oriented Lakehouse
Data mesh reframes the lakehouse around decentralized ownership: rather than one central team managing all data, each business domain (finance, marketing, product) owns its own data as a product, with centralized standards for governance, catalog, and access control tying the domains together. This pattern fits large organizations where a single central data team has become a bottleneck, and where domain teams have enough technical capability to own their own pipelines responsibly. It requires more organizational maturity to implement well than medallion architecture alone, since it depends on genuinely shared tooling and standards to avoid recreating silos under a different name.
Hub-and-Spoke Lakehouse
In a hub-and-spoke pattern, a central, governed lakehouse (the hub) holds the authoritative, cleaned data, while individual teams or business units maintain their own smaller “spoke” environments – data marts or workspaces – built from data pulled out of the hub for domain-specific analysis. This pattern suits organizations that want strong central governance over source-of-truth data while still giving individual teams flexibility to experiment or build specialized views without affecting the central system. It’s a reasonable middle ground between full centralization and full data mesh decentralization.
Lambda Architecture
Lambda architecture combines two parallel processing paths: a batch layer that processes data on a schedule for comprehensive, accurate historical views, and a speed layer that processes data in near-real-time for low-latency, if slightly less complete, current views. Applied to a lakehouse, this typically means batch pipelines writing to gold-layer tables on a schedule, alongside a streaming pipeline serving a real-time view that eventually gets reconciled with the batch-processed data. This pattern fits use cases genuinely requiring both real-time dashboards and rigorously accurate historical reporting, at the cost of maintaining two parallel processing paths.
Kappa Architecture
Kappa architecture is a response to Lambda’s complexity: rather than maintaining separate batch and streaming pipelines, everything flows through a single streaming pipeline, with batch-style historical processing handled by replaying the stream from an earlier point when needed. This significantly simplifies the pipeline architecture – one code path instead of two – but requires a streaming platform capable of retaining and replaying historical data efficiently, and works best for organizations whose workloads are naturally stream-oriented already, rather than a mix of batch and real-time needs.
Data Vault on a Lakehouse
Data Vault is a modeling methodology – originally developed for data warehouses – that separates data into hubs (core business entities), links (relationships between entities), and satellites (descriptive, historized attributes). Applied within a lakehouse, this pattern provides strong auditability and historical tracking, since Data Vault’s design inherently preserves a full history of changes rather than overwriting records. This makes it a strong fit for heavily regulated industries – financial services, healthcare – where auditability and the ability to reconstruct historical states are compliance requirements, not just nice-to-haves. Our post on data lakehouse architecture for healthcare touches on the kind of compliance context where this pattern often becomes relevant.
Comparing the Patterns
Medallion architecture is the right default for most organizations without a specific reason to choose something else – it’s simple, well-understood, and works at almost any scale.
Data mesh fits large organizations with multiple mature domain teams and a central team that’s become a bottleneck, but requires real organizational investment to implement well.
Hub-and-spoke fits organizations wanting strong central governance with some team-level flexibility, without going as far as full domain decentralization.
Lambda architecture fits use cases genuinely needing both real-time and rigorously accurate batch views simultaneously, and can justify its added complexity for those specific requirements.
Kappa architecture fits organizations whose workloads are already fundamentally stream-oriented, where maintaining a separate batch path would be redundant complexity.
Data Vault fits regulated industries where full historical auditability is a compliance requirement, more than a general-purpose choice for most organizations.
Can You Combine Patterns?
Yes, and in practice, many real architectures do. It’s common to run medallion architecture within each domain of a data mesh implementation – the domain-ownership pattern determines who owns what, while medallion architecture determines how each domain organizes its own data internally. Similarly, a hub-and-spoke lakehouse might use Data Vault modeling specifically within the hub, for entities where historical auditability matters, while spokes use simpler modeling for their own specific analysis. These patterns aren’t mutually exclusive – they answer different questions (who owns data, how is it staged, how is it modeled) and can be layered together deliberately.
For a look at how this kind of structure supports genuine organization-wide consistency, our post on creating a single source of truth using data lakehouse architecture covers the governance principles that apply regardless of which specific pattern you choose.
Common Mistakes When Choosing a Pattern
Adopting data mesh without the organizational maturity to support it. Data mesh requires genuinely capable domain teams and shared tooling discipline – attempting it without both tends to recreate the exact silos it’s meant to prevent, just with a trendier name.
Choosing Lambda architecture by default rather than because it’s actually needed. Maintaining two parallel processing paths is real ongoing complexity; it’s only worth it when you genuinely need both real-time and comprehensive batch views, not as a default “just in case” choice.
Ignoring auditability requirements until a compliance review forces the issue. Organizations in regulated industries that skip Data Vault or equivalent historization patterns early often face an expensive retrofit once an audit surfaces the gap.
Treating pattern choice as permanent and unchangeable. Most organizations’ architecture evolves – starting with medallion architecture and layering in data mesh principles later as the organization grows is a reasonable, common trajectory, not a failure to plan correctly the first time.
Choosing the Right Pattern for Your Organization
Architecture patterns aren’t one-size-fits-all, and the right choice depends on your organization’s size, team structure, regulatory context, and latency requirements – not which pattern is generating the most conference talks this year. At Algoscale, our Data Lake Services team has implemented all of these patterns across different client environments and helps organizations choose (and combine) the ones that actually fit their situation.
To see the broader range of data engineering and analytics work we do beyond architecture pattern selection specifically, take a look at what Algoscale builds across the data stack.
Why Algoscale
A few things shape how we actually deliver on data lake and data engineering work, beyond the architecture and practices covered above:
● Pre-built accelerators. We don’t start every engagement from a blank slate – proprietary accelerators built from prior implementations speed up common data source integration and analytics patterns.
● Faster time to value. For a focused initial scope covering core data sources and first analytics use cases, our accelerators typically compress development timelines to around four weeks, rather than the several months a from-scratch build often takes.
● Built on a scalable framework. Our implementation approach follows a repeatable, scale-ready framework, so the architecture built for your first use case extends cleanly as data volume and teams grow, rather than requiring a redo.
● Microsoft Solution Partner for Data & AI. Algoscale holds Microsoft Solution Partner status for Data & AI, including specific expertise implementing Microsoft Fabric as a modern data warehouse.
Frequently Asked Questions
1. Which architecture pattern should a small or mid-sized company start with?
Medallion architecture is the most reasonable default for most organizations without a specific reason to choose otherwise – it’s simple, well-documented, and doesn’t require the organizational maturity that patterns like data mesh depend on.
2. Is data mesh just medallion architecture with a different name?
No. Data mesh addresses organizational ownership – who owns which data and how domains coordinate. Medallion architecture addresses data staging – how raw data gets refined into business-ready data. They answer different questions and are commonly used together, not as substitutes for each other.
3. Do we need Kafka or a similar streaming platform to use Kappa architecture?
Yes, effectively – Kappa architecture depends on a streaming platform capable of retaining and replaying historical data efficiently, since that replay capability substitutes for a separate batch processing path.
4. Is Data Vault modeling only relevant for regulated industries?
It’s most commonly adopted in regulated industries because of the compliance and auditability benefits, but any organization with a genuine need for detailed historical tracking of how data changed over time can benefit from it, regardless of formal regulatory requirements.
5. Can we switch architecture patterns after we’ve already built our lakehouse?
Yes, though the effort involved varies by pattern. Moving from an ungoverned structure to medallion architecture is a relatively contained project. Moving to data mesh is a bigger organizational and technical undertaking, since it involves redistributing ownership, not just restructuring pipelines.
6. How do we know if Lambda architecture’s added complexity is actually worth it for us?
If you have specific, genuine requirements for both low-latency real-time views and comprehensive, fully accurate historical reporting from the same data, Lambda’s complexity is justified. If you only need one or the other, a simpler single-path architecture (batch-only or streaming-only) is usually the better choice.
