From Lakehouse to Online: Data Platforms Are Being Re-Architected for Low Latency and Cost Controls
Data platforms are converging toward “always-on” performance for mixed workloads: batch analytics, near-real-time decisions, and online point queries, with aggressive focus on cost-per-query and...
Data infrastructure planning is getting squeezed from two directions at once: product teams want lower-latency access to “all the data,” and finance teams want predictable unit economics. The result is a fast-moving convergence where lake and warehouse stacks are being tuned to serve mixed workloads, including online point queries and AI-driven search, without blowing up cost or ops burden.
Engineering teams are pushing compute and storage knobs harder than they have in years. AWS is positioning EMR Serverless performance gains through bigger workers and shuffle-optimized disks, reporting materially faster TPC-style query runtimes, and also shipping an “upgrade agent” to reduce the migration tax as Spark moves to 4.x (AWS Big Data Blog: EMR Serverless performance, Spark Upgrade Agent). That combination matters: faster runtimes reduce cost and wall-clock, and automation reduces the organizational drag that usually delays platform upgrades.
Warehouse vendors are telling a parallel story, but with cost control as the headline. Snowflake highlights custom incrementalization cutting Dynamic Table auto-clustering costs by 99% for a real customer, and pushes Adaptive Compute as “no manual tuning” price-performance for variable and AI workloads across clouds (Snowflake: Dynamic Tables incrementalization, Adaptive Compute GA). The common theme is a shift from “buy more capacity” to “shape the workload,” with incremental processing, better scheduling, and adaptive resource allocation becoming core architectural primitives.
Practitioner write-ups show where the architecture is heading: online access patterns on top of lake storage. Spotify describes indexing a data lake to serve online point queries at low latency, effectively treating lake data as something that can back user-facing services, not only dashboards (Spotify Engineering). At the same time, application patterns are pulling data platforms closer to AI workloads: AWS demonstrates semantic search plus highlighting for contract compliance in OpenSearch, and Microsoft proposes a three-layer routing architecture for agent traffic on AKS (AWS OpenSearch semantic search; InfoQ on AKS LLM routing). Mixed workloads are no longer theoretical, they are arriving as product requirements.
CTO-level implications land in three places. First, architecture: plan for “two-speed” access to the same governed data, where batch and interactive share lineage but not necessarily the same execution path (indexes, caches, materializations, incremental tables). Second, cost governance: treat cost-per-query (or cost-per-decision) as a platform KPI and invest in mechanisms that reduce unnecessary recompute (incrementalization, shuffle optimization, adaptive scheduling). Third, capacity reality: power constraints are becoming a gating factor for scaling data centres, which increases the premium on efficiency and workload shaping (BBC on power deposits for data centres).
Actionable next steps for engineering leaders: (1) inventory your top 10 expensive queries/jobs and classify which are candidates for incrementalization or materialization, (2) add an explicit “online read path” design review for any initiative that wants lake data in production services, and (3) establish platform SLOs that include both latency and unit cost, then fund the engineering work that moves the curve (upgrade automation, indexing strategies, adaptive compute policies). The teams that win will ship faster decisions at lower marginal cost, even as infrastructure constraints tighten.
Sources
- https://aws.amazon.com/blogs/big-data/accelerate-spark-on-emr-serverless-with-larger-workers-and-shuffle-optimized-disks/
- https://aws.amazon.com/blogs/big-data/automate-spark-scala-migration-to-4-x-with-aws-spark-upgrade-agent/
- https://www.snowflake.com/content/snowflake-site/global/en/blog/thrive-dynamic-table-costs-custom-incremental
- https://www.snowflake.com/en/blog/adaptive-compute-ga-azure-google-cloud/
- https://engineering.atspotify.com/2026/7/indexing-the-data-lake-for-online-point-queries
- https://aws.amazon.com/blogs/big-data/build-a-contract-compliance-search-system-with-amazon-opensearch/
- https://www.infoq.com/news/2026/07/microsoft-agents-aks-routing/
- https://www.bbc.co.uk/news/articles/c9q90q9qnn2o