Client overview
- Industry focus
- FinTech analytics
- Portfolio segment
- SaaS / Enterprise
- Organization profile
- Embedded analytics arm inside payments infra provider
Customers demanded intra-minute fraud and authorization insights during events like singles day sales. Legacy OLAP cube nightly loads insulted SaaS positioning. Security reviews insisted cell-level tenant isolation without duplicating clusters per customer.
Problem
Batch warehouse cadence could not answer operational questions; semantic drift broke trust when metrics conflicted across teams.
Dashboard SQL authored directly against raw topics produced five versions of "approval rate." Analysts exported CSVs defeating product stickiness.
Pinpointing anomalies required correlating card bins, merchant categories, and geo — queries timed out on cold aggregates.
SOC2 auditors asked how tenant isolation maps to storage layout — docs were aspirational.
Solution
Streaming medallion layers into Pinot/StarRocks-class OLAP, dbt-modeled semantic metrics, Looker/embedded SDK with signed embed tokens, row/column policies, and proactive cache warming for top slices.
Kafka streams cleaned and enriched events with deterministic late-arrival windows using watermarks aligned to acquirer settlement cycles.
Semantic layer checked in Git with CI verifying metric formulas against golden datasets; breaking changes triggered consumer notifications.
Pinot segment placement enforced tenant keys; SSD tiers pinned hot merchants during campaigns.
Implementation
- 1
Metric civil war resolution
Facilitated workshops to retire tribal definitions; published glossary with ownership RACI.
- 2
Streaming correctness
Exactly-once semantics where finance needed; approximate windows labeled honestly for ops dashboards.
- 3
Customer-facing perf
Cache policies keyed by dashboard templates; prefetch based on navigation graphs from product analytics.
Tools & platforms
- Kafka
- Flink
- Apache Pinot
- dbt
- Cube.dev semantic layer
- Embedded analytics SDK
Engineering challenges addressed
- Balancing freshness vs. cost — tiered query modes exposed honestly in UI.
Program artifacts & environments
Tech stack
- Kafka
- Flink
- Apache Pinot
- dbt
- React
- Node.js
- Kubernetes
- AWS
- Protobuf
Results
- p99 dashboard query latency 840ms at 10× prior event volume
- Semantic conflict tickets down 73% quarter over quarter
- Embedded analytics attach rate +22 points among enterprise merchants
Quantified impact
840ms p99 query latency
Pinned dashboards during synthetic peak replay.
+22 pts attach rate
Measured from product-led expansion cohort.
Key takeaways
- Real-time analytics products sell definitions first — velocity second.
- Semantic CI prevents dashboard debt from metastasizing silently.
- Tenant isolation stories must be boringly concrete for enterprise procurement.