MicroServices
New Data Pipelines Based on Confluent Cloud
Situation
Accelerate the drug discovery process
by streamlining the analysis of biological
image data
Task
Use Confluent Cloud and Kafka to create a scalable, highly reliable data pipeline based on real-time event
streaming
Action
Integrate Confluent Cloud and Apache Kafka to build an event-driven microservices architecture to minimize administrative overhead, enable faster iterations on experimental results and simplify migration by reusing existing microservices.
Result
Drug discovery pipeline
stages accelerated
• Flexible, highly available data pipeline established
• Stable operation in production since launch
DriveCentric Builds Next-Generation Architecture
for CRM with Confluent
Situation
Enable continued rapid growth by addressing the scalability limitations of a legacy multilayered web application for
automotive CRM
Task
Use Confluent to transition to a scalable microservices-based architecture
oriented around data in motion
Action
DriveCentric is transitioning from its legacy architecture with a phased approach in which teams replace functionality in the core system with new microservices.
Result
• Scalability for ongoing
growth ensured
• Peak workloads handled seamlessly
• Focus on delivering value maintained
• Development and production support simplified
Event-Driven Microservices Architecture
Situation
Drive a digital transformation at the
the largest bank in Indonesia to improve
the bank’s market position and increase financial inclusion across the country
Task
Move from synchronous to asynchronous microservices development on an enterprise-ready platform. Enable stream processing for real-time data
processing in flight
Action
Use Confluent Platform and Apache
Kafka to deploy an event-driven
microservices architecture that powers
big data analytics for real-time credit
scoring, fraud detection, and merchant
assessment services
Result
• Fraud detection performed
in real-time
• Loan disbursement times cut from 2 weeks to 2 minutes
• ISO-certified open API created
• Loan defaults predicted proactively; NPL at 0%
Real-time Data pipelines using Kafka
Trading Platform
Situation
Develop a new trading platform that supports high-volume, high-speed trading and provides clients with access to real-time data.
Task
Use Confluent Platform to implement a reliable, scalable persistence layer for market orders that supports millisecond latencies and billions of messages per day.
Action
Replace the existing trading infrastructure with the Kafka, use Kafka Streams API library to perform data enrichment
in real-time.
Result
• Reliable 24/5 operations achieved and maintained
• Stringent performance requirements exceeded
• Dedicated, expert support received
Driving Modernization and Transformation with an Event-Based Architecture
Situation
Power a modernization and digitization initiative to respond to increasing customer expectations and regulatory pressure in the insurance industry
Task
Build and deploy a new IT architecture based on event streaming with Confluent Platform
Action
Image
Result
•An entirely new enterprise-scale event-streaming platform rolled out in one year
•IT teams empowered to do their jobs faster and with higher efficiency
•Data replication and streaming times reduced from hours or days to just seconds
Hyper personalized online Experience
Situation
Maximize customer satisfaction and
revenue growth by creating a hyper-personalized online retail experience, turning each customer visit into a one-on-one marketing opportunity.
Task
Use Confluent to combine historical
customer data with real-time digital signals from customers, generating hyper-personalized content – for example, targeted special offers – which is inserted in real-time back into the customer’s session.
Action
Image
Result
•Real-time hyper personalization
of the customer experience
•Increased customer conversions
•Accelerated innovation
Confluent Cloud frees up
developers’ time
Real-Time Analytics (with Imply)
Combating fraud
Situation
Most analytics systems respond too slowly to support quick interactive investigation and timely disposition of flagged transactions. Slow response leads to lower productivity for fraud teams and a longer time to resolution for each case.
Task
Integrate Imply with Event Driven data, Follow Data Access in Real-Time (DART) architecture. Replace slow queries by Pivot’s Data Cubes.
Action
Imply with Event-driven architecture gives the applications the ability to respond to changes in the environment in real-time, and it powers our fraud prevention services.
Result
Slow queries and syntax struggles have been replaced by Imply Pivot’s Data Cubes. Instead of obsolete reports and dispersed data, we now have a highly interactive real-time dashboard that incorporates both internal customer interaction events and data from each of our vendors. With this powerful tool, the team is armed with a holistic view of both fraud and our application ecosystem.
Real-time analytics
Situation
Data volume is increasing and the current solution not able to handle huge amount of data. Existing solution did not have a concept of deep storage. Data was tied to each running instance and attempts at using non-local storage resulted in significant impacts on cost and performance.
Task
Integrate Imply for real time analytics, a platform for scalability and customer usage.
Model hot, warm, and cold data Reduce the number of downstream manipulations required.
Periodically assess, review, and reestablish trust in your platform and data
Action
Migration of a core component of our network analytics to the Imply platform.
Result
Imply has led to an increase in our ‘average’ data temperature. More of our data has shifted into the ‘hot’ (0.1 – 3s, recent, highly concurrent, highly interactive) and ‘warm’ (5 – 30s, less recent, highly concurrent, some interactivity) zones.
Digital advertising analytics
Situation
MoPub, a Twitter company, provides monetization solutions for mobile app publishers and developers around the globe. MoPub operates at a massive scale — Over 1.7 billion monthly unique devices, 1T+ trillion monthly ad requests, 52,000+ apps, and 180+ demand side partners on our platform. Our customers, partners, and internal teams depend on fast, easy access to our data to answer business questions, troubleshoot issues and optimize revenue
Task
Bring the power of Apache Druid in MoPub Analytics. Apache Druid (incubating) provides a set of capabilities that borrow from OLAP, time series, and search systems. It can achieve low latency (seconds) for queries at the expense of some query flexibility.
Action
Image
Result
Performanced drastically improved and Querying Terabytes of Data in Seconds is possible.
Monitoring
Detecting Malicious Activity in Real Time
Situation
Identifying fraudulent and malicious activity is crucial when it comes to meeting service level objectives. Users were creating several accounts from different IP addresses in order to receive multiple vouchers.
Task
Facilitate comprehensive security analysis of malicious patterns across the entire tech stack.
Action
Integrate Datadog Security Monitoring tools. Apply flexible detection rules to identify a wide range of attacker techniques in real time.
Result
By using Datadog, PedidosYa eliminated the month-long detection delay that was hampering their efforts to curb fraudulent activity in their large-scale system. After implementing Datadog Security Monitoring
Transaction Monitoring for Anti-Money Laundering (AML)
Situation
Anti-Money Laundering (AML) is a comprehensive set of processes, regulations and rules that together cohesively combat money laundering, terrorism funding and other financial crimes such as identity fraud.
Task
Build a solution on Apache Druid to handle the AML investigation for the compliance team. The AML (anti-money laundering) workflow generates alerts which can be tracked within Druid.
Action
The transactional data is ingested from RDBMS to S3 and ingested back to Druid at regular intervals. Investigators can now slice and dice over millions of data with low latency.
Result
Gererate accurate reports of new and emerging risks for regulators and executives. Evaluate transactions in real time for any suspicious.
Agility and Scale with Manged Cloud Set-up and Services
AI for Secure Cryptocurrency Exchange
Situation
Coinbase works in Cryptocurrencies. They are growing faster then ever. We need to provide a seamless experience for consumers while taking steps to secure the environment.
Task
Secure the application and integrate Artificial intelligence (AI) using machine learning tools from Amazon Web Services (AWS).
Action
Coinbase uses Amazon SageMaker to develop machine learning algorithms for image analysis to defeat scammers. Engineers at Coinbase developed a machine learning-driven system that recognizes mismatches and anomalies in sources of user identification, allowing them to quickly take action against potential sources of fraud.
Result
SegaMaker with AI helps us to balance risks for Coinbase. Face identification helps us to identify scammers.
Virtual Car Launch on AWS
Situation
Even before the COVID-19 pandemic temporarily closed dealerships worldwide, the average car-shopping experience was trending from traditional showrooms to the internet: the average number of times a car buyer visits a dealership before a purchase has dropped from 7 to 1.5 in the past decade
Task
We needed to bring that physical machine to the cloud and keep the power needed to serve high-quality content. Automotive visualization software specialist ZeroLight offers SpotLight Suite, a cloud-based platform that brands, agencies, and dealers use to customize sales and marketing to each shopper
Action
To offer customers a seamless experience, ZeroLight needs readily accessible compute power—so it turned to Amazon Web Services (AWS), which offers globally available GPU instances, low-latency content-delivery tools, and a large selection of advanced artificial intelligence services to help marketers find and engage with customers. ZeroLight implemented Amazon Elastic Compute Cloud (Amazon EC2) G4 Instances powered by NVIDIA T4 Tensor Core GPUs.
Result
Shoppers can configure the car to meet their preferences using ZeroLight’s Palette+, powered by Amazon EC2 G4 Instances. When visitors reach the Lucid website, AWS needs just 5 seconds to find their location across the United States, Europe, or the United Arab Emirates; trigger the engine on ZeroLight; begin 3D streaming; and deliver the first live image. Each session is assigned a dedicated EC2 instance, enabling Lucid to deliver immersive, 360-degree visualizations
Abacus Medicine sustains its rapid growth with cloud solutions
Situation
Distrubutor of affordable medicine aims to quickly provide better healthcare to customers.
Task
We implemented Dynamics 365 Finance and Operations in the cloud, as it could scale up and down with us very fast. We could also stick to the platform’s design and customize not more than 15 percent of the system.”
Action
Employees handling the warehousing, finance, sales, and production can input data on Dynamics 365 or applications built on Power Apps, and managers can view real-time business reports on Power BI
Result
Dynamics 365 and Microsoft Power Platform support. Thanks to immediate information flow across departments and countries, management has real-time insights and can make data-driven decisions. Customers, in turn, receive correct shipments and have quicker financial transactions.”