![](https://static.wixstatic.com/media/820ac3_5b480c10d07240abb1211ea00303682a~mv2.jpg/v1/fill/w_62,h_41,al_c,q_80,usm_0.66_1.00_0.01,blur_2,enc_auto/820ac3_5b480c10d07240abb1211ea00303682a~mv2.jpg)
Services: Cloud Configuration Services
Big Data Volumes
Big Data Volumes for Telecom
Telecom is one of the biggest data consuming industries. Using Kafka-based real-time data processing systems can provide Telco’s tremendous benefits in terms of better decision making and revenue generation.
​
For example, from the call detail records (CDRs) alone, huge volumes of data are transacted every day. A Telco needs to process these CDRs real-time on a day-to-day basis.
​
A Kafka-based system allows:
​
-
The CDR files received from the mobile tower is streamed record by record to Kafka.
​
-
Aggregator application aggregates the CDRs based on the MSISDN for Call durations and Mobile Data consumptions.
​
-
Aggregated data CDRs are then streamed back to Kafka.
​
-
It is then stored in Elastic Search.
​
-
Aggregated reports are created and visualised in a Kibana dashboard.
![Kafka-based System for CDR Aggregation](https://static.wixstatic.com/media/820ac3_21a8a8d830d14810a3a7c76317859c00~mv2.png/v1/fill/w_70,h_28,al_c,q_85,usm_0.66_1.00_0.01,blur_2,enc_auto/Big%20data%20Volumes_%20png.png)
Kafka-based System for CDR Aggregation
Technical Insight
Telco Call Data Record Aggregation with huge volumes near real-time:
​
-
To simulate the mobile tower, CDR files are generated by a Spring Boot microservice application and then they are streamed record by record to Kafka using the CDR Kafka Producer.
​
-
Microservice aggregator application written using Apache Flink aggregates the CDRs based on the MSISDN for Call durations and Mobile Data consumptions.
​
-
Aggregated data CDRs is then streamed back to Kafka using Apache Flink producer.
​
-
Kafka Connector is used to read the CDR aggregated data and update the data in Elastic Search.
​
-
The aggregated reports are created and visualised in a Kibana dashboard.
![CDR Aggregation in real-time](https://static.wixstatic.com/media/820ac3_286cf7284162456690263ca7342d0350~mv2.png/v1/fill/w_81,h_45,al_c,q_85,usm_0.66_1.00_0.01,blur_2,enc_auto/Screen%20Shot%202020-03-18%20at%2014_10_31.png)