Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
Apache: Big Data Europe 2016
Click here to Register or for more information 
View analytic
Wednesday, November 16 • 12:00 - 12:50
Mining and Identifying Security Threat Using Spark SQL, HBase and Solr - Manidipa Mitra, ValueLabs

Sign up or log in to save this to your schedule and see who's attending!

This presentation will talk about how to deisgn a highly effective scalable/performant distributed system to find the identity theft and fraud by mining billions of records related to share holding for a leading financial organization. This will also discuss on how Tera bytes of data can be migrated from Oracle to Hadoop, stored in parquet format, processed in a distributed computing framework with Spark DataFrame and pushed to different service layer (HBase, Impala, Solr, HDFS) depends on the query/access pattern. This design will also throw light on how the frequent transactions were handled and data were pre-processed end of the day to meet the seconds response time SLA, creating thousands of report by mining millions of record in minutes time.

Speakers
avatar for Manidipa Mitra

Manidipa Mitra

Director, ValueLabs
Manidipa Mitra heads the Big Data CoE in ValueLabs having extensive experience in building industry specific solution using distributed computing and cloud technologies . Having 16+ years of software industry experience and in-depth knowledge on disruptive-technologies, Cloud and Storage . She is holding dual graduate degree in Physics and Computer science. Manidipa previously Invited as Speaker in Grace Hopper Conference 2013, and presently... Read More →


Wednesday November 16, 2016 12:00 - 12:50
Giralda III/IV

Attendees (10)