Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
Apache: Big Data Europe 2016
Click here to Register or for more information 
View analytic
Tuesday, November 15 • 13:00 - 13:50
Elastic Spark Programming Framework - A Dependency Injection Based Programming Framework for Spark Applications - Bruce Kuo

Sign up or log in to save this to your schedule and see who's attending!

Apache Spark is the hottest computing engine nowadays.

More people in big data industry start to use Spark in production systems,

such as machine learning and data ETL applications.

However, developers take quality of code seriously in production systems.

In the past experience, we find there is a gap between development and production.

The difficulties we meet when developing Spark applications in production systems are:

(1) hard to communicate with components,

(2) indirect management of application arguments,

(3) inadequate code maintainability.

To solve these problems and make development smoother, we propose

a dependency-injection-based programming framework on JVM systems.

It provides basic management, monitoring, and better communication mechanisms.

The huge flexibility can help developers writing Spark applications and integrating with components in a better manner.

Speakers
avatar for Bruce Kuo

Bruce Kuo

Software Engineer, Yahoo!
Chun-Ting Kuo (Bruce) works at Yahoo as a data engineer, and he dedicates his work on developing data products and scientific applications. His experience covers Spark, Hadoop, algorithms, and a little machine learrning. When he is free, he loves to code and know novel techniques.


Tuesday November 15, 2016 13:00 - 13:50
Giralda I/II

Attendees (16)