Posts

Showing posts from April, 2014

Simple log analysis with Apache Spark Java API

In this post we will try to redo the sample that we did in my previous post Simple log analysis with Apache Spark, using the Spark JAVA api and since i am more accustomed to maven we will create a simple maven project to accomplish this task. For this post i will not use a separate Spark cluster, we will just use an embedded Spark that will automatically startup a server when running the code.
First lets create a new maven project. I am using Intellij Idea. I will name my project LogAnalizer. The first thing you would need to do is add the maven dependencies to your pom file. To do this add the following lines into the pom.xml file in your newly created Maven project. You will also need to add the repository.


<repositories> <repository> <id>Akka repository</id> <url>http://repo.akka.io/releases</url> </repository> </repositories><dependencies> <dependency> <!-- Spark dependen…