Tag Archives: Spark

Book review: Spark in Action, 2nd edition

Posted on September 1, 2019 by Marcel-Jan Krijgsman

There are lots of books on Spark, but not a lot that aimed at the data engineer. Data engineers use Spark to ingest and transform data, which is different from what data scientists use it for. On the Roaring Elephant … Continue reading →

Posted in Data engineering, Spark | Tagged , , , | 2 Comments

Dataworks Summit Berlin 2018, day two

Posted on April 21, 2018 by Marcel-Jan Krijgsman

Back for round two of keynotes, good technical sessions and discussing them with fellow data specialists in between. Keynotes First up was  Frank Säuberlich from Teradata, who had an interesting example of machine learning for fraud detection at Danske Bank. … Continue reading →

Posted in Conferences, Events | Tagged , , , , , , , , , , , | Leave a comment

Recovering your HDP 2.6.1 Sandbox on VirtualBox after a restart

Posted on November 17, 2017 by Marcel-Jan Krijgsman

If you’ve worked with the Hortonworks Data Platform 2.x sandbox of later versions in VirtualBox and made it shutdown rather vigorously, you might have noticed that you won’t get past this startup screen when you try to start it up … Continue reading →

Posted in Apache Products for Outsiders, Howto, Learning Big Data | Tagged , , , , , , , , , | 2 Comments

Tutorial: Let’s throw some asteroids in Apache Hive

Posted on November 14, 2017 by Marcel-Jan Krijgsman

This is a tutorial on how to import data (with fixed lenght) in Apache Hive (in Hortonworks Data Platform 2.6.1). The idea is that any non-Hive, non-Hadoop savvy people can follow along, so let me know if I succeeded (make … Continue reading →

Posted in Apache Products for Outsiders, Learning Big Data | Tagged , , , , , , , , | Leave a comment