Category Archives: Data engineering

Things I’ve learned about metadata for a data lake

Posted on September 7, 2019 by Marcel-Jan Krijgsman

I’ve been thinking of writing a blogpost about Apache Atlas. For one and a half years I’ve gained a unique experience with this product that I would like to share with the world. But first we need to talk about … Continue reading →

Posted in Data engineering | Tagged , , , , , , , | Leave a comment

Book review: Spark in Action, 2nd edition

Posted on September 1, 2019 by Marcel-Jan Krijgsman

There are lots of books on Spark, but not a lot that aimed at the data engineer. Data engineers use Spark to ingest and transform data, which is different from what data scientists use it for. On the Roaring Elephant … Continue reading →

Posted in Data engineering, Spark | Tagged , , , | 2 Comments