Category Archives: Data engineering

Tech dossier: pandas

Posted on September 26, 2019 by Marcel-Jan Krijgsman

I’m keeping tech dossiers in Evernote on open source products I want to keep track of.  And I decided to put them on my blog. My previous ones were on Kubernetes and Elasticsearch. This one is on the Python data … Continue reading →

Posted in Data engineering, Python, Tech dossier | Tagged , , , , , | Leave a comment

Things I’ve learned about metadata for a data lake

Posted on September 7, 2019 by Marcel-Jan Krijgsman

I’ve been thinking of writing a blogpost about Apache Atlas. For one and a half years I’ve gained a unique experience with this product that I would like to share with the world. But first we need to talk about … Continue reading →

Posted in Data engineering | Tagged , , , , , , , | Leave a comment

Book review: Spark in Action, 2nd edition

Posted on September 1, 2019 by Marcel-Jan Krijgsman

There are lots of books on Spark, but not a lot that aimed at the data engineer. Data engineers use Spark to ingest and transform data, which is different from what data scientists use it for. On the Roaring Elephant … Continue reading →

Posted in Data engineering, Spark | Tagged , , , | 2 Comments