Category Archives: Data engineering

Digging into video files for geolocations

Posted on May 21, 2022 by Marcel-Jan Krijgsman

So far I’ve found geolocations in XML metadata that my actioncam stores on disk as seperate .XML files and I’ve found them in JPG files. When I showed the cool maps I made to my father, he asked if I … Continue reading →

Posted in Data engineering, Howto, Python | Tagged , , , , , | Leave a comment

Photo locations, marker icons and displaying photos on my map

Posted on May 11, 2022 by Marcel-Jan Krijgsman

When I was finished last week with creating my video location map in Python, I thought “shame I can’t plot photo locations”. That’s because my Fuji X-T30 camera doesn’t store GPS info. When I bought the camera I assumed every … Continue reading →

Posted in Data engineering, Howto, Python | Tagged , , , , , , , | Leave a comment

Making my video location map even better with Folium

Posted on May 4, 2022 by Marcel-Jan Krijgsman

Yesterday I shared how I plotted locations of videos shot with my Sony FDR-X3000 camera on a map. I was already pretty happy. Then I got a tip from Twitter user Bob Haffner (@bobhaffner): why not use Folium to create … Continue reading →

Posted in Data engineering, Howto, Python | Tagged , , , | Leave a comment

Plotting video locations from my Sony camera in Python

Posted on May 3, 2022 by Marcel-Jan Krijgsman

Two years ago I bought a Sony FDR-X3000 actioncam to record video on my bike rides. And I’m really happy about it. It’s just great reliving my rides in 4K, going downhill for kilometers from some col I climbed. I … Continue reading →

Posted in Data engineering, Howto, Python | Tagged , , , , , , | Leave a comment

My Github repo got 50 stars

Posted on March 19, 2022 by Marcel-Jan Krijgsman

I never imagined myself as a maintainer of a data engineering related open source thing. Yet. But when I was working on our data engineering course, I needed some kind of data lake software. At first I used the Cloudera … Continue reading →

Posted in Apache Products for Outsiders, Data engineering, Learning Big Data | Tagged , , , , | Leave a comment

Five years of data engineering

Posted on March 1, 2022 by Marcel-Jan Krijgsman

Five years ago I made the switch from Oracle database administration to data engineering. It has been quite a ride. I made a video about this to celebrate.

Posted in Active Learning, Data engineering, Learning Big Data | Tagged | Leave a comment

What a year 2021 has been

Posted on January 2, 2022 by Marcel-Jan Krijgsman

So at the end of 2021 I found myself in the waiting room of an emergency dentist. An infection above my front teeth became unbearable. Fortunately antibiotics makes my live much better now. Let that event not colour my view … Continue reading →

Posted in Active Learning, Data engineering | Tagged , , , , , , , | Leave a comment

What I think data engineering is (revisited)

Posted on May 19, 2021 by Marcel-Jan Krijgsman

Four years now I’ve been working as a data engineer. And when I started writing about how to enter this field (because people sometimes ask me), I found out it’s beter to start writing about what data engineering actually is. … Continue reading →

Posted in Data engineering | Tagged , | Leave a comment

Don’t do data management just because you have to

Posted on February 3, 2021 by Marcel-Jan Krijgsman

Lately more and more organizations are doing data management. Suddenly there are data owners, data stewards and metadata repositories (in whatever form) everywhere. We all seem to do this mainly because we have to. Because of the GDPR or the … Continue reading →

Posted in Data engineering | Tagged , , , , , | Leave a comment

Tech dossier: pandas

Posted on September 26, 2019 by Marcel-Jan Krijgsman

I’m keeping tech dossiers in Evernote on open source products I want to keep track of.  And I decided to put them on my blog. My previous ones were on Kubernetes and Elasticsearch. This one is on the Python data … Continue reading →

Posted in Data engineering, Python, Tech dossier | Tagged , , , , , | Leave a comment