Recent Comments
Author Archives: Marcel-Jan Krijgsman
Profiling data with ydata in PySpark
When you got a dataset to explore, there are several ways to do that in PySpark. You can do a describe or a summary. But if you want something a little more advanced, and if you want to get a … Continue reading
Posted in Data management, Python, Spark
Tagged data profiling, data quality, PySpark, Python
Leave a comment
My experiences with Azure Purview
At my last customer I have extensively worked with Ataccama, a data management product. It has a data catalog to store metadata on datasets, and it can do data quality checks. In Azure Microsoft has a data management product too. … Continue reading
Posted in Azure, Data engineering, Data management
Tagged azure, data catalog, data management, data quality, purview
Leave a comment
Things I learned about Azure Data Fabric
Currently I’m helping colleagues to read open data in Azure Data Fabric. Here are some of my experiences with it. I don’t want to do an extensive description of what Data Fabric is. In short, if you have an organisational … Continue reading
Posted in Azure, Cloud, Things I Learned
Tagged azure, Data Fabric, Data Factory, Lakehouse, Notebook, Python, Things I Learned
Leave a comment
A great time at PyCon Ireland 2024
I think it was last year when I announced that I wanted to go back to conferences again. Preferably as a speaker. But what conference is the best for data engineers? I couldn’t quite figure it out. Then the call … Continue reading
Using OCR to get data from my Robi scale
How it started For several years I kept track of my weight and fat with a Soehnle Body Balance, which I bought in 2018. That worked quite well until I saw more and more these weird deviations. Take a look … Continue reading
Showing a gift total on a Raspberry Pi with an e-ink display – how hard could it be?
TL;DR: These Python and Raspberry Pi projects. They are fun aren’t they? And often they look deceptively simple. But you don’t see all the projects that failed and usually not where they struggled. This project got stuck (and almost failed) … Continue reading
Posted in Howto, Python
Tagged chromium browser, e-ink display, failure, Python, Raspberry Pi, requests-html, scraping
Leave a comment
How I memorise my lines (and other things) with Anki
In my spare time I do stage acting. And there is almost no better feeling having performed a play really well. But to do so, you need to learn your lines. Preferably you learn your text well in advance, so … Continue reading
Posted in Active Learning, Weird experiments
Tagged Anki, flashcards, learning, Memorisation, spaced repetition, Stage acting
Leave a comment
Categorising text with ChatGPT. Results may be messy.
I have a hobby project I’m working on. It’s an astronomy news feed reader. Long story short: I currently gather links to interesting articles about astronomy by hand. And I want to automate this, so that I have more time … Continue reading
A Strava dashboard on a Raspberry Pi (Part 3): The Strava API
This is part 3 of a series of blogposts on how I created a Strava dashboard on a Inky Impression e-ink display with a Raspberry Pi. OAuth2 This was the part that I expected to be the hard part: getting … Continue reading
Posted in Howto, Python
Tagged API, Graziano Fuccio, Oauth2, Python, strava, stravainky_dashboard.py
7 Comments
A Strava dashboard on a Raspberry Pi (Part 2): Installing software
In last blogpost we set up the Raspberry Pi, attached the Inky Impression display and got the Raspberry Pi ready for remote access. Time to get the Inky Impression software installed and make the Inky Impression screen display something. Your … Continue reading
Posted in Howto, Python
Tagged Inky Impression, Pimoroni, Raspberry Pi, Remote SSH, Visual Studio Code
2 Comments