I won’t end this week without a few weekendlinks. This time: SciBabe, an ugly rock from Mars and Have I Been Powned Domain Search.
Yvette d’Entremont, also known as SciBabe, writes daily about either adorable creatures, strange deceases, monstrous chemicals, nuclear f%$k-ups and ways Australian flora and fauna can harm or kill you.
https://scibabe.com/daily-mos-the-ferocious-lemming/
In this blogpost I will reveal something about myself, which I never shared anywhere in my long life on the Internet. It’s rather personal and I don’t like to share a lot on that online. The more because of what the great data aggregators nowadays do with that. But to tell this story, I need to explain what happened to me and how that changed things. Also, I’m fine, really. I do 100+ km bike rides now and this week I rode 60 kms with an average speed of 33.5 km/hr, so that hopefully proves I am currently pretty healthy.
Stories like this always start by saying that when I grew up I was in every aspect a normal kid. But I’m actually not sure :). Anyway at age 11 my parents noticed that I was not growing very much. This fact was underscored by the fact that my 4 year younger brother was getting taller than me: I was about 120 cm (4 feet) at the time. My mother decided this needed to be checked out by a doctor. And before I knew it, I was dragged to a local medical facility where a nurse drew some blood. I can honestly say I hated people poking needles in my arm very much. (more…)
The weather is looking good for this weekend. Time to do some cycling after a creative, but also hectic week. Talking about cycling…
You think my 160 km bike ride was crazy? How about a guy who did every kilometer in the Tour de France plus the distances between race starts and finishes (for which the riders take trains, busses and airplanes). A distance of 5500 km and double the altitude gain! That is the Alt Tour that Lachlan Morton rode.
All this with no support team or teammembers to keep him out of the wind. In fact, without hotels even. He has to gather his own food. And his nights of sleep were also less than to be desired. But nevertheless he made it in half the time of the real Tour de France.
Here is the third weekendlinks edition.
In the previous weekend I rode the Vael Ouwe bike ride. We had lots of luck with the weather. My video, made with my Sony actioncam, turned out really nice. The ride through the heather with wildflowers was beautiful and at the halfway point we rode past Radio Kootwijk, a large art deco style building that has been used for radio transmission. Near the end we climbed the Posbank twice. You can find details of the route here.
This is my second Weekendlink post with interesting links for you to enjoy in the weekend.
I’ve written many years ago on my old Oracle blog how important it is to be able to give good, clear presentations. Garr Reynolds’s Presentation Zen blog and book have been very influential on my presentation techniques. I was wondering what he was up to nowadays, apart from the 3rd edition of his Presentation Zen book. Turns out he has some good advise for your next online presentation on Zoom or WebEx.
Back when I worked for Transfer Solutions we had an internal weekly newsletter that appeared every end of the week. At the end of it there was a small section by me called “weekendlinks”: 3 or 4 things I found on the Internet that were interesting, funny or thoughtprovoking for people to look at on the Friday afternoon. When I left Transfer Solutions I’ve been on the lookout for a place for my weekendlinks, but either the company where I worked didn’t have a similar newsletter, the culture was too serious or “not a good fit” otherwise.
But since this is my blog and I might just as well do as I please here, I decided to revive my weekendlinks here. It will be my Five Bullet Friday. So here goes:
Teaching complex matters to people is an art. Explaining modern software concepts to children is even more a challenge. But there are people who try it. There already was a Children’s Illustrated Guide to Kubernetes. Well, now there’s one for Apache Kafka too. Expect some tortured analogies though.
https://www.gentlydownthe.stream/
Maybe I should write a children’s book to data engineering one day?
Four years now I’ve been working as a data engineer. And when I started writing about how to enter this field (because people sometimes ask me), I found out it’s beter to start writing about what data engineering actually is. Because my view on that has changed. And actually, data engineering changed as well.
Back in 2017, when I made the jump from Oracle database administration, I thought, or was hoping, that a data engineer more or less was a data administrator in Big Data. Sure, it took a bit more programming skills and DevOps and all that, but I thought my experience in operations would largely pay off.
On the other hand, weren’t data engineers supposed to support data scientists, so the data would be prepped for them and they could iterate over this data faster? I found out data engineers exist without data scientists just as well. They provide data to the whole organization, so it can be data driven. Or management at least hopes it will be.
Lately more and more organizations are doing data management. Suddenly there are data owners, data stewards and metadata repositories (in whatever form) everywhere. We all seem to do this mainly because we have to. Because of the GDPR or the California Consumer Privacy Act (CCPA). Or because other institutions demand we can explain where our data comes from.
But in my oppinion there is one important reason that mostly is overlooked. One that nevertheless has an important positive impact on business results, but also doesn’t seem to end up in the KPI’s. And that is how much time it takes to find the right data when building data products. (more…)
TL;DR: I made a Docker compose that runs Hadoop, Spark and Hive in a multi-container environment. You can find the necessary files for it here: https://github.com/Marcel-Jan/docker-hadoop-spark [Update 2021-11-09: Since Docker Desktop turned “Expose daemon on tcp://localhost:2375 without TLS” off by default there have been all kinds of connection problems running the complete docker-compose. Turning this option on again (Settings > General > Expose daemon on tcp://localhost:2375 without TLS) makes it all work. I’m still looking Read more
I was looking for a way to detect leakage in my appartement with some kind of IoT solution. Someone on the Dutch technology forum Tweakers.net told me Xiaomi Humidity sensors, combined with a Zigbee2MQTT might be a good fit. The sensors are quite cheap and so is the CC2531 sniffer stick to receive the data sent over the Zigbee protocol. So that’s what I set out to do. And in these two videos you see Read more