So at the end of 2021 I found myself in the waiting room of an emergency dentist. An infection above my front teeth became unbearable. Fortunately antibiotics makes my live much better now. Let that event not colour my view on 2021. For me 2021 was a great year, despite Read more
The weather is looking good for this weekend. Time to do some cycling after a creative, but also hectic week. Talking about cycling…
You think my 160 km bike ride was crazy? How about a guy who did every kilometer in the Tour de France plus the distances between race starts and finishes (for which the riders take trains, busses and airplanes). A distance of 5500 km and double the altitude gain! That is the Alt Tour that Lachlan Morton rode.
All this with no support team or teammembers to keep him out of the wind. In fact, without hotels even. He has to gather his own food. And his nights of sleep were also less than to be desired. But nevertheless he made it in half the time of the real Tour de France.
Four years now I’ve been working as a data engineer. And when I started writing about how to enter this field (because people sometimes ask me), I found out it’s beter to start writing about what data engineering actually is. Because my view on that has changed. And actually, data engineering changed as well.
Back in 2017, when I made the jump from Oracle database administration, I thought, or was hoping, that a data engineer more or less was a data administrator in Big Data. Sure, it took a bit more programming skills and DevOps and all that, but I thought my experience in operations would largely pay off.
On the other hand, weren’t data engineers supposed to support data scientists, so the data would be prepped for them and they could iterate over this data faster? I found out data engineers exist without data scientists just as well. They provide data to the whole organization, so it can be data driven. Or management at least hopes it will be.