metadata

Don’t do data management just because you have to

Lately more and more organizations are doing data management. Suddenly there are data owners, data stewards and metadata repositories (in whatever form) everywhere. We all seem to do this mainly because we have to. Because of the GDPR or the California Consumer Privacy Act (CCPA). Or because other institutions demand we can explain where our data comes from.

But in my oppinion there is one important reason that mostly is overlooked. One that nevertheless has an important positive impact on business results, but also doesn’t seem to end up in the KPI’s. And that is how much time it takes to find the right data when building data products. (more…)

By Marcel-Jan Krijgsman, ago

Things I’ve learned about metadata for a data lake

I’ve been thinking of writing a blogpost about Apache Atlas. For one and a half years I’ve gained a unique experience with this product that I would like to share with the world.

But first we need to talk about metadata. That is one of the important uses of Apache Atlas. Meaningful metadata won’t get in there by accident. Maybe you are just starting your journey into metadata. I’m here to say that it’s going to take work. Not just by you, but everyone in your organization who has a stake in data. So in this blogpost I will be talking more about the organizational side of metadata and not so much on the technical side.

What do I mean by metadata?

Metadata can mean many things. Search it and you’ll find that there’s metadata used to “get to know you better” by companies, or in other words: for ad targeting. There also is metadata used by intelligence agencies to find out if you plan to do anything bad. But the metadata I’m talking about is the kind of information that you can use to find data in an organization.

(more…)

By Marcel-Jan Krijgsman, ago