-
Recent Posts
Recent Comments
- Marcel-Jan Krijgsman on Dataworks Summit Berlin 2018, day one
- Wangda Tan on Dataworks Summit Berlin 2018, day one
- Building HDP 2.6 on AWS, Part 3: the worker nodes | Expedition Data on Building HDP 2.6 on AWS, Part 2: the master nodes
- Building HDP 2.6 on AWS, Part 3: the worker nodes | Expedition Data on Building HDP 2.6 on AWS, Part 1: the edge node
- shr on Building HDP 2.6 on AWS, Part 2: the master nodes
Archives
Categories
Meta
- Privacy & Cookies: This site uses cookies. Don't remember for what. You can be assured that I, the author, don't do anything with your personal data, except looking in the stats once in a while if anybody is reading this blog. But some of the plugins might do stuff. Please check the cookie cutter cookie policy. Cookie Policy
Category Archives: Learning Big Data
Making a Hertzsprung-Russell diagram from Gaia DR2 data with Elasticsearch
Elasticsearch was one of the open source products on my list to try out, ever since I got rejected for a couple of assignments as a consultant last year. Apparently it’s a popular product. But why do you need a … Continue reading
Posted in Learning Big Data, NoSQL
Tagged 2001 A Space Odyssey, ElasticSearch, Frank Kane, Gaia, Hertzsprung-Russell diagram, Kibana, Logstash, Udemy, Vega
Leave a comment
Building HDP 2.6 on AWS, Part 3: the worker nodes
This is part 3 in a series on how to build a Hortonworks Data Platform 2.6 cluster on AWS. By now we have an edge node to run Ambari Server, three master nodes for Hadoop name nodes and such. Now … Continue reading
Posted in Howto, Learning Big Data
Tagged Amazon Web Services, AWS, cloning nodes, Hadoop, HDP, Hortonworks Data Platform, Ubuntu Server, worker nodes
Leave a comment
Hadoop High Availability In A Hurry – Part 2: YARN
If you don’t know a lot about YARN and why it’s called a data operating system, you’re in luck. I found it necessary to explain how YARN works before I could explain the solutions for high availability. At first YARN … Continue reading
Posted in Apache Products for Outsiders, Learning Big Data
Tagged Application Master, Container, Hadoop, Node Manager, Resource Manager, YARN, ZooKeeper
1 Comment
Hadoop High Availability In A Hurry – Part 1: HDFS
I’ve been studying for a couple of hours how Hadoop high availability works, for the HDPCA exam. And now I’ve condensed that knowledge to a video on HDFS HA in just under 9 minutes. Enjoy!
Posted in Apache Products for Outsiders, Learning Big Data
Tagged DataNode, edits file, Fencing, fsimage, Hadoop, HDFS, High availability, JournalNode, NameNode, Split brain, ZKFC, ZooKeeper
1 Comment
Certifying as HDP Certified Administrator
Let’s talk about certification. The thing by which you try to show potential employers and customers that you actually know what you are doing at work. My only experience up to last Tuesday with IT product-related certifications was with Oracle’s … Continue reading
Posted in Learning Big Data
Tagged certification, HDPCA, Hortonworks, Hortonworks University
Leave a comment
Building HDP 2.6 on AWS, Part 2: the master nodes
This is part 2 in a series on how to build a Hortonworks Data Platform 2.6 cluster on AWS. In part 1 we created an edge node where we will later install Ambari Server. The next step is creating the … Continue reading
Posted in Howto, Learning Big Data
Tagged Amazon Web Services, AWS, cloning nodes, Hadoop, HDP, Hortonworks Data Platform, master node, Ubuntu Server
5 Comments
Building HDP 2.6 on AWS, Part 1: the edge node
Installing Hortonworks Data Platform 2.6 on Amazon Web Services (Amazon’s cloud platform), how hard could it be? It’s click, click, next, next, confirm, right? Well-lll, not quite. Especially if HDP or AWS is new to you. There are many steps … Continue reading
Posted in Howto, Learning Big Data
Tagged Amazon Web Services, AWS, edge node, Hadoop, HDP, Hortonworks Data Platform, Ubuntu Server
2 Comments