{"id":29060,"date":"2015-10-28T00:15:49","date_gmt":"2015-10-27T18:45:49","guid":{"rendered":"https:\/\/2thenew.today\/blog\/?p=29060"},"modified":"2016-01-19T13:29:02","modified_gmt":"2016-01-19T07:59:02","slug":"zookeeper-leader-election-simplified","status":"publish","type":"post","link":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/","title":{"rendered":"ZooKeeper Leader-Election simplified"},"content":{"rendered":"<h4>Background<\/h4>\n<p>Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility &#8211; so that the service remains available.<br \/>\nSo, the use-case was something like &#8211; Only single node should behave as a master node and it will coordinate with all the worker nodes to process the required tasks.<\/p>\n<p>Clearly, it\u2019s a leader-election recipe. This is supported in Curator\/Zookeeper API, but we found it little complex in terms of blocking the threads to claim the leadership for a longer duration. Otherwise, it was not functioning well.<\/p>\n<h3>Solution<\/h3>\n<p>So, we figured out a very simple way to determine the master node. Here are the steps below:<\/p>\n<ul>\n<li>All the nodes will register themselves to a specific ZK path. Let\u2019s say it\u2019s \u201c\/my\/project\/coordinators\/${machine-ip-address}\u201c<\/li>\n<li>Example: If Node1 (running on IP 127.0.0.1) joins the cluster &#8211; the ZK node path will appear like \u201cmy\/project\/coordinators\/127.0.0.1\/\u201c. We picked IP address, but it\u2019s up to you &#8211; whatever convention you like to follow.<\/li>\n<li>Write an Algo which determines the leader\/master node\n<ul>\n<li>Every coordinator node will read all the children node names (i.e. list of IPs).<\/li>\n<li>Sort the node-names and pick the very first one &#8211; Declare it as master node.<\/li>\n<li>Below is the code snippet to determine if the node is master node or not\n<p>[code]<br \/>\npublic static boolean isMasterCoordinatorNode(String nodeId) throws Exception {<br \/>\n        List&lt;String&gt; coordinatorNodes = getCuratorClient().getChildren().forPath(&quot;my\/project\/coordinators&quot;);<br \/>\n        if (coordinatorNodes!=null &amp;&amp; coordinatorNodes.size()&gt;0)<br \/>\n        {<br \/>\n            TreeSet&lt;String&gt; set = new TreeSet&lt;String&gt;(coordinatorNodes);<br \/>\n            String firstNodeId = set.first();<br \/>\n            if(firstNodeId.equals(nodeId)){<br \/>\n                return true;<br \/>\n            }<br \/>\n        }<br \/>\n        return false;<br \/>\n    }<\/p>\n<p>public synchronized static CuratorFramework getCuratorClient(){<br \/>\n        if(_client == null){<br \/>\n            String zookeeperStr = &quot;127.0.0.1:2181&quot;; \/\/ zookeeper address<br \/>\n            RetryPolicy retryPolicy = new ExponentialBackoffRetry(1000, 3);<br \/>\n            _client = CuratorFrameworkFactory.newClient(zookeeperStr, retryPolicy);<br \/>\n            _client.start();<br \/>\n        }<br \/>\n        return _client;<br \/>\n    }<br \/>\n[\/code]<\/p>\n<\/li>\n<\/ul>\n<\/li>\n<li>Other coordinator nodes (which are not picked as master node) &#8211; won\u2019t do anything.<\/li>\n<li><strong>Ensure that the registered ZooKeeper nodes are all ephemeral nodes<\/strong>, so that even if the master node goes down, the immediate next (available) node will become the master node.<\/li>\n<\/ul>\n<p>In our case this worked very well. \ud83d\ude42<\/p>\n<p><em>Here&#8217;s my previous blog as an introduction to Curator framework<br \/>\n<a href=\"https:\/\/2thenew.today\/blog\/curator-framework-for-apache-zookeeper\/\">Curator Framework for Apache ZooKeeper<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility &#8211; so that the service remains available. So, the use-case was something like &#8211; Only single node should behave as a [&hellip;]<\/p>\n","protected":false},"author":606,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":33,"footnotes":""},"categories":[1395,446],"tags":[2664,2663,2669,2668],"class_list":["post-29060","post","type-post","status-publish","format-standard","hentry","category-big-data","category-java","tag-apache-curator","tag-apache-zookeeper","tag-cluster-management","tag-leader-election"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Salil Kumar\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"TO THE NEW BLOG\" \/>\n\t\t<meta property=\"og:type\" content=\"blog\" \/>\n\t\t<meta property=\"og:title\" content=\"ZooKeeper Leader-Election simplified | TO THE NEW Blog\" \/>\n\t\t<meta property=\"og:description\" content=\"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/2thenew.today\/blog\/wp-content\/themes\/ttn\/images\/social-logo.png\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/2thenew.today\/blog\/wp-content\/themes\/ttn\/images\/social-logo.png\" \/>\n\t\t<meta name=\"twitter:card\" content=\"summary\" \/>\n\t\t<meta name=\"twitter:site\" content=\"@tothenew\" \/>\n\t\t<meta name=\"twitter:title\" content=\"ZooKeeper Leader-Election simplified | TO THE NEW Blog\" \/>\n\t\t<meta name=\"twitter:description\" content=\"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a\" \/>\n\t\t<meta name=\"twitter:image\" content=\"https:\/\/2thenew.today\/blog\/wp-content\/themes\/ttn\/images\/social-logo.png\" \/>\n\t\t<script type=\"application\/ld+json\" class=\"aioseo-schema\">\n\t\t\t{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#article\",\"name\":\"ZooKeeper Leader-Election simplified | TO THE NEW Blog\",\"headline\":\"ZooKeeper Leader-Election simplified\",\"author\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/author\\\/salil-3\\\/#author\"},\"publisher\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/#organization\"},\"datePublished\":\"2015-10-28T00:15:49+05:30\",\"dateModified\":\"2016-01-19T13:29:02+05:30\",\"inLanguage\":\"en-US\",\"commentCount\":1,\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#webpage\"},\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#webpage\"},\"articleSection\":\"Big Data, Java\\\/JVM, Apache Curator, Apache ZooKeeper, Cluster Management, Leader-Election\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#breadcrumblist\",\"itemListElement\":[{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog#listItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.tothenew.com\\\/blog\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/category\\\/java\\\/#listItem\",\"name\":\"Java\\\/JVM\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/category\\\/java\\\/#listItem\",\"position\":2,\"name\":\"Java\\\/JVM\",\"item\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/category\\\/java\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#listItem\",\"name\":\"ZooKeeper Leader-Election simplified\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog#listItem\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#listItem\",\"position\":3,\"name\":\"ZooKeeper Leader-Election simplified\",\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/category\\\/java\\\/#listItem\",\"name\":\"Java\\\/JVM\"}}]},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/#organization\",\"name\":\"TO THE NEW Blog\",\"url\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/author\\\/salil-3\\\/#author\",\"url\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/author\\\/salil-3\\\/\",\"name\":\"Salil Kumar\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#authorImage\",\"url\":\"https:\\\/\\\/newersworld-sf-static.tothenew.net\\\/prod\\\/profilePicFolder\\\/487d6b33-0950-4f6b-b106-a51c27f3e8ca_salil.kumar@tothenew.com.jpg\",\"width\":96,\"height\":96,\"caption\":\"Salil Kumar\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#webpage\",\"url\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/\",\"name\":\"ZooKeeper Leader-Election simplified | TO THE NEW Blog\",\"description\":\"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a\",\"inLanguage\":\"en-US\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/#website\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/zookeeper-leader-election-simplified\\\/#breadcrumblist\"},\"author\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/author\\\/salil-3\\\/#author\"},\"creator\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/author\\\/salil-3\\\/#author\"},\"datePublished\":\"2015-10-28T00:15:49+05:30\",\"dateModified\":\"2016-01-19T13:29:02+05:30\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/\",\"name\":\"TO THE NEW Blog\",\"inLanguage\":\"en-US\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.tothenew.com\\\/blog\\\/#organization\"}}]}\n\t\t<\/script>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"ZooKeeper Leader-Election simplified | TO THE NEW Blog","description":"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a","canonical_url":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#article","name":"ZooKeeper Leader-Election simplified | TO THE NEW Blog","headline":"ZooKeeper Leader-Election simplified","author":{"@id":"https:\/\/2thenew.today\/blog\/author\/salil-3\/#author"},"publisher":{"@id":"https:\/\/2thenew.today\/blog\/#organization"},"datePublished":"2015-10-28T00:15:49+05:30","dateModified":"2016-01-19T13:29:02+05:30","inLanguage":"en-US","commentCount":1,"mainEntityOfPage":{"@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#webpage"},"isPartOf":{"@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#webpage"},"articleSection":"Big Data, Java\/JVM, Apache Curator, Apache ZooKeeper, Cluster Management, Leader-Election"},{"@type":"BreadcrumbList","@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#breadcrumblist","itemListElement":[{"@type":"ListItem","@id":"https:\/\/2thenew.today\/blog#listItem","position":1,"name":"Home","item":"https:\/\/2thenew.today\/blog","nextItem":{"@type":"ListItem","@id":"https:\/\/2thenew.today\/blog\/category\/java\/#listItem","name":"Java\/JVM"}},{"@type":"ListItem","@id":"https:\/\/2thenew.today\/blog\/category\/java\/#listItem","position":2,"name":"Java\/JVM","item":"https:\/\/2thenew.today\/blog\/category\/java\/","nextItem":{"@type":"ListItem","@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#listItem","name":"ZooKeeper Leader-Election simplified"},"previousItem":{"@type":"ListItem","@id":"https:\/\/2thenew.today\/blog#listItem","name":"Home"}},{"@type":"ListItem","@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#listItem","position":3,"name":"ZooKeeper Leader-Election simplified","previousItem":{"@type":"ListItem","@id":"https:\/\/2thenew.today\/blog\/category\/java\/#listItem","name":"Java\/JVM"}}]},{"@type":"Organization","@id":"https:\/\/2thenew.today\/blog\/#organization","name":"TO THE NEW Blog","url":"https:\/\/2thenew.today\/blog\/"},{"@type":"Person","@id":"https:\/\/2thenew.today\/blog\/author\/salil-3\/#author","url":"https:\/\/2thenew.today\/blog\/author\/salil-3\/","name":"Salil Kumar","image":{"@type":"ImageObject","@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#authorImage","url":"https:\/\/newersworld-sf-static.tothenew.net\/prod\/profilePicFolder\/487d6b33-0950-4f6b-b106-a51c27f3e8ca_salil.kumar@tothenew.com.jpg","width":96,"height":96,"caption":"Salil Kumar"}},{"@type":"WebPage","@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#webpage","url":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/","name":"ZooKeeper Leader-Election simplified | TO THE NEW Blog","description":"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a","inLanguage":"en-US","isPartOf":{"@id":"https:\/\/2thenew.today\/blog\/#website"},"breadcrumb":{"@id":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/#breadcrumblist"},"author":{"@id":"https:\/\/2thenew.today\/blog\/author\/salil-3\/#author"},"creator":{"@id":"https:\/\/2thenew.today\/blog\/author\/salil-3\/#author"},"datePublished":"2015-10-28T00:15:49+05:30","dateModified":"2016-01-19T13:29:02+05:30"},{"@type":"WebSite","@id":"https:\/\/2thenew.today\/blog\/#website","url":"https:\/\/2thenew.today\/blog\/","name":"TO THE NEW Blog","inLanguage":"en-US","publisher":{"@id":"https:\/\/2thenew.today\/blog\/#organization"}}]},"og:locale":"en_US","og:site_name":"TO THE NEW BLOG","og:type":"blog","og:title":"ZooKeeper Leader-Election simplified | TO THE NEW Blog","og:description":"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a","og:url":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/","og:image":"https:\/\/2thenew.today\/blog\/wp-content\/themes\/ttn\/images\/social-logo.png","og:image:secure_url":"https:\/\/2thenew.today\/blog\/wp-content\/themes\/ttn\/images\/social-logo.png","twitter:card":"summary","twitter:site":"@tothenew","twitter:title":"ZooKeeper Leader-Election simplified | TO THE NEW Blog","twitter:description":"Background Recently (in a project) we were required to determine the master node from a pool of similar type of nodes. And if master node fails, any other node should take on the responsibility - so that the service remains available. So, the use-case was something like - Only single node should behave as a","twitter:image":"https:\/\/2thenew.today\/blog\/wp-content\/themes\/ttn\/images\/social-logo.png"},"aioseo_meta_data":{"post_id":"29060","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":"","og_description":"","og_object_type":"blog","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":"","og_custom_url":null,"og_article_section":"","og_article_tags":"","twitter_use_og":false,"twitter_card":"summary","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"Article","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"limit_modified_date":false,"created":"2021-04-30 06:38:00","updated":"2024-02-29 10:01:04","focus_keyword":null,"additional_keywords":null,"truseo_locale":null,"ai":null,"breadcrumb_settings":null,"seo_analyzer_scan_date":null},"aioseo_breadcrumb":"<div class=\"aioseo-breadcrumbs\"><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/2thenew.today\/blog\" title=\"Home\">Home<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/2thenew.today\/blog\/category\/java\/\" title=\"Java\/JVM\">Java\/JVM<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\tZooKeeper Leader-Election simplified\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/2thenew.today\/blog"},{"label":"Java\/JVM","link":"https:\/\/2thenew.today\/blog\/category\/java\/"},{"label":"ZooKeeper Leader-Election simplified","link":"https:\/\/2thenew.today\/blog\/zookeeper-leader-election-simplified\/"}],"_links":{"self":[{"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/posts\/29060","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/users\/606"}],"replies":[{"embeddable":true,"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/comments?post=29060"}],"version-history":[{"count":0,"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/posts\/29060\/revisions"}],"wp:attachment":[{"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/media?parent=29060"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/categories?post=29060"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/2thenew.today\/blog\/wp-json\/wp\/v2\/tags?post=29060"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}