Kubernetes Bare-Metal: ETCD Cluster Setup

by Tohain Kuberneteson Posted on

ETCD is a distributed, reliable key-value store for the most critical data of a distributed system.

Bootstrap ETCD Cluster Node

To setup an ETCD cluster node, use the following steps:

  • Install etcd package

    sudo apt-get install etcd

  • Certificates must be already generated

    sudo mkdir /etc/cert
sudo cp ~/cert/ca.crt /etc/cert/
sudo cp ~/cert/etcd-one.crt /etc/cert/
sudo cp ~/cert/etcd-one.key /etc/cert/
sudo cp ~/cert/etcd-one-peer.crt /etc/cert/
sudo cp ~/cert/etcd-one-peer.key /etc/cert/

  • Update etcd configuration in /etc/default/etcd

    --- a/etcd    2023-12-28 21:02:40.673873306 +0700
+++ b/etcd    2023-12-28 21:02:40.673873306 +0700
@@ -11,7 +11,7 @@
## using discovery, each member must have a unique name. Hostname or
## machine-id can be a good choice.
## default: "default"
-# ETCD_NAME="default"
+ETCD_NAME="etcd-one"

##### --data-dir
## Path to the data directory.
@@ -54,7 +54,7 @@
## default: "http://localhost:2380"
## example: "http://10.0.0.1:2380"
## invalid example: "http://example.com:2380" (domain name is invalid for binding)
-# ETCD_LISTEN_PEER_URLS="http://localhost:2380"
+ETCD_LISTEN_PEER_URLS="https://10.0.0.5:2380"

##### --listen-client-urls
## List of URLs to listen on for client traffic. This flag tells the etcd to
@@ -68,7 +68,7 @@
## default: "http://localhost:2379"
## example: "http://10.0.0.1:2379"
## invalid example: "http://example.com:2379" (domain name is invalid for binding)
-# ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
+ETCD_LISTEN_CLIENT_URLS="https://10.0.0.5:2379"

##### --max-snapshots
## Maximum number of snapshot files to retain (0 is unlimited)
@@ -152,7 +106,7 @@
## domain names.
## default: "http://localhost:2380"
## example: "http://example.com:2380, http://10.0.0.1:2380"
-# ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380"
+ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.0.0.5:2380"

##### --initial-cluster
## Initial cluster configuration for bootstrapping.
@@ -160,7 +114,7 @@
## default uses default for the key because this is the default for the
## --name flag.
## default: "default=http://localhost:2380"
-# ETCD_INITIAL_CLUSTER="default=http://localhost:2380"
+ETCD_INITIAL_CLUSTER="etcd-one=https://10.0.0.5:2380,etcd-two=https://10.0.0.6:2380,etcd-three=https://10.0.0.7:2380"

##### --initial-cluster-state
## Initial cluster state ("new" or "existing"). Set to new for all members
@@ -184,7 +138,7 @@
## file descriptors) are eventually depleted.
## default: "http://localhost:2379"
## example: "http://example.com:2379, http://10.0.0.1:2379"
-# ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
+ETCD_ADVERTISE_CLIENT_URLS="https://10.0.0.5:2379"

##### --discovery
## Discovery URL used to bootstrap the cluster.
@@ -276,12 +229,12 @@
##### --cert-file
## Path to the client server TLS cert file.
## default: none
-# ETCD_CERT_FILE
+ETCD_CERT_FILE="/etc/cert/etcd-one.crt"

##### --key-file
## Path to the client server TLS key file.
## default: none
-# ETCD_KEY_FILE
+ETCD_KEY_FILE="/etc/cert/etcd-one.key"

##### --client-cert-auth
## Enable client cert authentication.
@@ -297,7 +250,7 @@
##### --trusted-ca-file
## Path to the client server TLS trusted CA key file.
## default: none
-# ETCD_TRUSTED_CA_FILE
+ETCD_TRUSTED_CA_FILE="/etc/cert/ca.crt"

##### --auto-tls
## Client TLS using generated certificates
@@ -314,12 +267,12 @@
##### --peer-cert-file
## Path to the peer server TLS cert file.
## default: none
-# ETCD_PEER_CERT_FILE
+ETCD_PEER_CERT_FILE="/etc/cert/etcd-one-peer.crt"

##### --peer-key-file
## Path to the peer server TLS key file.
## default: none
-# ETCD_PEER_KEY_FILE
+ETCD_PEER_KEY_FILE="/etc/cert/etcd-one-peer.key"

##### --peer-client-cert-auth
## Enable peer client cert authentication.
@@ -334,7 +287,7 @@
##### --peer-trusted-ca-file
## Path to the peer server TLS trusted CA file.
## default: none
-# ETCD_PEER_TRUSTED_CA_FILE
+ETCD_PEER_TRUSTED_CA_FILE="/etc/cert/ca.crt"

##### --peer-auto-tls
## Peer TLS using generated certificates

  • Restart etcd service

    sudo systemctl restart etcd.service

Verify ETCD Cluster Setup

To check ETCD cluster nodes health, issue:

ETCDCTL_API=3 etcdctl --endpoints=10.0.0.5:2379,10.0.0.6:2379,10.0.0.7:2379 --cacert=/etc/cert/ca.crt endpoint health
10.0.0.5:2379 is healthy: successfully committed proposal: took = 8.512824ms
10.0.0.6:2379 is healthy: successfully committed proposal: took = 14.603578ms
10.0.0.7:2379 is healthy: successfully committed proposal: took = 16.655811ms

What's Next

→ Initialize or upgrade cluster
← Resources preparation

Leave a Reply