Gateways allow secure communication between clusters, they circumvent the requirement for a flat network which all services are running on and can be even run on the public internet to facilitate cloud to cloud communication.
In this example we have two services:
- Web (onprem)
- API (cloud)
Web service is running in a VM in the
onprem datacenter and needs to communicate with the
API service which is running in Kubernetes in the
cloud datacenter. The network which the
onprem datacenter runs in is not accessible from the
cloud datacenter’s network.
To enable traffic to flow between the two datacenters we are going to deploy a Consul Gateway into both Datacenters. The Gateway will bridge the internal network and the wan network allowing traffic to be transparently proxied between datacenters.
examples/gateways/docker-compose.yml expands on what we were using in the previous example. We have added the same web service which was used in the first example. If you examine the configuration
examples/gateways/server_config/web.hcl you will also see that this is exactly the same as the first example. Unlike the first example this time the
API service is going to be hosted separate datacenter and in a Kubernetes cluster.
To enable cross-datacenter service mesh traffic we use a Gateway, a Gateway is a bridge between two networks. To run a Gateway we can use a very similar technique to running the data plane. That is the
consul connect envoy command. Gateways like Data plane proxies are just Envoy instances which have been specifically configured by Consul.
For our Web service to communicate to the API it will still use the data plane (Envoy), however; rather than the data plane talking direct to the API data plane in the other cluster it will talk to its local gateway. The gateway can bridge two networks and will forward the connection to the other datacenters gateway. Since the two gateways are on the same network this communication is possible, the gateway in the
onprem datacenter could not however talk direct to the API application which is running in the
cloud datacenter as these are on different networks. For this reason the
onprem gateway sends the traffic to the
cloud gateway, it is then the responsibility of the
cloud gateway which also sits inside the Kubernetes cluster to forward the request to the API.
The nice thing about gateways is that they can not decrypt the traffic they are transferring, Consul uses SNI headers to enable transparent TLS proxying of your data.
Let’s see how we run a gateway, the command we use to start a gateway is as follows. We are still using the
consul connect envoy main command however this time we are using the flags:
-mesh-gateway- Generate the bootstrap config required for envoy
-register- Register the new service inside Consul
-address- The local address and port to bind the gateway to
wan-address- The address and port used for external connections
connect envoy \
-address 10.5.0.33:8443 \
For our simple Docker example we can add a gateway using another service declaration, add the following block to the
docker-compose.yml located in the
Once you have added the gateway you can start your VM cluster.
➜ docker-compose up
Creating network "gateways_onprem" with driver "bridge"
Creating network "gateways_wan" with driver "bridge"
Creating gateways_consul_1 ... done
Creating gateways_web_1 ... done
Attaching to gateways_web_1, gateways_consul_1
If you look at the Consul UI http://localhost:8500/ui, you will see that the gateway has been started and is registered in Consul. Note the
red x next to the web-sidecar-proxy. The reason that this service is marked as unhealthy is because the required upstream service
API does not exist. We are going to look at that next.
Now we have the gateway running in our Virtual Machine environment we need to create a gateway in Kubernetes. This is achieved by overriding values in the Helm values.
When we federated the two datacenters we used the following config:
The Helm chart allows us to configure how we can run a Gateway in Kubernetes using the
meshGateway block. In this block we are going to configure the
wanAddress similar to how we did this for the Virtual Machine. We are going to set this value to the IP address of an external load balancer which we will later point at the service created by the Helm Chart.
We also want to create a Kubernetes service for the Gateways. The settings for the Gateway will differ from environment to environment but typically you will create
n Gateway instances and point an external or internal loadbalancer at them. It is the address of the loadbalancer which needs to be accessible to other datacenters.
helm-values.yml file in the folder
examples/gateways has already been pre-populated with these values. We can start our K8s cluster using the same command in the previous section. Run this in your terminal now:
yard up --consul-port 18500 \
--consul-values $PWD/helm-values.yml \
--network gateways_wan \
yard expose --bind-ip none \
--network gateways_wan \
--network-ip 22.214.171.124 \
--service-name svc/consul-consul-server \
--port 8600:8600 \
--port 8500:8500 \
--port 8302:8302 \
--port 8301:8301 \
Now we have the Consul cluster and the Gateway running we need to expose a loabalancer to it so that it is accessible from the
onprem cluster. If you were building this setup in the Cloud you would most likely use a Kubernetes
Service of type
LoadBalancer to create an expose the gateways. Running things locally we can not do that so we will have to simulate the process instead.
We can use
Shipyards capability to expose a Kubernetes service to either your local computer or another docker network. Using this tool we can create our fake loadbalancer which will expose the gateway to the
Run the following command in your terminal:
yard expose \
--bind-ip none \
--network gateways_wan \
--network-ip 126.96.36.199 \
--service-name svc/consul-consul-mesh-gateway \
Now everything is up and running, both clusters and their respective gateways you can deploy the api to the
➜ kubectl apply -f api.yaml
If you look at the Consul UI http://localhost:18500/ui, you should see your
api service and the
Your clusters and applications are now all up and running, however; if you visit the
web service http://localhost:9090/ui. You will see that there is still an upstream error.
Even though the two datacenters are federated and there are gateways linking the two systems, by default the service mesh will always resolve to the local datacenter. In our setup we do not have an
api service running in the
onprem datacenter so this is causing an error.
We can resolve this by explicitly telling Consul to resolve the
API service to the
cloud datacenter with L7 config.
service-resolver allows you to configure features for service resolution such as
subsets and much more.
In this example you are going to use the
redirect capability which allows the explicit resolution of a service to a particular datacenter. By default a service will always attempt to resolve to its own datacenter. The
service-resolver config entry allows you to override that behaviour.
The following example can be found in the file
kind = "service-resolver"
name = "api"
service = "api"
datacenter = "cloud"
Configuration entries can be written using with the
consul config write command.
Write this file to your cluster:
consul config write global_defaults.hcl
consul config write api_resolver.hcl
When you view the
Web service in your browser the
API will now be resolving correctly.
In this section you have learned how to use Consul Gateways to allow traffic to transparently and securely be routed from datacenter to datacenter. In the next example we will put all of this together to see how we can migrate our applications.
Before continuing don’t forget to clean up:
## Stopping Kubernetes and cleaning resources
INFO Removing cluster [shipyard]
INFO ...Removing server
INFO ...Removing docker image volume
INFO Removed cluster [shipyard]
Stopping gateway_consul_1 ... done
Stopping gateway_api_1 ... done