Appendix A: Setting External Load-Balancer¶
VPSA Object Storage is created by default with internal load balancer to distribute object operations between proxy virtual controllers (VC’s). When VPSA Object Storage cluster scales out and connections load is increasing, switching to an external load balancer might be required.
VPSA Object Storage provides a very smooth and easy way to switch between internal and external load balancer setups.
Internal Load Balancer runs in one of the VC’s
External Load Balancer(s) runs outside of the ZIOS VC’s
Below are instructions for setting up an external load balancer to terminate SSL connections and distribute the load over all VC’s.
There are many load balancer solutions in the market, setting them all up is quite similar procedure. This appendix gives an example of HAproxy, an open-source TCP/HTTP load-balancing proxy server that can be found in www.haproxy.org
The recommended configuration below will allow the following:
SSL Termination is done on the external load balancer for both object operation API’s and GUI connections. Authentication connections are always terminated in Object Storage.
Custom SSL certificate (PEM) located on the load balancer is used for SSL connections
Object operation connections are redirected to ZIOS proxy VC’s
Object operation connections are distributed between VC’s unevenly (proxy-only VCs to take more load than storage VC’s, and HA VCs to take the lowest load)
Redirected object operation connections will include the original client IP in a special header added by the load balancer (for logging in ZIOS proxy)
HTTP-based health check is performed by the load balancer to probe all ZIOS proxy VC’s
Authentication connections are redirected to ZObject Storage floating IP (SSL pass-through terminated on ZIOS, Custom SSL certificate must be uploaded to ZIOS as well).
GUI connections are redirected to ZIOS floating IP
Graphical statistics interface is enabled on the load balancer
Apply the following configuration to your ZIOS Settings:
Set the internet-facing domain-name/IP of the external LoadBalancer as ZIOS API Hostname / IP (zadara-qa.com which resolves to the external LB IP 18.104.22.168, is set in this example as ZIOS API Hostname)
Upload your custom SSL certificate (will be used for authentication connections). The certificate should match the custom domain name.
Set SSL Termination to “External”
HAProxy Installation and configuration instructions:
sudo add-apt-repository -y ppa:vbernat/haproxy-1.5 sudo apt-get update sudo apt-get install -y haproxy
- Upload your custom SSL certificate to HAProxy server. In this example
the certificate PEM file is placed under /etc/ssl/private/zadara_custom.pem
Edit /etc/haproxy/haproxy.cfg to include the following:
| global | maxconn 2048 | log /dev/log local0 | log /dev/log local1 notice | chroot /var/lib/haproxy | stats socket /run/haproxy/admin.sock mode 660 level admin | stats timeout 30s | user haproxy | group haproxy | daemon | tune.ssl.default-dh-param 2048 | # Default SSL material locations | ca-base /etc/ssl/certs | crt-base /etc/ssl/private | # Default ciphers to use on SSL-enabled listening sockets. | # For more information, see ciphers(1SSL). This list is from: | # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/ssl-default-bind-ciphers | ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS | ssl-default-bind-options no-sslv3 | | defaults | log global | mode http | option httplog | option dontlognull | timeout connect 5000 | timeout client 50000 | timeout server 50000 | errorfile 400 /etc/haproxy/errors/400.http | errorfile 403 /etc/haproxy/errors/403.http | errorfile 408 /etc/haproxy/errors/408.http | errorfile 500 /etc/haproxy/errors/500.http | errorfile 502 /etc/haproxy/errors/502.http | errorfile 503 /etc/haproxy/errors/503.http | errorfile 504 /etc/haproxy/errors/504.http | frontend fe-object-operations | bind 22.214.171.124:443 ssl crt /etc/ssl/private/zadara\_custom.pem | mode http | default\_backend be-zios-object-operations | | frontend fe-auth | bind 126.96.36.199:5000 | option tcplog | mode tcp | default\_backend be-floating-zios-auth | | frontend fe-gui | bind 188.8.131.52:8443 ssl crt /etc/ssl/private/zadara\_custom.pem | mode http | default\_backend be-floating-zios-gui | | backend be-zios-object-operations | mode http | balance roundrobin | option forwardfor | option httpclose | option httpchk HEAD /healthcheck HTTP/1.0 | server ziosStorageProxy0 184.108.40.206:8080 weight 10 check | server ziosStorageProxy1 220.127.116.11:8080 weight 10 check | server ziosStorageProxy2 18.104.22.168:8080 weight 50 check | server ziosProxyOnly3 22.214.171.124:8080 weight 100 check | server ziosProxyOnly4 126.96.36.199:8080 weight 100 check | | backend be-floating-zios-auth | mode tcp | server ziosFloating 188.8.131.52:5000 | | backend be-floating-zios-gui | mode http | server ziosFloating 184.108.40.206:80 | | listen stats \*:1936 | stats enable | stats uri / | stats auth zadara:zadara
Enable HAProxy logging (Optional)
Edit rsyslog conf:
| sudo vi /etc/rsyslog.conf | # provides UDP syslog reception | $ModLoad imudp | $UDPServerRun 514 | # provides TCP syslog reception | $ModLoad imtcp | $InputTCPServerRun 514
Restart the service:
sudo service rsyslog restart
Restart HAProxy service:
sudo service haproxy restart
Monitor statistics by browsing to http://<HAProxy server IP>:1936/ Credentials: zadara/zadara