HAproxy reload config file with uninterrupt session

HAProxy is a high performance load balancer. It is very light-weight, and free, making it a great option if you are in the market for a load balancer and need to keep your costs down.

Lately we’ve been making a lot of load balancer changes at work to accommodate new systems and services. Even though we have two load balancers running with keepalived taking care of any failover situations, I was thinking about how we go about reloading our configuration files. In the event of a change, the “common” way to get the changes to take effect is to run /etc/init.d/haproxy restart. This is bad for a couple major reasons:

You are temporarily shutting your load balancer down
You are severing any current connections going through the load balancer
You might say, “if you have two load balancers with keepalived, restarting the service should be fine since keepalived will handle the failover.” This, however, isn’t always true. Keepalived uses advertisements to determine when to fail over. The default advertisement interval is 1 second (configurable in keepalived.conf). The skew time helps to keep everyone from trying to transition at once. It is a number between 0 and 1, based on the formula (256 – priority) / 256. As defined in the RFC, the backup must receive an advertisement from the master every (3 * advert_int) + skew_time seconds. If it doesn’t hear anything from the master, it takes over.

Let’s assume you are using the default interval of 1 second. On my test machine, this is the duration of time it takes to restart haproxy:

time /etc/init.d/haproxy restart
Restarting haproxy haproxy
   ...done.real    0m0.022s
user    0m0.000s
sys     0m0.016s

In this situation, haproxy would restart much faster than your 1 second interval. You could get lucky and happen to restart it just before the check, but luck is not consistent enough to be useful. Also, in very high-traffic situations, you’ll be causing a lot of connection issues. So we cannot rely on keepalived to solve the first problem, and it definitely doesn’t solve the second problem.

After sifting through haproxy documentation (the text-based documentation, not the man page) (/usr/share/doc/haproxy/haproxy-en.txt.gz on Ubuntu), I came across this:

    313
    314     global    315         daemon    316         quiet    317         nbproc  2
    318         pidfile /var/run/haproxy-private.pid    319
    320     # to stop only those processes among others :    321     # kill $(</var/run/haproxy-private.pid)    
    322
    323     # to reload a new configuration with minimal service impact and without    
    324     # breaking existing sessions :    
    325     # haproxy -f haproxy.cfg -p $(</var/run/haproxy-private.pid) -st $(</var/run/haproxy-private.pid)

That last command is the one of interest. The -p asks the process to write down each of its children’s pids to the specified pid file, and the -st specifies a list of pids to send a SIGTERM to after startup. But it does this in an interesting way:

    609 The '-st' and '-sf' command line options are used to inform previously running
    610 processes that a configuration is being reloaded. They will receive the SIGTTOU    
    611 signal to ask them to temporarily stop listening to the ports so that the new
    612 process can grab them. If anything wrong happens, the new process will send
    613 them a SIGTTIN to tell them to re-listen to the ports and continue their normal
    614 work. Otherwise, it will either ask them to finish (-sf) their work then softly    
    615 exit, or immediately terminate (-st), breaking existing sessions. A typical use    
    616 of this allows a configuration reload without service interruption :    
    617
    618  # haproxy -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

The end-result is a reload of the configuration file which is not visible by the customer. It also solves the second problem! Let’s look at an example of the command and look at the time compared to our above example:

# time haproxy -f /etc/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

real    0m0.018s
user    0m0.000s
sys     0m0.004s

I’ve specified the config file I want to use and the pid file haproxy is currently using. The $(cat /var/run/haproxy.pid) takes the output of cat /var/run/haproxy.pid and passes it in to the -sf parameter as a list, which is what it is expecting. You will notice that the time is actually faster too (.012s sys, and .004s real). It may not seem like much, but if you are dealing with very high volumes of traffic, this can be pretty important. Luckily for us it doesn’t matter because we’ve been able to reload the haproxy configuration without dropping any connections and without causing any customer-facing issues.

UPDATE: There is a reload in some of the init.d scripts (I haven’t checked every OS, so this can vary), but it uses the -st option which will break existing sessions, as opposed to using -sf to do a graceful hand-off. You can modify the haproxy_reload() function to use the -sf if you want. I also find it a bit confusing that the documentation uses $(cat /path/to/pidfile) whereas this haproxy_reload() function uses $(<$PIDFILE). Either should work, but really, way to lead by example…

转自:http://www.cnblogs.com/Bozh/p/4169969.html

原创文章,作者:s19930811,如若转载,请注明出处:http://www.178linux.com/2453

(0)
上一篇 2015-04-03 22:07
下一篇 2015-04-03 22:10

相关推荐

  • AB测试命令

    [root@chuangyuegou ~]# ab -n 10000 -c 49 http://192.168.0.99/index.html This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd,…

    Linux干货 2016-12-05
  • 步入LINUX

    初次融入LINUX的环境。。。

    Linux干货 2018-03-26
  • 计算机组成原理

                                 计算机组成原理 Linux 操作系统是UNIX 操作系统的一种克隆系统。它诞生于1991 年的10 月5 日(这是第一次正式向外公布的时间)。以后借助于…

    Linux干货 2016-10-27
  • 几个正则表达式和find查找语句事例

    显示当前系统上root、fedora、或user1用户的默认shell; ~]# grep -E "^(root|bin|hadop)" /etc/passwd | cut -d: -f7 找出/etc/rc.d/init.d/functions文件中某单词后面跟一组小括号的行,形如: hello(); ~]#grep -E -o &qu…

    Linux干货 2016-10-30
  • N29第一周作业:初识Linux系统

    1、描述计算机的组成及其功能。
    2、按系列罗列Linux的发行版,并描述不同发行版之间的联系与区别。
    3、描述Linux的哲学思想,并按照自己的理解对其进行解释性描述。
    4、说明Linux系统上命令的使用格式;详细介绍ifconfig、echo、tty、startx、export、pwd、history、shutdown、poweroff、reboot、hwclock、date命令的使用,并配合相应的示例来阐述。
    5、如何在Linux系统上获取命令的帮助信息,请详细列出,并描述man文档的章节是如何划分的。
    6、请罗列Linux发行版的基础目录名称命名法则及功用规定

    2018-03-04
  • 利用keepalived搭建高可用集群

    在一个系统中,常常存在一些单点服务器,为了提高整个系统的稳定性,我们常常需要对这些单点服务做高可用配置;keepalived即为一种常用的高可用配置服务; Keepalived的工作模式有以下两种: 1、 主/备:即单虚拟路径器,仅配置一个VIP; 2、 主/主:即多个虚拟路径器,配置多个VIP,每个虚拟路径器作为其中某个VIP的的master虚拟路径器; …

    Linux干货 2016-11-01