HAproxy reload config file with uninterrupt session

HAProxy is a high performance load balancer. It is very light-weight, and free, making it a great option if you are in the market for a load balancer and need to keep your costs down.

Lately we’ve been making a lot of load balancer changes at work to accommodate new systems and services. Even though we have two load balancers running with keepalived taking care of any failover situations, I was thinking about how we go about reloading our configuration files. In the event of a change, the “common” way to get the changes to take effect is to run /etc/init.d/haproxy restart. This is bad for a couple major reasons:

You are temporarily shutting your load balancer down
You are severing any current connections going through the load balancer
You might say, “if you have two load balancers with keepalived, restarting the service should be fine since keepalived will handle the failover.” This, however, isn’t always true. Keepalived uses advertisements to determine when to fail over. The default advertisement interval is 1 second (configurable in keepalived.conf). The skew time helps to keep everyone from trying to transition at once. It is a number between 0 and 1, based on the formula (256 – priority) / 256. As defined in the RFC, the backup must receive an advertisement from the master every (3 * advert_int) + skew_time seconds. If it doesn’t hear anything from the master, it takes over.

Let’s assume you are using the default interval of 1 second. On my test machine, this is the duration of time it takes to restart haproxy:

time /etc/init.d/haproxy restart
Restarting haproxy haproxy
   ...done.real    0m0.022s
user    0m0.000s
sys     0m0.016s

In this situation, haproxy would restart much faster than your 1 second interval. You could get lucky and happen to restart it just before the check, but luck is not consistent enough to be useful. Also, in very high-traffic situations, you’ll be causing a lot of connection issues. So we cannot rely on keepalived to solve the first problem, and it definitely doesn’t solve the second problem.

After sifting through haproxy documentation (the text-based documentation, not the man page) (/usr/share/doc/haproxy/haproxy-en.txt.gz on Ubuntu), I came across this:

    313
    314     global    315         daemon    316         quiet    317         nbproc  2
    318         pidfile /var/run/haproxy-private.pid    319
    320     # to stop only those processes among others :    321     # kill $(</var/run/haproxy-private.pid)    
    322
    323     # to reload a new configuration with minimal service impact and without    
    324     # breaking existing sessions :    
    325     # haproxy -f haproxy.cfg -p $(</var/run/haproxy-private.pid) -st $(</var/run/haproxy-private.pid)

That last command is the one of interest. The -p asks the process to write down each of its children’s pids to the specified pid file, and the -st specifies a list of pids to send a SIGTERM to after startup. But it does this in an interesting way:

    609 The '-st' and '-sf' command line options are used to inform previously running
    610 processes that a configuration is being reloaded. They will receive the SIGTTOU    
    611 signal to ask them to temporarily stop listening to the ports so that the new
    612 process can grab them. If anything wrong happens, the new process will send
    613 them a SIGTTIN to tell them to re-listen to the ports and continue their normal
    614 work. Otherwise, it will either ask them to finish (-sf) their work then softly    
    615 exit, or immediately terminate (-st), breaking existing sessions. A typical use    
    616 of this allows a configuration reload without service interruption :    
    617
    618  # haproxy -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

The end-result is a reload of the configuration file which is not visible by the customer. It also solves the second problem! Let’s look at an example of the command and look at the time compared to our above example:

# time haproxy -f /etc/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

real    0m0.018s
user    0m0.000s
sys     0m0.004s

I’ve specified the config file I want to use and the pid file haproxy is currently using. The $(cat /var/run/haproxy.pid) takes the output of cat /var/run/haproxy.pid and passes it in to the -sf parameter as a list, which is what it is expecting. You will notice that the time is actually faster too (.012s sys, and .004s real). It may not seem like much, but if you are dealing with very high volumes of traffic, this can be pretty important. Luckily for us it doesn’t matter because we’ve been able to reload the haproxy configuration without dropping any connections and without causing any customer-facing issues.

UPDATE: There is a reload in some of the init.d scripts (I haven’t checked every OS, so this can vary), but it uses the -st option which will break existing sessions, as opposed to using -sf to do a graceful hand-off. You can modify the haproxy_reload() function to use the -sf if you want. I also find it a bit confusing that the documentation uses $(cat /path/to/pidfile) whereas this haproxy_reload() function uses $(<$PIDFILE). Either should work, but really, way to lead by example…

转自:http://www.cnblogs.com/Bozh/p/4169969.html

原创文章,作者:s19930811,如若转载,请注明出处:http://www.178linux.com/2453

(0)
s19930811s19930811
上一篇 2015-04-03 22:07
下一篇 2015-04-03 22:10

相关推荐

  • Linux发行版概述

    Linux发行版概述 Linux发行版有数百种之多,最主流的三个分支为Debain、Slackware、RedHat Debain Debain是三大主流发行版中唯一由社区维护的版本,无商业版本,相对较为轻巧,对使用者的技术要求较高 * Ubuntu、Knopix为Debian的主要子分支,其中Knopix是以安全著称的 Slackware(SUSE) SU…

    Linux干货 2017-07-02
  • 简单的启动排错

    一、出现 grub> 字样,等待交互式输入指令,说明MBR中Bootloader的stage1、stage1_5、stage2都正常,但丢失了/boot/grub.conf 文件,所以无法加载启动项菜单,就无法读取默认启动的kernel、initrd等文件路径。 解决办法:无需救援模式,直接在此交互式界面指定相关文件的加载路径。 (1) grub&gt…

    Linux干货 2016-11-26
  • LInux用户和组管理详解

    一、用户和组的概念 Linux系统是多任务、多用户的分时操作系统。用户要使用系统资源,需要向管理员验证身份,由管理员进行认证和授权,并对用户的行为进行追踪和审计,以保障系统资源的合理分配以及系统的安全。 要达到账号管理的目的,需要对用户和用户组进行管理。下面分别来介绍用户管理和用户组管理。 每个用户都有唯一的一个编号来标识,这个标识号叫Uid,每个用户至少属…

    Linux干货 2016-08-02
  • GOPS 2016全球运维大会 • 北京站

    GOPS 2016全球运维大会 • 北京站已经圆满落幕,最新GOPS大会推荐:GOPS2017全球运维大会 • 深圳站 基本信息: GOPS2017全球运维大会 • 深圳站 时间:2017-04-21 08:00:00 至 2017-04-22 18:00:00结束 地点:深圳    会议规模:5000人 会议详情:http://www.…

    Linux干货 2016-11-03
  • vim实用技能

    vim编辑器 简介 vi: Visual Interface,文本编辑器 文本:ASCII, Unicode 文本编辑种类: 行编辑器: sed 全屏编辑器:nano, vi VIM – Vi IMproved 使用 vim:模式化的编辑 基本模式: 编辑模式,命令模式 输入模式 末行模式: 内置的命令行接口 打开文件: # vim [OPTIO…

    Linux干货 2018-01-16
  • linux命令的使用格式

    Linux命令的一般格式为: [root@node1 ~]#command [选项] [参数] 1、ifconfig 设备 参数 参数: up:启动指定设备 down:关闭指定设备 arp:指定设备是否支持ARP协议 -a:显示全部接口信息 例如:[root@node1 ~]#ifconfig eth0 up #启动eth0网卡 2、echo打印 &nbsp…

    Linux干货 2016-09-17