首页 > 技术文章 > LVS主从部署配置和使用

renyongbin 2017-12-06 11:01 原文

  LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立,是中国国内最早出现的自由软件项目之一。

  LVS是Linux Virtual Server的简称,也就是Linux虚拟服务器, 是一个由章文嵩博士发起的自由软件项目,它的官方站点是www.linuxvirtualserver.org。现在LVS已经是 Linux标准内核的一部分,在Linux2.4内核以前,使用LVS时必须要重新编译内核以支持LVS功能模块,但是从Linux2.4内核以后,已经完全内置了LVS的各个功能模块,无需给内核打任何补丁,可以直接使用LVS提供的各种功能。
使用LVS技术要达到的目标是:通过LVS提供的负载均衡技术和Linux操作系统实现一个高性能、高可用的服务器群集,它具有良好可靠性、可扩展性和可操作性。从而以低廉的成本实现最优的服务性能。

  使用LVS架设的服务器集群系统有三个部分组成:最前端的负载均衡层,用Load Balancer表示,中间的服务器群组层,用Server Array表示,最底端的数据共享存储层,用Shared Storage表示,在用户看来,所有的内部应用都是透明的,用户只是在使用一个虚拟服务器提供的高性能服务。

  接下来开始部署配置LVS主从:

  免责声明:小伙伴在上生产环境前先测试有没有问题哈!!!

  

本次试验共计使用4台虚拟机,都是centos6.3.

 

192.168.146.164

192.168.146.160

192.168.146.162

192.168.146.163

 

具体实验环境配置如下:

192.168.146.101  LVS_VIP

192.168.146.164  LVS_MASTER

192.168.146.160  LVS_SALVE

192.168.146.162  WEB1

192.168.146.163  WEB2

 

 

 

 

192.168.146.164  LVS_MASTER:

安装IPVSADM:

[root@localhost ~]# yum -y install ipvsadm

为了测试方便,暂时关闭防火墙:

[root@localhost ~]# service iptables stop

安装KeepAlived:

[root@localhost ~]# cd /usr/src

[root@localhost src]# yum -y install openssl-devel

root@localhost src]# wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz

[root@localhost src]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/popt-static-1.13-7.el6.x86_64.rpm

[root@localhost src]# yum -y install popt-static-1.13-7.el6.x86_64.rpm

[root@localhost src]# yum -y install kernel-devel make gcc openssl-devel libnl* popt*

[root@localhost src]# ln -s /usr/src/kernels/2.6.32-220.13.1.el6.x86_64/ /usr/src/linux

[root@localhost src]# tar zxvf keepalived-1.2.7.tar.gz

[root@localhost src]# cd keepalived-1.2.7

[root@localhost keepalived-1.2.7]# ./configure --with-kernel-dir=/usr/src/kernels/2.6.32-358.2.1.el6.x86_64/

 

[root@localhost keepalived-1.2.7]# make && make install

[root@localhost keepalived-1.2.7]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/

[root@localhost keepalived-1.2.7]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

[root@localhost keepalived-1.2.7]# mkdir /etc/keepalived

[root@localhost keepalived-1.2.7]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

[root@localhost keepalived-1.2.7]# cp /usr/local/sbin/keepalived /usr/sbin/

KeepAlived安装完毕,然后进行配置。

 

 

打开IP Forward 功能(LVS现有三种负载均衡规则都需要打开此功能,如果不打开此功能,下面的配置配得再好都无济于事。)

打开后修改里面“net.ipv4.ip_forward = 1”

[root@localhost keepalived-1.2.7]# cat /etc/sysctl.conf

# Kernel sysctl configuration file for Red Hat Linux

#

# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and

# sysctl.conf(5) for more details.

 

# Controls IP packet forwarding

net.ipv4.ip_forward = 1

 

执行如下命令使设置立即生效:

[root@localhost keepalived-1.2.7]# sysctl -p

net.ipv4.ip_forward = 1

net.ipv4.conf.default.rp_filter = 1

net.ipv4.conf.default.accept_source_route = 0

kernel.sysrq = 0

kernel.core_uses_pid = 1

net.ipv4.tcp_syncookies = 1

error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key

error: "net.bridge.bridge-nf-call-iptables" is an unknown key

error: "net.bridge.bridge-nf-call-arptables" is an unknown key

kernel.msgmnb = 65536

kernel.msgmax = 65536

kernel.shmmax = 68719476736

kernel.shmall = 4294967296

 

配置keepalived:

配置文件在这个位置: /etc/keepalived/keepalived.conf

 

 

[root@localhost keepalived-1.2.7]# cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived 

global_defs { 

    notification_email {  

        test@sina.com  

    }  

    notification_email_from admin@test.com 

    smtp_server 127.0.0.1 

    smtp_connect_timeout 30 

    router_id LVS_DEVEL

 

vrrp_instance VI_1 { 

    state MASTER 

    interface eth1 

    virtual_router_id 60 

    priority 100 

    advert_int 1 

    authentication { 

        auth_type PASS 

        auth_pass 1111 

    } 

    virtual_ipaddress { 

        192.168.146.101 

    } 

 

virtual_server 192.168.146.101 8080 { 

    delay_loop 6 

    lb_algo rr  

    lb_kind DR 

    nat_mask 255.255.255.0 

    persistence_timeout 50 

    protocol TCP 

 

    real_server 192.168.146.162 8080 { 

        weight 1 

        TCP_CHECK { 

            connect_timeout 3 

            nb_get_retry 3 

            delay_before_retry 3 

        } 

    } 

 

    real_server 192.168.146.163 8080 { 

        weight 1 

        TCP_CHECK { 

            connect_timeout 3 

            nb_get_retry 3 

            delay_before_retry 3 

        } 

    } 

}

 

至此,keepalived配置完成。

把keepalived加入开机自启动,并开启keepalived服务:

[root@localhost keepalived-1.2.7]# chkconfig keepalived on

[root@localhost keepalived-1.2.7]# service keepalived start

Starting keepalived:                                       [  OK  ]

 

查看keepalived进程:

[root@localhost keepalived-1.2.7]# ps aux | grep keepalived

root      1117  0.0  0.0  42172  1000 ?        Ss   14:37   0:00 keepalived -D

root      1119  0.0  0.2  44400  2296 ?        S    14:37   0:00 keepalived -D

root      1120  0.0  0.1  44276  1664 ?        S    14:37   0:00 keepalived -D

root      1165  0.0  0.0 103268   884 pts/5    S+   14:48   0:00 grep keepalived

 

查看虚拟IP是否加上:

[root@localhost keepalived-1.2.7]# ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:b2:88:f7 brd ff:ff:ff:ff:ff:ff

    inet 192.168.146.164/24 brd 192.168.146.255 scope global eth1

    inet 192.168.146.101/32 scope global eth1

    inet6 fe80::20c:29ff:feb2:88f7/64 scope link

       valid_lft forever preferred_lft forever

 

红色部分显示已经自动加上虚拟IP了。

 

至此,LVS_MASTER服务器已经配置好并启动了。

接下来开始配置部署LVS从服务器:

 

 

 

 

 

 

 

192.168.146.160  LVS_SALVE:

具体过程同192.168.146.164  LVS_MASTER!!

 

接下里配置部署两台web服务器:

配置部署192.168.146.162 web1:

配置虚拟IP启动脚本:

[root@localhost ~]# cat /etc/init.d/realserver.sh

SNS_VIP=192.168.146.101 

. /etc/rc.d/init.d/functions 

case "$1" in 

start) 

 ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP 

 /sbin/route add -host $SNS_VIP dev lo:0 

 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore 

 echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce 

 echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore 

 echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce 

 sysctl -p >/dev/null 2>&1 

 echo "RealServer Start OK" 

 ;; 

stop) 

 ifconfig lo:0 down 

 route del $SNS_VIP >/dev/null 2>&1 

 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore 

 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce 

 echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore 

 echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce 

 echo "RealServer Stoped" 

 ;; 

 *) 

 echo "Usage: $0 {start|stop}" 

 exit 1 

esac 

exit 0

 

配置部署tomcat:

[root@localhost ~]# yum -y install tomcat6 tomcat6-webapps tomcat6-admin-webapps

[root@localhost ~]# chkconfig tomcat6 on

[root@localhost ~]# service tomcat6 start

Starting tomcat6:                                          [  OK  ]

 

 

为了测试方便暂时关闭防火墙:

[root@localhost ~]# service iptables stop

打开浏览器测试:http://192.168.146.142:8080

会看到TOMCAT的熟悉页面了。

 

为了测试负载均衡,我们将这个页面改下,以更好的标识这个网页是本服务器的

Tomcat6安装目录位于/usr/share/tomcat6,所以我们要编辑tomcat下的webapps/ROOT/index.html这个文件。

[root@localhost ~]# cd /usr/share/tomcat6/webapps/ROOT/

[root@localhost ROOT]# cat /dev/null > index.html

[root@localhost ROOT]# cat index.html

web1 192.168.146.162

启动虚拟IP的脚本:

[root@localhost ROOT]# sh /etc/init.d/realserver.sh start

RealServer Start OK

eth1      Link encap:Ethernet  HWaddr 00:0C:29:1F:0F:F3 

          inet addr:192.168.146.162  Bcast:192.168.146.255  Mask:255.255.255.0

          inet6 addr: fe80::20c:29ff:fe1f:ff3/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:89543 errors:0 dropped:0 overruns:0 frame:0

          TX packets:47914 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:112074876 (106.8 MiB)  TX bytes:2870905 (2.7 MiB)

 

lo        Link encap:Local Loopback 

          inet addr:127.0.0.1  Mask:255.0.0.0

          inet6 addr: ::1/128 Scope:Host

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

          RX packets:22 errors:0 dropped:0 overruns:0 frame:0

          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:1408 (1.3 KiB)  TX bytes:1408 (1.3 KiB)

 

lo:0      Link encap:Local Loopback 

          inet addr:192.168.146.101  Mask:255.255.255.255

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

 

运行后会看到网络有了一个虚拟IP,即红色标注部分。

 

 

去LVS_MASTER服务器的终端查看下ipvsadm,查看已经连接上了WEB1服务器(即红色标注部分):

[root@localhost keepalived-1.2.7]# ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.146.101:8080 rr persistent 50

  -> 192.168.146.162:8080         Route   1      0          0

 

 

已经可以看到有服务器加入进来了。

此时我们访问网页http://192.168.146.101:8080,出现界面显示web1 192.168.146.162;或者使用命令行测试转发情况都可以查看。

 

 

 

 

配置部署192.168.146.163 web2:

具体配置部署过程同web1!!

 

 

 

 

 

    

 

 

推荐阅读