Tomcat通过Redis实现session共享的完整部署记录

洗尽了浮华

发布于 2018-09-28 11:10:04

5.2K0

发布于 2018-09-28 11:10:04

对于生产环境有了一定规模的tomcat集群业务来说,要实现session会话共享,比较稳妥的方式就是使用数据库持久化session.为什么要持久化session（共享session）呢？因为在客户端每个用户的Session对象存在Servlet容器中，如果Tomcat服务器重启或者宕机的话，那么该session就会丢失，而客户端的操作会由于session丢失而造成数据丢失；如果当前用户访问量巨大，每个用户的Session里存放大量数据的话，那么就很占用服务器大量的内存，进而致使服务器性能受到影响。数据库持久化session，分为物理数据库和内存数据库。物理数据库备份session，由于其性能原因，不推荐；内存数据库可以使用redis和memcached，之前已经介绍了memcached方式实现session共享了,下面说下redis方式:

但是有一点需要清楚:就是Redis这种方式目前还暂不支持Tomcat8环境(因为现在网上插件不支持tomcat8,非要支持tomcat8,则需修改插件jar包的源代码)!下面就先说下Tomcat7+Redis+Nginx实现session会话共享的操作记录:

实现达到的效果

当客户端访问Nginx服务器时，Nginx负载均衡会自动将请求转发到Tomcat1节点或Tomcat2节点服务器，以减轻Tomcat压力，从而达到Tomcat集群化部署，为了使各Tomcat之间共享同一个Session，将采用Redis缓存服务来集中管理Session存储。Nginx实现负载均衡，并使用Redis实现session共享。本案例采用的是redis单点方式(可以给这个redis节点配置一个从节点,采用redis主从模式,连接redis的master节点.redis默认不支持主主模式)，如果使用redis集群方式,则采用Sentinels连接。

1) 基础信息

ip                 主机名           应用                       端口
192.168.10.200     Nginx-node       nginx1.12.2                80
192.168.10.201     Tomcat-node1     java8.131、tomcat7.0.90    8080
192.168.10.202     Tomcat-node2     java8.131、tomcat7.0.90    8080
192.168.10.203     redis-node       redis4.0.1                6379

下面操作在三台机器上同样执行：
[root@Nginx-node ~]# cat /etc/redhat-release
CentOS release 6.9 (Final)
    
为了方便测试，关闭iptables防火墙和selinux。如果是生产环境，开启iptables后，需要开放对应的应用端口。
[root@Nginx-node ~]# setenforce 0
[root@Nginx-node ~]# getenforce
disabled
[root@Nginx-node ~]# cat /etc/sysconfig/selinux |grep "SELINUX=disabled"
SELINUX=disabled
[root@Nginx-node ~]# /etc/init.d/iptables stop

本案例环境部署中所需的软件下载地址：https://pan.baidu.com/s/1f9GM4iVAzfJsjskpAwv8Og
提取密码:dsfs
下载到服务器上的/usr/local/src目录下.另外:节点服务器的系统时间一定要保持一致!!

2) 安装Nginx（在192.168.10.200机器上操作）

[root@Nginx-node ~]# cd /usr/local/src/
[root@Nginx-node src]# ll
total 8920
-rw-rw-r--. 1 root root  981687 Oct 27  2017 nginx-1.12.2.tar.gz
-rw-rw-r--. 1 root root 5453234 Aug 23  2018 openssl-1.1.0i.tar.gz
-rw-rw-r--. 1 root root 2081413 Aug 23  2018 pcre-8.42.tar.gz
-rw-rw-r--. 1 root root  607698 Jan 16  2017 zlib-1.2.11.tar.gz
 
安装依赖包
[root@Nginx-node src]# yum -y install gcc gcc-c++
 
安装pcre库
[root@Nginx-node src]# tar -zvxf pcre-8.42.tar.gz
[root@Nginx-node src]# cd pcre-8.42
[root@Nginx-node pcre-8.42]# ./configure && make && make install
 
安装zlib库
[root@Nginx-node pcre-8.42]# cd /usr/local/src/
[root@Nginx-node src]# tar -zvxf zlib-1.2.11.tar.gz
[root@Nginx-node src]# cd zlib-1.2.11
[root@Nginx-node zlib-1.2.11]# ./configure && make && make install
 
安装openssl
[root@Nginx-node zlib-1.2.11]# cd /usr/local/src/
[root@Nginx-node src]# tar -zvxf openssl-1.1.0i.tar.gz
[root@Nginx-node src]# cd openssl-1.1.0i
[root@Nginx-node openssl-1.1.0i]# ./config && make && make install
 
安装nginx，特别注意要指定prce zlib openssl原码包位置
[root@Nginx-node openssl-1.1.0i]# cd /usr/local/src/
[root@Nginx-node src]# tar -zvxf nginx-1.12.2.tar.gz
[root@Nginx-node src]# cd nginx-1.12.2                
[root@Nginx-node nginx-1.12.2]# ./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-pcre=/usr/local/src/pcre-8.42 --with-zlib=/usr/local/src/zlib-1.2.11  --with-openssl=/usr/local/src/openssl-1.1.0i
[root@Nginx-node nginx-1.12.2]# make && make install
 
安装成功后配置nginx
[root@Nginx-node nginx-1.12.2]# cd /usr/local/nginx/conf/
[root@Nginx-node conf]# cp nginx.conf nginx.conf.bak
[root@Nginx-node conf]# cat nginx.conf
#user  nobody;
worker_processes  8;
  
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
  
#pid        logs/nginx.pid;
  
worker_rlimit_nofile 65535;
events {
   use epoll;
    worker_connections  65535;
}
  
http {
    include       mime.types;
    default_type  application/octet-stream;
    charset utf-8;
          
    ######
    ## set access log format
    ######
    log_format  main  '$http_x_forwarded_for $remote_addr $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_cookie" $host $request_time';
  
    #######
    ## http setting
    #######
    sendfile       on;
    tcp_nopush     on;
    tcp_nodelay    on;
    keepalive_timeout  65;
    fastcgi_connect_timeout 30000;
    fastcgi_send_timeout 30000;
    fastcgi_read_timeout 30000;
    fastcgi_buffer_size 256k;
    fastcgi_buffers 8 256k;
    fastcgi_busy_buffers_size 256k;
    fastcgi_temp_file_write_size 256k;
    fastcgi_intercept_errors on;
  
    ##cache##
    client_header_timeout 60s;
    client_body_timeout 60s;
    client_max_body_size 10m;
    client_body_buffer_size 1m;
    proxy_connect_timeout 5;
    proxy_read_timeout 60;
    proxy_send_timeout 5;             
    proxy_buffer_size 64k;
    proxy_buffers 4 128k;
    proxy_busy_buffers_size 128k;
    proxy_temp_file_write_size 1m;
    proxy_temp_path /home/temp_dir;
    proxy_cache_path /home/cache levels=1:2 keys_zone=cache_one:200m inactive=1d max_size=30g;
    ##end##
  
    gzip  on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 9;
    gzip_types       text/plain application/x-javascript text/css application/xml text/javascript application/x-httpd-php;
    gzip_vary on;
  
    ## includes vhosts
    include vhosts/*.conf;
}
 
[root@Nginx-node conf]# mkdir vhosts
[root@Nginx-node conf]# cd vhosts/
[root@Nginx-node vhosts]# vim lb_tomcat.conf
  upstream tomcat-lb {
      server 192.168.10.201:8080;
      server 192.168.10.202:8080;
      }
                    
      server {
      listen  80;
      server_name www.kevin.com;
      location / {
          proxy_pass http://tomcat-lb;
          proxy_set_header  X-Real-IP $remote_addr;
          proxy_set_header  REMOTE-HOST $remote_addr;
          proxy_set_header  Host $host;
          proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
        }
        location ~ .*\.(gif|jpg|png|htm|html|css|ico|flv|swf)(.*) {
              proxy_pass http://tomcat-lb;
              proxy_redirect off;
              proxy_set_header Host $host;
              proxy_cache cache_one;
              proxy_cache_valid 200 302 1h;
              proxy_cache_valid 301 1d;
              proxy_cache_valid any 10m;
              expires 30d;
              proxy_cache_key $host$uri$is_args$args;
        }
}
 
[root@Nginx-node vhosts]# /usr/local/nginx/sbin/nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
[root@Nginx-node conf]# /usr/local/nginx/sbin/nginx
[root@Nginx-node conf]# lsof -i:80
COMMAND   PID   USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
nginx   25292   root    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25293 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25294 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25295 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25296 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25297 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25298 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25299 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
nginx   25300 nobody    6u  IPv4 19679665      0t0  TCP *:http (LISTEN)
 
将域名www.kevin.com解析到192.168.10.200上，访问http://www.kevin.com，发现访问请求结果会负载到192.168.10.201和192.168.10.202的tomcat上了。

3）安装tomcat（在192.168.10.201和192.168.10.202两台机器上操作）

安装java8环境。先卸载掉系统自带的java7，然后安装java8
[root@Tomcat-node1 ~]# java -version
java version "1.7.0_131"
OpenJDK Runtime Environment (rhel-2.6.9.0.el6_8-x86_64 u131-b00)
OpenJDK 64-Bit Server VM (build 24.131-b00, mixed mode)
[root@Tomcat-node1 ~]# yum -y remove java-1.7.0-openjdk*
[root@Tomcat-node1 ~]# yum -y remove tzdata-java.noarch
[root@Tomcat-node1 ~]# java -version                 
-bash: /usr/bin/java: No such file or directory
   
[root@Tomcat-node1 ~]# ll /usr/local/src/jdk-8u131-linux-x64_.rpm
-rw-rw-r--. 1 root root 169983496 Nov 19  2017 /usr/local/src/jdk-8u131-linux-x64_.rpm
[root@Tomcat-node1 ~]# rpm -ivh /usr/local/src/jdk-8u131-linux-x64_.rpm --force
[root@Tomcat-node1 ~]# vim /etc/profile
......
JAVA_HOME=/usr/java/jdk1.8.0_131
JAVA_BIN=/usr/java/jdk1.8.0_131/bin
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/bin:/sbin/
CLASSPATH=.:/lib/dt.jar:/lib/tools.jar
export JAVA_HOME JAVA_BIN PATH CLASSPATH
   
[root@Tomcat-node1 ~]# source /etc/profile
[root@Tomcat-node1 ~]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
You have new mail in /var/spool/mail/root
   
安装配置tomcat7
[root@Tomcat-node1 ~]# cd /usr/local/src/
[root@Tomcat-node1 src]# ll apache-tomcat-7.0.90.tar.gz
-rw-rw-r--. 1 root root 9472492 Nov  9  2017 apache-tomcat-7.0.90.tar.gz
[root@Tomcat-node1 src]# tar -zvxf apache-tomcat-7.0.90.tar.gz
[root@Tomcat-node1 src]# mv apache-tomcat-7.0.90 /usr/local/tomcat7
   
配置tomcat7
[root@Tomcat-node1 ~]# cd /usr/local/tomcat7/conf/
[root@Tomcat-node1 conf]# cp server.xml server.xml.bak
[root@Tomcat-node1 conf]# vim server.xml
.......
 <Engine name="Catalina" defaultHost="localhost">                #这一行不需要修改,没必要动
.......
 <Host name="localhost" appBase="/data/webapps/" unpackWARS="true" autoDeploy="true">
        <Context path="" docBase="/data/webapps" reloadable="true" />
......
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="/data/logs"
               prefix="web1_access_log." suffix=".txt"
               pattern="%h %l %u %t "%r" %s %b" />
  </Host>


创建项目目录
[root@Tomcat-node1 conf]# mkdir -p /data/webapps/{WEB-INF,META-INF,classes,lib}
[root@Tomcat-node1 conf]# mkdir /data/logs

创建测试文件
[root@Tomcat-node1 ~]# touch /data/webapps/index.jsp 
[root@Tomcat-node1 ~]# vim /data/webapps/index.jsp         
<html>
    <body bgcolor="green">     
    <center>    
    <%=  request.getSession().getId()  %>    
    <h1>192.168.10.201</h1>
    <h1>port:8080</h1>
    <h1>this is Tomcat-node1! </h1> 
    </center>
    </body>
</html>
 
<%@ page contentType="text/html;charset=UTF-8" isELIgnored="false"%>
SessionID:<%=session.getId()%><BR>
SessionIP:<%=request.getServerName()%> <BR>
SessionPort:<%=request.getServerPort()%>
<%     out.println("This is Tomcat server 201 !");     %>
You have new mail in /var/spool/mail/root


另一个节点的测试文件为:
[root@Tomcat-node2 ~]# vim /data/webapps/index.jsp 
<html>
    <body bgcolor="yellow">     
    <center>    
    <%=  request.getSession().getId()  %>    
    <h1>192.168.10.202</h1>
    <h1>port:8080</h1>
    <h1>this is Tomcat-node2! </h1> 
    </center>
    </body>
</html>
 
<%@ page contentType="text/html;charset=UTF-8" isELIgnored="false"%>
SessionID:<%=session.getId()%><BR>
SessionIP:<%=request.getServerName()%> <BR>
SessionPort:<%=request.getServerPort()%>
<%     out.println("This is Tomcat server 202 !");     %>


接着配置Redis+Session共享
[root@Tomcat-node1 ~]# cd /usr/local/tomcat7/conf/
[root@Tomcat-node1 conf]# cp context.xml context.xml.bak 
[root@Tomcat-node1 conf]# cat context.xml
......
    <Valve className="com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve" />  
    <Manager className="com.orangefunction.tomcat.redissessions.RedisSessionManager"  
             host="192.168.10.203"  
             port="6379"  
             database="0" 
             maxInactiveInterval="60" />


===============================================================
温馨提示:
如果远程redis设置了密码,则就多加一行配置:password="123456"
===============================================================


最后将依赖的三个jar包拷贝到tomcat7的lib目录下
[root@Tomcat-node1 jar]# pwd
/usr/local/src/tomcat_redis_session/jar
[root@Tomcat-node1 jar]# ll
total 676
-rw-rw-r--. 1 root root 111969 Jul 29  2015 commons-pool2-2.4.2.jar
-rw-rw-r--. 1 root root 553762 Oct 16  2017 jedis-2.9.0.jar
-rw-rw-r--. 1 root root  20241 Aug 27 16:45 tomcat-redis-session-manager1.2.jar
[root@Tomcat-node1 jar]# \cp -rf ./* /usr/local/tomcat7/lib/


启动tomcat
[root@Tomcat-node1 src]# /usr/local/tomcat8/bin/startup.sh
Using CATALINA_BASE:   /usr/local/tomcat8
Using CATALINA_HOME:   /usr/local/tomcat8
Using CATALINA_TMPDIR: /usr/local/tomcat8/temp
Using JRE_HOME:        /usr/java/jdk1.8.0_131
Using CLASSPATH:       /usr/local/tomcat8/bin/bootstrap.jar:/usr/local/tomcat8/bin/tomcat-juli.jar
Tomcat started.
You have new mail in /var/spool/mail/root
[root@Tomcat-node1 src]# ps -ef|grep tomcat
root      8477     1 87 03:11 pts/0    00:00:03 /usr/java/jdk1.8.0_131/bin/java -Djava.util.logging.config.file=/usr/local/tomcat8/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -classpath /usr/local/tomcat8/bin/bootstrap.jar:/usr/local/tomcat8/bin/tomcat-juli.jar -Dcatalina.base=/usr/local/tomcat8 -Dcatalina.home=/usr/local/tomcat8 -Djava.io.tmpdir=/usr/local/tomcat8/temp org.apache.catalina.startup.Bootstrap start
root      8528  6829  0 03:11 pts/0    00:00:00 grep tomcat
[root@Tomcat-node1 src]# lsof -i:8080
COMMAND  PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
java    8477 root   49u  IPv6 12974768      0t0  TCP *:webcache (LISTEN)

4) redis安装

[root@redis-node ~]# cd /usr/local/src/
[root@redis-node src]# ll redis-4.0.1.tar.gz 
-rw-rw-r-- 1 root root 1711660 Aug 27 16:19 redis-4.0.1.tar.gz

编写redis一键安装脚本
[root@redis-node src]# cat redis_install.sh 
#!/usr/bin/env bash
# It's Used to be install redis.
# Created on 2018/08/27 11:18.
# @author: wangshibo.
# Version: 1.0
   
function install_redis () {
#################################################################################################
        cd /usr/local/src
        tar -zxvf /usr/local/src/redis-4.0.1.tar.gz
        cd redis-4.0.1
        make PREFIX=/usr/local/redis install
        mkdir -p /usr/local/redis/{etc,var}
        rsync -avz redis.conf  /usr/local/redis/etc/
        sed -i 's@pidfile.*@pidfile /var/run/redis-server.pid@' /usr/local/redis/etc/redis.conf
        sed -i "s@logfile.*@logfile /usr/local/redis/var/redis.log@" /usr/local/redis/etc/redis.conf
        sed -i "s@^dir.*@dir /usr/local/redis/var@" /usr/local/redis/etc/redis.conf
        sed -i 's/daemonize no/daemonize yes/g' /usr/local/redis/etc/redis.conf
        sed -i 's/^# bind 127.0.0.1/bind 0.0.0.0/g' /usr/local/redis/etc/redis.conf
 #################################################################################################
}
   
install_redis


赋予脚本执行权限,并进行安装
[root@redis-node src]# chmod 755 /usr/local/src/redis_install.sh
[root@redis-node src]# /bin/bash -x /usr/local/src/redis_install.sh


编写redis-server启动脚本
[root@redis-node src]# cat /etc/init.d/redis-server 
#!/bin/bash
#
# redis - this script starts and stops the redis-server daemon
#
# chkconfig:   - 85 15
# description:  Redis is a persistent key-value database
# processname: redis-server
# config:      /usr/local/redis/etc/redis.conf
# config:      /etc/sysconfig/redis
# pidfile:     /usr/local/redis/var/redis-server.pid
  
# Source function library.
. /etc/rc.d/init.d/functions
  
# Source networking configuration.
. /etc/sysconfig/network
  
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
  
redis="/usr/local/redis/bin/redis-server"
prog=$(basename $redis)
  
REDIS_CONF_FILE="/usr/local/redis/etc/redis.conf"
  
[ -f /etc/sysconfig/redis ] && . /etc/sysconfig/redis
  
lockfile=/var/lock/subsys/redis-server
  
start() {
    [ -x $redis ] || exit 5
    [ -f $REDIS_CONF_FILE ] || exit 6
    echo -n $"Starting $prog: "
    daemon $redis $REDIS_CONF_FILE
    retval=$?
    echo
    [ $retval -eq 0 ] && touch $lockfile
    return $retval
}
  
stop() {
    echo -n $"Stopping $prog: "
    killproc $prog
    retval=$?
    echo
    [ $retval -eq 0 ] && rm -f $lockfile
    return $retval
}
  
restart() {
    stop
    start
}
  
reload() {
    echo -n $"Reloading $prog: "
    killproc $redis -HUP
    RETVAL=$?
    echo
}
  
force_reload() {
    restart
}
  
rh_status() {
    status $prog
}
  
rh_status_q() {
    rh_status >/dev/null 2>&1
}
  
case "$1" in
    start)
        rh_status_q && exit 0
        $1
        ;;
    stop)
        rh_status_q || exit 0
        $1
        ;;
    restart)
        $1
        ;;
    reload)
        rh_status_q || exit 7
        $1
        ;;
    force-reload)
        force_reload
        ;;
    status)
        rh_status
        ;;
    condrestart|try-restart)
        rh_status_q || exit 0
            ;;
    *)
        echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload}"
        exit 2
esac


赋予脚本执行权限,并启动redis-server
[root@redis-node src]# /etc/init.d/redis-server start
[root@redis-node src]# lsof -i:6379
COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
redis-ser 24439 root    6u  IPv4 24642923      0t0  TCP dns02.kevin.cn:6379 (LISTEN)

温馨提示:
需要将redis.conf文件中的bind改为本机ip.不能使用默认的127.0.0.1,否则远程连接该redis就会失败
[root@redis-node src]# vim /usr/local/redis/etc/redis.conf
.....
bind 192.168.10.203

重启redis-server服务
[root@redis-node src]# /etc/init.d/redis-server restart
Stopping redis-server:                                     [  OK  ]
Starting redis-server:                                     [  OK  ]
[root@redis-node src]# lsof -i:6379
COMMAND    PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
redis-ser 8184 root    6u  IPv4 24688720      0t0  TCP dns02.kevin.cn:6379 (LISTEN)

最好在tomcat两个节点上使用"telnet 192.168.10.203 6379"验证下redis是否能成功连接

5) tomcat通过redis共享session测试

访问http://www.kevin.com,不断刷新页面,发现页面的其他信息改变,但是sessionid保持不变,即说明实现了session共享!

关闭两个节点中的任意一个tomcat服务,继续访问页面,发现sessionid任然保持不变!

然后在redis机器上查看有没有上面sessionid的key值存在

[root@redis-node ~]# /usr/local/redis/bin/redis-cli -h 192.168.10.203 -p 6379
192.168.10.203:6379> keys *
1) "F7FE0AE8BD1654584A330A804A7BE5FE"
192.168.10.203:6379>

========================================================================

上面采用的是redis单节点方式(主从模式),如果是redis集群方式,则采用"Nginx+Tomcat + Redis Sentinel",大致操作如下:

1) 部署Redis+redis集群,参考文档:https://www.cnblogs.com/kevingrace/p/9004460.html
2) redis集群方式的session共享配置为(conf/context.xml):
    <Valve className="com.orangefunction.tomcat.redissessions.RedisSessionHandlerValve" />
    <Manager className="com.orangefunction.tomcat.redissessions.RedisSessionManager"
             sentinels="192.168.9.223:30001,192.168.9.223:30002" />          #这里是redis哨兵的地址,多个sentinels集群就写多个ip地址
             maxInactiveInterval="60"
             sentinelMaster="mymaster"                                       #这里是sentinel的master名称，一定不要写错
             password="123456"                                               #还有认证密码

Tomcat+Redis Sentinel工作原理:

原此时我们连接的不在是redis主从,而是直接连接哨兵，哨兵会监测后端的redis主从，当主挂掉之后，会挑选一台从当主机，然后在自身的状态中更新，此时，就不用担心主挂掉之后，IP飘移的问题了，哨兵可以多个，当某个哨兵挂掉也不会影响。

网络拓扑图如下: