Etcd+Confd实现Nginx配置文件动态更新

2018-10-23

nginx

简介
安装
配置

简介

如上图是一个很简单的架构，生产环境中经常会进行灰度发布，需要下掉一部分的节点。如果靠人工操作很容易错误，这里通过Etcd和Confd来实现nginx upstream的动态更新

etcd: 分布式KV存储系统，一般用于共享配置和服务注册与发现

confd：管理本地应用配置文件，使用etcd或consul存储的数据渲染模板，还支持redis、zookeeper等, 通过watch定期监测对应的etcd中目录变化，获取最新的Value，然后渲染模板，更新配置文件

安装

安装etcd

1 2	yum -y install etcd systemctl start etcd

安装confd

1
2
3

wget https://github.com/kelseyhightower/confd/releases/download/v0.16.0/confd-0.16.0-linux-arm64
mkdir -p /etc/etcd/{conf.d,templates}
mv confd-0.16.0-linux-arm64 /usr/bin/confd

conf.d 目录存放.toml配置文件
templates 目录存放.tmpl配置模版文件

配置

创建nginx配置和模版

配置文件cat conf.d/test.conf.toml

[template]
src = "test.conf.tmpl"
dest = "/tmp/test.conf"
keys = [
    "/nginx",
]
check_cmd = "/usr/sbin/nginx -t -c {{.src}}"
reload_cmd = "/usr/sbin/nginx -s reload"

模版文件cat templates/test.conf.tmpl

upstream www_{{getv "/nginx/www/server/server_name"}} {
    {{range getvs "/nginx/www/upstream/*"}}
         server {{.}};
    {{end}}
}
server {
    server_name         {{getv "/nginx/www/server/server_name"}};
    location / {
        proxy_pass        http://www_{{getv "/nginx/www/server/server_name"}};
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_redirect    off;
    }
}

配置etcd

1
2
3

etcdctl set /nginx/https/www/server/server_name test.com
etcdctl set /nginx/https/www/upstream/server1 192.168.1.110
etcdctl set /nginx/https/www/upstream/server2 192.168.1.111

启动confd监听

1	confd -watch -backend etcd -node http://127.0.0.1:2379

查看生产的nginx配置文件

cat /tmp/test.conf
upstream www_test.com {
         server 192.168.1.110;
         server 192.168.1.111;
}
server {
    server_name         test.com;
    location / {
        proxy_pass        http://www_test.com;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_redirect    off;
    }
}

配置文件生成完成

上一篇：Flask任务调度APScheduler

下一篇：django分布式链路监控之Zipkin