Rabbitmq consumers disappear 消费者无故消失

背景：

线上使用 rabbitmq，consumers 经常无故消失

查找问题

1. 先是查看代码，发现 consumer 消费逻辑都没有报错，也没有发现 connection close 的情况，未发现问题所在

2. 查 rabbitmq log

进入 /var/log/rabbitmq 目录，查看log
请填写图片描述

发现出现问题时都会有这个 error, 消费者执行时间过长，未交付确认，导致超时：

2021-12-01 10:15:51.084 [warning] <0.5900.100> Consumer ctag-/home/deploy/apps/simking_go_task/current/simking_go_task-16 on channel 1 has timed out waiting for delivery acknowledgement. Timeout used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more
2021-12-01 10:15:51.085 [error] <0.5900.100> Channel error on connection <0.5891.100> (127.0.0.1:47526 -> 127.0.0.1:5672, vhost: '/', user: 'simking'), channel 1:
operation none caused a channel exception precondition_failed: delivery acknowledgement on channel 1 timed out. Timeout value used: 1800000 ms. This timeout value can be configured, see consumers doc guide to learn more
2021-12-01 10:56:57.447 [warning] <0.5904.100> closing AMQP connection <0.5904.100> (127.0.0.1:47528 -> 127.0.0.1:5672, vhost: '/', user: 'simking'):
client unexpectedly closed TCP connection
2021-12-01 10:56:57.447 [warning] <0.5891.100> closing AMQP connection <0.5891.100> (127.0.0.1:47526 -> 127.0.0.1:5672, vhost: '/', user: 'simking'):
client unexpectedly closed TCP connection
2021-12-01 10:56:57.447 [info] <0.6980.100> Closing all channels from connection '127.0.0.1:47528 -> 127.0.0.1:5672' because it has been closed
2021-12-01 10:56:57.447 [info] <0.6982.100> Closing all channels from connection '127.0.0.1:47526 -> 127.0.0.1:5672' because it has been closed

根据错误及官网的文档(https://www.rabbitmq.com/consumers.html#acknowledgement-timeout) 发现：

请填写图片描述

在现代 RabbitMQ 版本中，对消费者交付确认强制执行超时。这有助于检测从不确认交付的有问题（卡住）的消费者。此类消费者可能会影响节点的磁盘数据压缩，并可能会导致节点耗尽磁盘空间。
如果消费者未确认其交付超过超时值（默认为 30 分钟），则其通道将关闭，并显示PRECONDITION_FAILED通道异常。该错误将由消费者连接到的节点记录。

3. 发现问题所在，现在就是修改配置

找 rabbitmq 配置文件修改，根据官网对于配置文件说明
https://www.rabbitmq.com/configure.html#config-location
https://www.rabbitmq.com/configure.html#verify-configuration-config-file-location
发现 /etc/rabbitmq/ 文件夹下并没有配置文件
sudo rabbitmqctl status 执行发现 config files 是空的，rabbitmq 安装默认不创建配置文件:
请填写图片描述

在官网 https://www.rabbitmq.com/configure.html#example-config 可以找到配置文件示例:
请填写图片描述

这里使用 rabbitmq.conf.example 配置示例文件链接： https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/docs/rabbitmq.conf.example

进入 /etc/rabbitmq 目录创建配置文件 rabbitmq.conf
修改参数 consumer_timeout

# 修改为 72小时
consumer_timeout = 259200000

5. 重新启动 rabbitmq

# 停止
sudo  systemctl stop rabbitmq-server
# 启动
sudo systemctl start rabbitmq-server

5. 查看状态

图中配置文件显示正确
请填写图片描述

使用命令 sudo rabbitmqctl eval 'application:get_all_env(rabbit).' 查看配置：
请填写图片描述