Technology Sharing

Prometheus Grafana monitor Linux host

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

1. Install Prometheus

1.1 Download Prometheus

Download URL

https://github.com/prometheus/prometheus/releases
  • 1

Select the required version

wget https://github.com/prometheus/prometheus/releases/download/v2.53.0/prometheus-2.53.0.linux-amd64.tar.gz
  • 1

1.2. Install Prometheus software

1.2.1. Create a user group

groupadd prometheus
useradd -g prometheus -m -s /sbin/nologin prometheus
  • 1
  • 2

1.2.2, decompression configuration

tar -zxvf prometheus-2.53.0.linux-amd64.tar.gz
mv prometheus-2.53.0.linux-amd64 /usr/local/prometheus
sed -i 's/localhost/ 172.25.0.166/g' /usr/local/prometheus/prometheus.yml
  • 1
  • 2
  • 3

1.2.3. Add Prometheus to the system environment variables

vim /etc/profile
  • 1
export PROMETHEUS_HOME=/usr/local/prometheus
PATH=$PATH:$PROMETHEUS_HOME
export PATH
  • 1
  • 2
  • 3

Reload system environment variable file

source /etc/profile

  • 1
  • 2

1.2.4. Check the version information of Prometheus

prometheus --version	
  • 1

1.2.5. Create a data directory

mkdir /usr/local/prometheus/data/
  • 1

1.2.6、Permission modification

chown -R prometheus.prometheus /usr/local/prometheus
  • 1

1.2.7. Check and load the configuration file

cd /usr/local/prometheus/
./promtool check config prometheus.yml
  • 1
  • 2

1.2.8. Create systemd service

cat <<EOF > /usr/lib/systemd/system/prometheus.service
[Unit]
Description=prometheus
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml --storage.tsdb.path=/usr/local/prometheus/data/ --web.enable-lifecycle --storage.tsdb.retention.time=30d
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12

Parameter Notes:

  • config.file: specifies the configuration file
  • web.enable-lifecycle: supports reloading configuration via http request
  • storage.tsdb.path: specifies the data storage directory (the default is the data directory of the current directory, if it does not exist, create a new one)
  • storage.tsdb.retention.time: specifies the data retention time (default 15d)

1.2.9. Start the service

systemctl daemon-reload
systemctl start prometheus
systemctl status prometheus && systemctl enable prometheus
  • 1
  • 2
  • 3

1.2.10. Confirm that the port is being monitored

ss -lnput | grep 9090
  • 1
tcp     LISTEN   0        512                    *:9090                *:*       users:(("prometheus",pid=20863,fd=7))
  • 1

1.2.11. Check and load the configuration file

http://172.25.0.166:9090/metrics
  • 1

insert image description here


2. Monitor Linux hosts

2.1. Install the exporter plugin on the Linux host

2.1.1、Download node_exporter

Download URL

https://github.com/prometheus/node_exporter/releases
  • 1

Download the required version

wget https://github.com/prometheus/node_exporter/releases/download/v1.8.1/node_exporter-1.8.1.linux-amd64.tar.gz
  • 1

2.1.2. Create a user group

groupadd prometheus
useradd -g prometheus -m -s /sbin/nologin prometheus
  • 1
  • 2

2.1.3. Install and decompress

tar -zxvf node_exporter-1.8.1.linux-amd64.tar.gz -C /usr/local/
cd /usr/local/
mv node_exporter-1.8.1.linux-amd64  node_exporter
  • 1
  • 2
  • 3

2.1.4、Permission modification

chown -R prometheus.prometheus /usr/local/node_exporter
  • 1

2.1.5. Add the node_exporter executable file directory to the environment variable

vim /etc/profile
  • 1
export NODE_EXPORTER=/usr/local/node_exporter
export PATH=$PATH:$NODE_EXPORTER
export PATH
  • 1
  • 2
  • 3

Reload system environment variable file

source /etc/profile
  • 1

2.1.6. Create systemd service

cat > /usr/lib/systemd/system/node_exporter.service <<EOF

[Unit]
Description=node_export
Documentation=https://github.com/prometheus/node_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/node_exporter/node_exporter
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14

2.1.7. Start the service

systemctl daemon-reload
systemctl enable node_exporter && systemctl start node_exporter
systemctl status node_exporter
  • 1
  • 2
  • 3

2.1.8. Confirm that the port is being monitored

ss -lnput | grep 9100
  • 1
tcp     LISTEN   0        512                    *:9100                *:*       users:(("node_exporter",pid=21904,fd=3))
  • 1

2.2. Add the exporter plug-in of the Linux host to the Prometheus software service

Modify the Prometheus configuration file

vim /usr/local/prometheus/prometheus.yml
  • 1
scrape_configs:
... 
  - job_name: 'Linux Node'					# 添加监控项的名字
    static_configs:							
      - targets: ['172.25.0.166:9100']		# 监控主机的ip地址和端口号
  • 1
  • 2
  • 3
  • 4
  • 5

Restart the prometheus service

systemctl restart prometheus
  • 1

2.3. Test Prometheus's monitoring of Linux host status

http://172.25.0.166:9090/targets?search=&scrapePool=Linux+Node
  • 1

insert image description here

http://172.25.0.166:9100/metrics
  • 1

insert image description here


3. Use Grafana to display data

3.1. Download grafana

Download URL

https://grafana.com/grafana/download
  • 1

Download the required version

wget  https://dl.grafana.com/enterprise/release/grafana-enterprise-11.1.0-1.x86_64.rpm
  • 1

3.2. Install grafana

rpm -ivh grafana-enterprise-11.1.0-1.x86_64.rpm
  • 1

3.3. Start grafana

systemctl daemon-reload 
systemctl enable grafana-server && systemctl start grafana-server
  • 1
  • 2

3.4、Grafana Chinese version

vim /etc/grafana/grafana.ini
  • 1
[users]
# 设置默认语言为中文 
default_language = zh-Hans
  • 1
  • 2
  • 3

Restart the service

systemctl restart grafana-server
  • 1

3.5. Download and import grafana template

https://grafana.com/grafana/dashboards/15172-node-exporter-for-prometheus-dashboard-based-on-11074/
  • 1

insert image description here

3.6. Load the grafana-piechart-panel plugin

grafana-cli plugins install grafana-piechart-panel
  • 1

or

tar -xvf grafana-piechart-panel.tar.gz
mv grafana-piechart-panel /var/lib/grafana/plugins/grafana-piechart-panel
  • 1
  • 2

3.7. Browser access

insert image description here


4. References

https://prometheus.io/
https://blog.csdn.net/heian_99/article/details/126989356
https://www.cnblogs.com/saneri/p/14667301.html
https://blog.csdn.net/qq_31725371/article/details/114697770