[Promethus] Prometheus 로 서버 모니터링 구축 - 설치
참고 : https://github.com/prometheus/prometheus
1-1. 설치(binary install )
prometheus 설치용 디렉토리 생성 후
git clone https://github.com/prometheus/prometheus.git
cd prometheus
make build
./prometheus --config.file=your_config.yml
1.1[troubleshooting]make 명령 실행 시 "make: go: Command not found" error 발생 시
1.1.1 현상
[root@server prometheus]# make build
make: go: Command not found
GOOS= GOARCH= go get -u github.com/prometheus/promu
/bin/sh: go: command not found
make: *** [promu] Error 127
1.1.2 해결방법
prometheus 은 golang 으로 제작되어 있다.
make 명령어 수행을 위해 서버 내 golang 설치가 필요하다.
'yum install go' 로 golang 설치 진행
[root@nbsoft prometheus]# yum install go
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
178 packages excluded due to repository priority protections
Resolving Dependencies
... 중략 ...
=================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================
Installing:
golang x86_64 1.9.4-1.amzn2.0.2 amzn2-core 611 k
Installing for dependencies:
cpp x86_64 7.3.1-5.amzn2.0.2 amzn2-core 9.2 M
gcc x86_64 7.3.1-5.amzn2.0.2 amzn2-core 22 M
glibc-devel x86_64 2.26-28.amzn2.0.1 amzn2-core 986 k
glibc-headers x86_64 2.26-28.amzn2.0.1 amzn2-core 505 k
golang-bin x86_64 1.9.4-1.amzn2.0.2 amzn2-core 48 M
golang-src noarch 1.9.4-1.amzn2.0.2 amzn2-core 5.1 M
kernel-headers x86_64 4.14.72-73.55.amzn2 amzn2-core 1.1 M
libatomic x86_64 7.3.1-5.amzn2.0.2 amzn2-core 40 k
libcilkrts x86_64 7.3.1-5.amzn2.0.2 amzn2-core 80 k
libitm x86_64 7.3.1-5.amzn2.0.2 amzn2-core 79 k
libmpc x86_64 1.0.1-3.amzn2.0.2 amzn2-core 52 k
libmpx x86_64 7.3.1-5.amzn2.0.2 amzn2-core 46 k
libquadmath x86_64 7.3.1-5.amzn2.0.2 amzn2-core 186 k
libsanitizer x86_64 7.3.1-5.amzn2.0.2 amzn2-core 649 k
mpfr x86_64 3.1.1-4.amzn2.0.2 amzn2-core 208 k
Transaction Summary
=================================================================================================================================================
Install 1 Package (+15 Dependent packages)
1-2 설치(ansible-prometheus 활용)
https://github.com/cloudalchemy/demo-site
ceps 와 마찬가지도 최근 유행하는 대부분의 open source 는 ansible 을 활용한 설치 방법이 잘 제공되고 있다.
Windows 의 install wizard 와 비슷한 느낌이다.
Ansible playbook download
mkdir prometheus-ansible
cd prometheus-ansible
git clone https://github.com/cloudalchemy/demo-site.git
cd demo-site
host inventory 수정 : 여기서는 server 와 client 모두 localhost 로 설정
[root@nbsoft demo-site]# cat hosts
demo ansible_host=demo.cloudalchemy.org ansible_user=root
#influx ansible_host=influx.cloudalchemy.org ansible_user=root
[prometheus]
localhost
[grafana]
localhost
[web]
localhost
ansible-galaxy 로 exporter, grafana 등 ansible 공식 저장소에 등록된 role 파일 다운
ansible-galaxy install -r roles/requirements.yml
group_vars/all/vault 파일 내 granfa password 지정
vault_grafana_password: <<INSERT_YOUR_GRAFANA_PASSWORD>>
group_vars/all/vars 파일 내 ansible_host 변수 설정
ansible_host : "127.0.0.1"
ansible-playbook으로 site.yml 실행
: Client 가 될 node_exporter, Prometheus 수집 서버가 될 prometheus, Dashboard 역할을 할 grafana 를 inventory 에서 지정한다.
- name: Deploy node_exporter
hosts: all
roles:
- cloudalchemy.node-exporter
tags:
- node_exporter
- name: Setup core monitoring software
hosts: prometheus
roles:
- cloudalchemy.blackbox-exporter
- cloudalchemy.snmp-exporter
- cloudalchemy.prometheus
- cloudalchemy.alertmanager
tags:
- prometheus
- name: Deploy grafana
hosts: grafana
roles:
- cloudalchemy.grafana
tags:
- grafana
설치 확인 방법 :
모니터링 대상 exporter 확인 : localhost의 9100 포트로 접속하여 metric을 확인한다.
[root@server demo-site]# curl http://localhost:9100
<html>
<head><title>Node Exporter</title></head>
<body>
<h1>Node Exporter</h1>
<p><a href="/metrics">Metrics</a></p>
</body>
</html>
prometheus 서버 확인 : localhost의 9090 포트로 접속하여 확인
[root@server demo-site]# curl http://localhost:9090
<a href="/graph">Found</a>.
[root@server demo-site]# systemctl status prometheus
● prometheus.service - Prometheus
Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2018-10-28 13:58:46 KST; 1min 37s ago
Main PID: 30931 (prometheus)
CGroup: /system.slice/prometheus.service
└─30931 /usr/local/bin/prometheus --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus --storage.ts...
Oct 28 13:58:46 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:46.989960595Z caller=main.go:242 vm_limits="(soft=unlimited, ...imited)"
Oct 28 13:58:46 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:46.990683801Z caller=main.go:554 msg="Starting TSDB ..."
Oct 28 13:58:47 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:47.003881935Z caller=web.go:397 component=web msg="Start list...0.0:9090
Oct 28 13:58:47 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:47.15982091Z caller=main.go:564 msg="TSDB started"
Oct 28 13:58:47 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:47.16023552Z caller=main.go:624 msg="Loading configuration fi...heus.yml
Oct 28 13:58:47 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:47.167632019Z caller=main.go:650 msg="Completed loading of co...heus.yml
Oct 28 13:58:47 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:47.167935447Z caller=main.go:523 msg="Server is ready to rece...quests."
Oct 28 13:58:47 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:47.374391069Z caller=main.go:624 msg="Loading configuration f...heus.yml
Oct 28 13:58:47 nbsoft prometheus[30931]: level=info ts=2018-10-28T04:58:47.375627109Z caller=main.go:650 msg="Completed loading of co...heus.yml
Oct 28 13:58:47 nbsoft systemd[1]: Reloaded Prometheus.
Hint: Some lines were ellipsized, use -l to show in full.
[root@nbsoft demo-site]#
