运维管理全过程中,许多情况下,业务流程运用会发生诈死的状况,运用过程一切正常,可是没法保证服务项目,这时监控过程没有实际意义,就必须监控接口
接口监控的方式 许多,可以用链接监控,可以写脚本开展监控
因为监控总体选用的是prometheus,因此在这里就立即用blackbox_exporter来做接口的监控
blackbox_exporter可以根据http、https、dns、tcp、ICMP对target开展探测,dns、tcp、ICMP都相对性简易,我这边主要是要监控一个登陆接口,因此得用https来开展探测,实际配备方式如下所示
布署blockbox_exporter
#建立blackbox_exporter的文件目录cd/usr/local/blackbox_exporter#下载blackbox_exporter的包wgethttps://github.com/prometheus/blackbox_exporter/releases/download/v0.19.0/blackbox_exporter-0.19.0.linux-amd64.tar.gz#缓解压力tar-zxvfblackbox_exporter-0.19.0.linux-amd64.tar.gz#重新命名mvblackbox_exporter-0.19.0.linux-amd64.tar.gzblackbox_exporter
用systemd管理方法blackbox_exporter
cat>/etc/systemd/system/blackbox_exporter.service<<"EOF"[Unit]Description=BlackboxExporterWants=network-online.targetAfter=network-online.target[Service]User=rootExecStart=/usr/local/blackbox_exporter/blackbox_exporter/blackbox_exporterRestart=on-failure[Install]WantedBy=default.targetEOF
针对blackbox_exporter管理方法得话,也有许多方式,你能立即nohup后台管理运行,还可以根据supervisor开展运行,我习惯性用systemd开展管理方法
随后加上开机自启动
systemctldaemon-reloadsystemctlenableblackbox_exportersystemctlstartblackbox_exporter
由于没特定日志輸出,因此是輸出到message日志中
默认设置带的blacbox.yml仅仅个非常简单配备,不可以满足需求,因此配备blackbox.yml中,加上一个控制模块
modules:xhj_login:#模块名字,prometheus环境变量时要配对prober:http#协议书timeout:30s#请求超时時间http:#控制模块的收集协议书method:POST#http要求的方式preferred_ip_protocol:"ip4"#应用的ipv4协议书headers:#配备post请求的header头Content-Type:application/jsonbody:''{"mobile":"13572801829","password":"ZWB123wyl"}''#post请求主要参数
配备进行后,储存blackbox.yml,运行blackbox_exporter
systemctlstartblackbox_exportersystemctlstatusblackbox_exporter●blackbox_exporter.service-BlackboxExporterLoaded:loaded(/etc/systemd/system/blackbox_exporter.service;enabled;vendorpreset:disabled)Active:active(running)sinceTue2022-01-0421:33:28CST;6sagoMainPID:24679(blackbox_export)Tasks:7Memory:1.9MCGroup:/system.slice/blackbox_exporter.service└─24679/data/prometheus/blackbox_exporter/blackbox_exporter/blackbox_exporterJan0421:33:28systemd[1]:StartedBlackboxExporter.Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.173Zcaller=main.go:224msg="Startingblackbox_exporter"version="(v...33d1ed0)"Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.173Zcaller=main.go:225build_context="(go=go1.16.4,user=root@2b025...2:56:44)"Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.173Zcaller=main.go:237msg="Loadedconfigfile"Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.174Zcaller=main.go:385msg="Listeningonaddress"address=:9115Jan0421:33:28blackbox_exporter[24679]:level=infots=2022-01-04T13:33:28.174Zcaller=tls_config.go:191msg="TLSisdisabled."http2=falseHint:Somelineswereellipsized,use-ltoshowinfull.
配备prometheus
#blackbox-job_name:"blackbox"metrics_path:/probeparams:module:[xhj_login]static_configs:-targets:-https://xxx.aaa.com/api/pc/user/login/passwordrelabel_configs:-source_labels:[._address__]target_label:._param_target-source_labels:[_param_target]target_label:instance-target_label:__address._replacement:172.17.0.1:9115……省去……省略
配备进行后,储存,热载入prometheus环境变量
curl-XPOSThttp://localhost:9090/-/reload
载入成功后,根据prometheus的UI查询下targets
这时实际上可以根据被要求端日志就可以见到blackbox_exporter进行的要求了,大家还可以根据curl要求blackbox_exporter来实现查询
#留意这儿用&标记连接好几个主要参数,必须转译curlhttp://172.17.0.1:9115/probe?target=https://xxx.aaa.com/api/pc/user/login/password\&module=xhj_login\&debug=true#查询回到結果Logsfortheprobe:ts=2022-01-04T14:10:32.979231489Zcaller=main.go:320module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Beginningprobe"probe=httptimeout_seconds=30ts=2022-01-04T14:10:32.979411891Zcaller=http.go:335module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Resolvingtargetaddress"ip_protocol=ip4ts=2022-01-04T14:10:32.986112778Zcaller=http.go:335module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Resolvedtargetaddress"ip=1.1.1.1ts=2022-01-04T14:10:32.986225541Zcaller=client.go:251module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="MakingHTTPrequest"url=https://1.1.1.1/api/pc/user/login/passwordhost=xxx.aaa.comts=2022-01-04T14:10:33.05701057Zcaller=main.go:130module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="ReceivedHTTPresponse"status_code=200ts=2022-01-04T14:10:33.057095911Zcaller=main.go:130module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Responsetimingsforroundtrip"roundtrip=0start=2022-01-04T22:10:32.986352765 08:00dnsDone=2022-01-04T22:10:32.986352765 08:00connectDone=2022-01-04T22:10:32.992658409 08:00gotConn=2022-01-04T22:10:33.038333687 08:00responseStart=2022-01-04T22:10:33.056951457 08:00tlsStart=2022-01-04T22:10:32.992701614 08:00tlsDone=2022-01-04T22:10:33.038235019 08:00end=2022-01-04T22:10:33.057074505 08:00ts=2022-01-04T14:10:33.057187486Zcaller=main.go:320module=xhj_logintarget=https://xxx.aaa.com/api/pc/user/login/passwordlevel=infomsg="Probesucceeded"duration_seconds=0.077883376
能够看见,http_status_code 200,probe_success为1,浏览一切正常
加上报警
建立rules
groups:-name:blackbox_networks_statsrules:-alert:PC登陆插口常见故障expr:probe_success==0for:1mlabels:severity:criticalannotations:summary:"PC登陆插口出现异常,没法正常登录,请立即查询!"description:"PC登陆插口出现异常,没法正常登录,请立即查询!"
建立进行后,热载入prometheus配备,随后查看prometheus的rules
Grafana加上监控图标
监管配备完,务必得配个数据可视化车内仪表盘,立即用5345模版导进
车内仪表盘中表明插口情况、是不是应用SSL、包含SSL证书到期時间、插口状态码、接口要求時间和DNS分析時间