分类云服务下的文章

August 23, 2018

阿里云云解析ddns的自动更新shell脚本

今天需要用到阿里云云解析做ddns，网上搜到到都是基于阿里云python sdk的脚本，这么一个简单功能没必要用python，找到一个openwrt的shell脚本，拿来改了一下运行环境，和环境变量，在x86下ubuntu 16.04测试通过。

原脚本地址https://github.com/h46incon/AliDDNSBash ，在此感谢

我修改过的脚本地址https://github.com/trepwq/aliyunddns

使用方法
安装依赖
首先需要一个shell

然后安装 bind-dig，curl，openssl-util。

修改脚本的setting代码段
其中DomainRecordId不清楚的话暂时不用修改，DNSServer修改为你在万网上使用的DNS服务器。如:

AccessKeyId="MyID"
AccessKeySec="MySecret"
DomainRecordId="00000"
DomainRR="www"
DomainName="example.com"
DomainType="A"
DNSServer="dns9.hichina.com"

如果不清楚DomainRecordId的话，修改main函数，在里面调用describe_record，如：

main()
    {
    describe_record
    #update_record
    }

然后执行这个脚本。如果没问题的话，就能获取到域名的所有解析记录的列表了：

{"PageNumber":1,"TotalCount":1,"PageSize":1,"RequestId":"0000","DomainRecords":{"Record":[{"RR":"www","Status":"ENABLE","Value":"8.8.8.8","RecordId":"21332133","Type":"A","DomainName":"example.com","Locked":false,"Line":"default","TTL":"600"},]}}HttpCode:200

上面的结果中，RecordId为21332133。得到结果后再修改DomainRecordId为正确的值。

修改main函数：

main()
{
    #describe_record
    update_record
}

执行脚本即可。脚本会在本机IP地址和当前域名解析设置不同的时候调用API更新设置。

最后，在crontab里添加每分钟执行一次即可

crontab -e
* * * * * /root/ddns.sh >>/dev/null 2>&1

July 25, 2018

阿里云日志服务每天自动发送统计http 4xx 5xx数量

借助阿里云日志服务cli还可以干更多，比如统计每天http 4xx 和 5xx数量。
这个脚本重点在awk的使用，提取字段和拼接字段。

www4xx5xx.sh

#! /bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:~/bin
export PATH
ALIYUN_LOG_CLI_ACCESSID=
ALIYUN_LOG_CLI_ACCESSKEY=
ALIYUN_LOG_CLI_ENDPOINT=cn-beijing.log.aliyuncs.com
export ALIYUN_LOG_CLI_ACCESSID
export ALIYUN_LOG_CLI_ACCESSKEY
export ALIYUN_LOG_CLI_ENDPOINT


function portal4xx(){
    todaytime=$(date "+%Y-%m-%d %H:%M:%S %Z")
    yesterdaytime=$(date -d '1 days ago' "+%Y-%m-%d %H:%M:%S %Z")
    current4xx=$(aliyun log get_log_all --project="www" --logstore="www-nginx-access" --query="status>=400 and status<500| select status,count(*)  as count group by status ORDER BY status ASC" --from_time="$yesterdaytime" --to_time="$todaytime" --jmes-filter="join('\n', map(&to_string(@), @))" |awk -F "[\"]" '{print $10":"$12","$2":"$4}' |awk '{{printf"%s\\n",$0}}')
    curl -H "Content-type: application/json" -X POST -d '{"text": "last 24h 4xx stats:\n'"${current4xx}"'"}' https://example/incoming/xxx
}

function portal5xx(){
    todaytime=$(date "+%Y-%m-%d %H:%M:%S %Z")
    yesterdaytime=$(date -d '1 days ago' "+%Y-%m-%d %H:%M:%S %Z")
    current5xx=$(aliyun log get_log_all --project="www" --logstore="www-nginx-access" --query="status>=500 and status<600| select status,count(*)  as count group by status ORDER BY status ASC" --from_time="$yesterdaytime" --to_time="$todaytime" --jmes-filter="join('\n', map(&to_string(@), @))" |awk -F "[\"]" '{print $10":"$12","$2":"$4}' |awk '{{printf"%s\\n",$0}}')
    curl -H "Content-type: application/json" -X POST -d '{"text": "last 24h 5xx stats:\n'"${current5xx}"'"}' https://example.com/incoming/xxx
}
portal4xx
portal5xx

上面脚本中比较折腾人的是要把下面格式写成json格式发出去

"count":"2171" "status":"403"
"count":"3529" "status":"404"
"count":"283" "status":"499"

把上面格式写成json如下

{"text": "last 24h 5xx stats:\ncount:2171,status:403\ncount:3529,status:404\ncount:283,status:499"}

双引号去掉容易，难的是把多行间真实的换行替换成字符串\n，这里使用awk '{{printf"%s\n",$0}}'

July 25, 2018

阿里云负载均衡访问控制（黑名单）自动添加IP的方法（续）

https://www.willnet.net/index.php/archives/105/
上文使用了阿里云日志服务的告警，由于告警有一些局限，比如最长只能统计60分钟的日志，最短查询间隔5分钟等，现在使用阿里云日志服务的cli替代告警。
首先需要安装日志服务cli

pip install -U aliyun-log-cli

然后编写脚本
autoaddblacklist.sh

#! /bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:~/bin
export PATH
ALIYUN_LOG_CLI_ACCESSID=
ALIYUN_LOG_CLI_ACCESSKEY=
ALIYUN_LOG_CLI_ENDPOINT=cn-beijing.log.aliyuncs.com
export ALIYUN_LOG_CLI_ACCESSID ALIYUN_LOG_CLI_ACCESSKEY ALIYUN_LOG_CLI_ENDPOINT
function autoaddblacklist(){
    nowtime=$(date "+%Y-%m-%d %H:%M:%S %Z")
        last5minutestime=$(date -d '5 minutes ago' "+%Y-%m-%d %H:%M:%S %Z")
    searchresult=$(aliyun log get_log_all --project="www" --logstore="www-nginx-access" --query="* | select remote_addr,count(*)  as count group by remote_addr order by count desc limit 1" --from_time="$last5minutestime" --to_time="$nowtime" --jmes-filter="join('\n', map(&to_string(@), @))")
    lastip=$(cat /root/aliyun/tmp/lastip.txt)
    currentip=$(echo $searchresult | awk -F "[\"]" '{print $16}')
    count=$(echo $searchresult | awk -F "[\"]" '{print $4}')
    if [ "$count" -gt 1000 ] && [ "$currentip"x != "$lastip"x ] ;then
        echo $currentip >/root/aliyun/tmp/lastip.txt
        echo $currentip|xargs python /root/aliyun/scripts/addaclentry.py >> /var/log/aliyun-log-alarm/www/autoaddblacklist.log 2>&1
        curl -H "Content-type: application/json" -X POST -d '{"text": "found ip:'"${currentip}"' request: '"${count}"' over 1000 in 5min,added to the www blacklist."}' https://example.com/incoming/xxxxx-xxxx-xxxx
    else
        date "+%Y-%m-%d %H:%M:%S" >> /var/log/aliyun-log-alarm/www/autoaddblacklist.log && echo "nothing to do" >> /var/log/aliyun-log-alarm/www/autoaddblacklist.log
    fi
}

autoaddblacklist

按需求定时执行上面的脚本就行了

July 16, 2018

阿里云负载均衡访问控制（黑名单）自动添加IP的方法

前文写了一个添加ip的python脚本，https://www.willnet.net/index.php/archives/104/
现在做一个自动添加方案，以阿里云日志服务收集nginx日志为例，目的是自动屏蔽每15分钟请求超过2500次的IP地址
1，阿里云日志服务新建搜索

* | select remote_addr,count(*)  as count group by remote_addr order by count desc limit 1

保存上面的搜索为快速查询
2，配置nginx新站点，反代8088端口，配置日志输出为
nginx.conf

log_format log-alarm-access '$request_body '

example.com.conf

server {
listen 80;
server_name example.com;
index index.html index.htm index.php;
  if ($http_x_forwarded_proto = "http") {
    return 307 https://$host$request_uri;
  }


location /requestoverlimit {
  proxy_pass         http://localhost:8088;
  proxy_redirect     off;
  proxy_set_header   Host              $host;
  proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
  proxy_set_header   X-Forwarded-Proto $scheme;
  access_log /var/log/nginx/example.com.access.log log-alarm-access;
}
}

3，新建python脚本
server.py

#!/usr/bin/env python
"""
Very simple HTTP server in python.
Usage::
    ./dummy-web-server.py [<port>]
Send a GET request::
    curl http://localhost
Send a HEAD request::
    curl -I http://localhost
Send a POST request::
    curl -d "foo=bar&bin=baz" http://localhost
"""
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import SocketServer

class S(BaseHTTPRequestHandler):
    def _set_headers(self):
        self.send_response(200)
        self.send_header('Content-type', 'text/html')
        self.end_headers()

    def do_GET(self):
        self._set_headers()
        self.wfile.write("<html><body><h1>hi!</h1></body></html>")

    def do_HEAD(self):
        self._set_headers()
        
    def do_POST(self):
        # Doesn't do anything with posted data
        self._set_headers()
        self.wfile.write("OK")
        
def run(server_class=HTTPServer, handler_class=S, port=8088):
    server_address = ('', port)
    httpd = server_class(server_address, handler_class)
    print 'Starting httpd...'
    httpd.serve_forever()

if __name__ == "__main__":
    from sys import argv

    if len(argv) == 2:
        run(port=int(argv[1]))
    else:
        run()

4，启动python server.py，并添加服务器启动命令到rc.local开机启动
5，阿里云日志服务把1里面保存的快速查询另存为告警，webhook地址填上面新建的网址，如图

6，新建定时任务，每五分钟执行

*/5 * * * *  tail -n 1 /var/log/nginx/example.com.access.log|awk -F ":" '{print $9}'|awk -F "]" '{print $1}'|xargs python /root/aliyun/scripts/addaclentry.py >> /var/log/autoaddblacklist/autoaddblacklist.log 2>&1

7，查看访问控制黑名单列表

ps:一键脚本，对比上一次IP地址是否已经添加到黑名单

#! /bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:~/bin
export PATH

function sameip(){
        lastip=$(cat /root/aliyun/server/lastip.txt)
        currentip=$(tail -n 1 /var/log/nginx/example.com.access.log|awk -F ":" '{print $12}'|awk -F "]" '{print $1}')
        if [ "$currentip"x != "$lastip"x ];then
                echo $currentip >/root/aliyun/server/lastip.txt
                cat /root/aliyun/server/lastip.txt|xargs python /root/aliyun/scripts/addaclentry.py >> /var/log/autoaddblacklist/autoaddblacklist.log 2>&1
        else
                date "+%Y-%m-%d %H:%M:%S" >> /var/log/autoaddblacklist/autoaddblacklist.log && echo "same ip address" >> /var/log/autoaddblacklist/autoaddblacklist.log
        fi
}

sameip

crontab定时，每分钟执行一次

* * * * * /root/aliyun/scripts/sameip.sh

July 13, 2018

阿里云负载均衡访问控制（黑名单）添加IP的脚本

python写的阿里云负载均衡访问控制添加IP的脚本，配合日志或报警执行自动添加ip。
需要先安装阿里云sdk

pip install aliyun-python-sdk-core
pip install aliyun-python-sdk-slb

addaclentry.py

#!/usr/bin/env python
#coding=utf-8

import sys
import json
from aliyunsdkcore import client
from aliyunsdkslb.request.v20140515 import AddAccessControlListEntryRequest
from aliyunsdkslb.request.v20140515 import DescribeAccessControlListAttributeRequest

clt = client.AcsClient('AccessKeyId','secret','cn-beijing')


AclEntryIP = [{u'entry': u'', u'comment': u''}]

AclEntryIP[0]["entry"] = sys.argv[1]+'/32'

AclEntryIP = json.dumps(AclEntryIP)

request = AddAccessControlListEntryRequest.AddAccessControlListEntryRequest()
request.set_accept_format('json')

request.add_query_param('RegionId', 'cn-beijing')
request.add_query_param('AclEntrys', AclEntryIP)
request.add_query_param('AclId', 'acl-fdsafdsafdsafdsaf')


## 发起请求
response = clt.do_action_with_exception(request)


# 查询
## 设置参数
request = DescribeAccessControlListAttributeRequest.DescribeAccessControlListAttributeRequest()
request.set_accept_format('json')

request.add_query_param('RegionId', 'cn-beijing')
request.add_query_param('AclId', 'acl-fdsafdsafdsafdsaf')

## 发起请求
response = clt.do_action_with_exception(request)

## 输出结果
print time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
print "黑名单列表：", response, '\n'

使用方法

python addaclentry.py 8.8.8.8