分类 技术 下的文章

Routeros基于shadowsocks的国际多线分流

以下内容未完全实践。

一般iplc线路机场都会提供多条不通地域的线路,比如京德,沪美,沪日,深港,深新等。
以上每条线路基本都是物理上的最优线路,比如京德线路可以做到北京到欧洲120ms左右,而通过大家常用的香港出口到欧洲的话大部分是绕道美国,少部分走东南亚中东进入欧洲,前者需要250ms+,后者150ms+,还有对于大陆北方用户来说,通过沪美线路到美国比通过香港出口到美国延迟要少30ms以上。
所以如果能充分利用这些线路,能大大优化翻墙体验,自己当宽带运营商,想怎么走就怎么走。

要实现多出口分流,需要用到策略路由,以下以routeros为主路由,openwrt为旁路由,使用深港作默认线路,欧洲走京德,北美走沪美。

首先需要获取世界各地ip地址,比如欧洲都有哪些IP地址,这些地址走京德线路,北美都有哪些ip,需要走沪美线路等,我在之前文章routeros配置vpn分流大陆ip 中介绍过怎么获取大陆ip地址列表,可以通过类似方法获得。由于是分国别IP,比如欧洲列表需要合并各国IP,减少条目,加快路由查询效率。此处为难点,涉及到很多问题,比如土耳其的IP不是欧洲,但是走京德线路比深港快,对比来说,俄罗斯远东ip走京德的话会绕很长的回头路,所以IP地址的界线在哪,怎么优化,由于合并了IP地址怎么后期添加等等,所以一个这是个难点。

使用三个openwrt做旁路由,openwrt-1使用京德线路,openwrt-2使用沪美线路,openwrt-0使用深港线路

配置策略路由,目的地址是欧洲IP转发给openwrt-1,目的地址是北美IP转发给openwrt-2,目的地址是大陆ip的走默认网关,其他IP地址转发给openwrt-0

还有dns问题,怎么高效的使用全球cdn

在此抛砖引玉,欢迎大家交流分享。

ios/ipados虚拟机utm的免越狱安装方法

使用utm官方推荐的altstore来安装https://getutm.app/install/

1,https://altstore.io/ ,下载altserver,并启动
2,https://altstore.io/ ,点击右上角Patreon,注册并加入altsrore开发者rileytestut的测试计划
3,有线连接iphone或者ipad到电脑,在finder里打开wifi同步
4,电脑菜单栏点击altserver图标,选择安装mail plug-in,打开mail程序,设置里启用插件
5,电脑菜单栏点击altserver图标,选择install altstore,会提示你输入apple id,之后会自动安装
6,ipad上,设置里面选择信任自己apple id的证书
7,ipad上,打开altstore程序,右下角设置,选择加入beta计划,然后link上门注册的Patreon账号
8,altstore程序里,浏览全部程序就能看到有beta标的altstore,点击安装
9,在altstore beta里,browse的右上角有sources,添加utm的官方源,https://alt.getutm.app
10,在altstore beta里,browse里刷新一下就能看到utm了,点击安装即可

typecho文章自动发布到telegram频道

最近开设了willnet.net的电报频道,想发布新文章后自动发布到电报频道
搜索了一圈没有发现能实现typecho新文章推送到电报频道的插件,只有一个comments2telegram可以发送新评论到电报,wordpress倒是有个wptelegram可以实现,但是一直担心wordpress的安全问题没有切换到wp。

没有办法,自己组装个轮子吧。github搜了一圈发现一个telegram.sh,简单高效,我只要用shell拿到最新文章的url就可以组装起来了。

首先要新建一个bot
telegram搜索botfather
开始后输入bot的名字和用户名会得到类似11234567890:fdhslfdskagjdgjdka的token,记下来填入下面的脚本中

然后获取你的频道id
telegram搜索getidsbot
转发频道的一条消息到这个bot就会得到频道id,类似-1001234567890,记下来填入下面脚本中

最后添加bot到你的频道,设定个定时任务每分钟执行脚本,判断是不是最新文章,是的话推送到频道。

下面是代码,没有几行,就贴这吧
telegram.sh

#!/bin/bash

VERSION="0.2"
TOKEN=""
CHATS=()
DEBUG=false
DRY_RUN=false

IMAGE_FILE=""
DOCUMENT_FILE=""
PARSE_MODE=""
CODE_MODE=0
CRON_MODE=0
ACTION=""
DISABLE_WEB_PAGE_PREVIEW=false
DISABLE_NOTIFICATION=false

URL="https://api.telegram.org/bot"
FILE_URL="https://api.telegram.org/file/bot"
CURL_OPTIONS="-s"

TELEGRAM_TOKEN="你的bot token"
TELEGRAM_CHAT="你的频道id"

HAS_JQ=false
hash jq >/dev/null 2>&1 && HAS_JQ=true
      

function help {
    version
    echo "Usage: $0 [options] [message]"
    echo
    echo "OPTIONS are:"
    echo "    -t <TOKEN>       Telegram bot token to use. See ENVIRONMENT for more information."
    echo "    -c <CHAT_ID>     Chat to use as recipient. Can be given more than once. See ENVIRONMENT for more information."
    echo "    -f <FILE>        Sends file."
    echo "    -i <FILE>        Sends file as image. This will fail if the file isn't an actual image file."
    echo "    -M               Enables Markdown processing at telegram."
    echo "    -H               Enables HTML processing at telegram."
    echo "    -C               Sends text as monospace code. Useful when piping command outputs into this tool."
    echo "    -r               Like -C, but if the first line starts with '+ ', it is specially formatted."
    echo "    -l               Fetch known chat_ids."
    echo "    -R               Receive a file sent via telegram."
    echo "    -D               Sets disable_web_page_preview parameter to, well, disable the preview for links to webpages."
    echo "                     Can also be set in config as TELEGRAM_DISABLE_WEB_PAGE_PREVIEW=true (see ENVIRONMENT)"
    echo "                     This feature is only supported for text messages, not when sending files or images (-f, -i)."
    echo "    -N               Diables notifications on clients. Users will receive a notification with no sound."
    echo "                     Can also be set in config as TELEGRAM_DISABLE_NOTIFICATION=true (see ENVIRONMENT)"
    echo
    echo "DEBUGGING OPTIONS are:"
    echo "    -v               Display lots of more or less useful information."
    echo "    -j               Pretend you don't have JQ installed."
    echo "    -n               Dry-run - don't send messages, only print them on screen."
    echo
    echo "Message can be '-', in that case STDIN will be used."
    echo
    echo "ENVIRONMENT"
    echo "    TOKEN and CHAT_ID are required. You can set them in four different ways:"
    echo "      1) globally in /etc/telegram.sh.conf"
    echo "      2) user-local in ~/.telegram.sh"
    echo "      3) via environment variables TELEGRAM_TOKEN and TELEGRAM_CHAT"
    echo "      4) via options -t and -c"
    echo "    Later methods overwrite earlier settings, so you can easily override global settings."
    echo "    Please be aware that you shuld keep your telegram token secret!" 
    echo
    exit
}

function version {
    echo "telegram.sh version $VERSION"
    echo "by Fabian Schlenz"
}

function list_chats {
    log "$URL$TOKEN"
    response=`curl $CURL_OPTIONS $URL$TOKEN/getUpdates`
    log "$response"
    
    if [ "$HAS_JQ" = true ]; then
        echo "These are the available chats that I can find right now. The ID is the number at the front."
        echo "If there are no chats or the chat you are looking for isn't there, run this command again"
        echo "after sending a message to your bot via telegram."
        echo
        jq -r '.result | .[].message.chat | "\(.id|tostring) - \(.first_name) \(.last_name) (@\(.username))"' 2>/dev/null <<< "$response" || {
            echo "Could not parse reponse from Telegram."
            echo "Response was: $response"
            exit 1
        }
    else
        echo "You don't have jq installed. I'm afraid I can't parse the JSON from telegram without it."
        echo "So I'll have you do it. ;-)"
        echo
        echo "Please look for your chat_id in this output by yourself."
        echo 'Look for something like "chat":{"id":<CHAT_ID> and verify that first_name, last_name and'
        echo "username match your expected chat."
        echo
        echo "If there are no chats listed or the chat you are looking for isn't there, try again after"
        echo "sending a message to your bot via telegram."
        echo
        echo $response
    fi
}

function receive_file {
    if [ "$HAS_JQ" = false ]; then
        echo "You need to have jq installed in order to be able to download files."
        exit 1
    fi
    
    result=`curl $CURL_OPTIONS $URL$TOKEN/getUpdates?allowed_updates=message`
    log "$result"
    # {"ok":true,"result":[
    #   {
    #     "update_id":441727866,
    #     "message":{
    #       "message_id":8339,
    #       "from":{"id":15773,"is_bot":false,"first_name":"Fabian","last_name":"Schlenz","username":"fabianonline","language_code":"de"},
    #       "chat":{"id":15773,"first_name":"Fabian","last_name":"Schlenz","username":"fabianonline","type":"private"},
    #       "date":1526564127,
    #       "document":{"file_name":"desktop.ini","file_id":"BQAav-HkXugI","file_size":282}}}]}
    file_id=`jq -r '.result[-1].message.document.file_id' <<< "$result"`
    log "file_id: $file_id"
    if [ "$file_id" == "null" ]; then
        echo "Last message received apparently didn't contain a file. Aborting."
        exit 1
    fi
    file_name=`jq -r '.result[-1].message.document.file_name' <<< "$result"`
    log "file_name: $file_name"
    result=`curl $CURL_OPTIONS $URL$TOKEN/getFile?file_id=$file_id`
    log $result
    # {"ok":true,"result":{"file_id":"BQAav-HkXugI","file_size":282,"file_path":"documents/file_271.ini"}}
    path=`jq -r '.result.file_path' <<< "$result"`
    log "path: $path"
    if [ "$path" == "null" ]; then
        echo "Could not parse telegram's response to getFile. Aborting."
        exit 1
    fi
    file_name="`date +%s`_$file_name"
    log "file_name: $file_name"
    if [ -e "$file_name" ]; then
        echo "File $file_name already exists. This is unexpected, so I'm quitting now."
        exit 1
    fi
    curl $FILE_URL$TOKEN/$path --output "$file_name"
    echo "File downloaded as $file_name"
}

function log {
    [ "$DEBUG" = true ] && echo "DEBUG: $1"
}

function check_file {
    if [ ! -e "$1" ]; then
        echo "The file $1 does not exist."
        exit 1
    fi
    
    size=$(stat -c%s "$1")
    if (( size > 52428800 )); then
        echo "File $1 is breaking the file size limit imposed on Telegram bots (currently 50MB)."
        exit 1
    fi
}

function escapeMarkdown {
    res="${1//\*/∗}"
    res="${res//_/_}"
    res="${res//\`/‵}"
    #res="${res//\//∕}"
    echo "$res"
}

while getopts "t:c:i:f:MHCrhlvjnRDN" opt; do
    case $opt in
        t)
            TOKEN="$OPTARG"
            ;;
        c)
            CHATS+=("$OPTARG")
            ;;
        i)
            IMAGE_FILE="$OPTARG"
            ;;
        f)
            DOCUMENT_FILE="$OPTARG"
            ;;
        M)
            PARSE_MODE="Markdown"
            ;;
        H)
            PARSE_MODE="HTML"
            ;;
        C)
            PARSE_MODE="Markdown"
            CODE_MODE=1
            ;;
        r)
            PARSE_MODE="Markdown"
            CRON_MODE=1
            ;;
        l)
            ACTION="list_chats"
            ;;
        v)
            DEBUG=true
            ;;
        j)
            HAS_JQ=false
            ;;
        n)
            DRY_RUN=true
            ;;
        R)
            ACTION="receive_file"
            ;;
        D)
            DISABLE_WEB_PAGE_PREVIEW=true
            ;;
        N)
            DISABLE_NOTIFICATION=true
            ;;
        ?|h)
            help
            ;;
        :)
            echo "Option -$OPTARG needs an argument."
            exit 1
            ;;
        \?)
            echo "Invalid option -$OPTARG"
            exit 1
            ;;
    esac
done

if [ "$CRON_MODE" -eq 1 ] && [ "$CODE_MODE" -eq 1 ]; then
    echo "You can either use -C or -r, but not both."
    exit 1
fi

log "TOKEN is now $TOKEN"
log "CHATS is now ${CHATS[*]}"

[ -z "$TOKEN" ] && TOKEN=$TELEGRAM_TOKEN
[ ${#CHATS[@]} -eq 0 ] && CHATS=($TELEGRAM_CHAT)

log "TOKEN is now $TOKEN"
log "CHATS is now ${CHATS[*]}"

log "Importing config file(s)..."

[ -r /etc/telegram.sh.conf ] && source /etc/telegram.sh.conf
[ -r ~/.telegram.sh ] && source ~/.telegram.sh

[ -z "$TOKEN" ] && TOKEN=$TELEGRAM_TOKEN
[ ${#CHATS[@]} -eq 0 ] && CHATS=($TELEGRAM_CHAT)
[ -n "$TELEGRAM_DISABLE_WEB_PAGE_PREVIEW" ] && DISABLE_WEB_PAGE_PREVIEW="$TELEGRAM_DISABLE_WEB_PAGE_PREVIEW"
[ -n "$TELEGRAM_DISABLE_NOTIFICATION" ] && DISABLE_NOTIFICATION="$TELEGRAM_DISABLE_NOTIFICATION"

log "TOKEN is now $TOKEN"
log "CHATS is now ${CHATS[*]}"
log "DISABLE_WEB_PAGE_PREVIEW is now $DISABLE_WEB_PAGE_PREVIEW"
log "DISABLE_NOTIFICATION is now $DISABLE_NOTIFICATION"

if [ -z "$TOKEN" ]; then
    echo "No bot token was given."
    exit 1
fi

if [ ${#CHATS[@]} -eq 0 ] && [ -z "$ACTION" ]; then
    echo "No chat(s) given."
    exit 1
fi

if [ "$ACTION" = "list_chats" ]; then
    list_chats
    exit 0
fi

if [ "$ACTION" = "receive_file" ]; then
    receive_file
    exit 0
fi

shift $((OPTIND - 1))
TEXT="$1"
log "Text: $TEXT"

[ "$TEXT" = "-" ] && TEXT=$(</dev/stdin)

log "Text: $TEXT"

if [ $CODE_MODE -eq 1 ]; then
    TEXT='```'$'\n'$TEXT$'\n''```'
fi

if [ $CRON_MODE -eq 1 ]; then
    ONLY_COMMANDS=1
    while read line; do
        if [ "${line:0:2}" = "+ " ]; then
            ONLY_COMMANDS=0
        fi
    done <<< "$TEXT"
    
    if [ "$ONLY_COMMANDS" -eq 1 ]; then
        exit 0
    fi
    
    TEXT='```'$'\n'$TEXT$'\n''```'
    
    #FIRST_LINE=1
    #BLOCK_OPEN=0
    #NEW_TEXT=""
    #while read line; do
    #    if [ "${line:0:2}" = "+ " ]; then
    #        if [ "$BLOCK_OPEN" -eq 1 ]; then
    #            NEW_TEXT="$NEW_TEXT"'```'$'\n'
    #            BLOCK_OPEN=0
    #        fi
    #        NEW_TEXT="$NEW_TEXT*`escapeMarkdown "${line:2}"`*"$'\n'
    #    else
    #        if [ "$BLOCK_OPEN" -eq 0 ]; then
    #            NEW_TEXT="$NEW_TEXT"'```'$'\n'
    #            BLOCK_OPEN=1
    #        fi
    #        NEW_TEXT="$NEW_TEXT$line"$'\n'
    #    fi
    #done <<< "$TEXT"
    #[ "$BLOCK_OPEN" -eq 1 ] && NEW_TEXT="$NEW_TEXT"'```'
    #TEXT="$NEW_TEXT"
fi

log "Text: $TEXT"

if [ -z "$TEXT" ] && [ -z "$DOCUMENT_FILE" ] && [ -z "$IMAGE_FILE" ]; then
    echo "Neither text nor image or other file given."
    exit 1
fi

if [ -n "$DOCUMENT_FILE" ] && [ -n "$IMAGE_FILE" ]; then
    echo "You can't send a file AND an image at the same time."
    exit 1
fi

if [ -n "$DOCUMENT_FILE" ]; then
    check_file "$DOCUMENT_FILE"
    
    CURL_OPTIONS="$CURL_OPTIONS --form document=@$DOCUMENT_FILE"
    CURL_OPTIONS="$CURL_OPTIONS --form caption=<-"
    METHOD="sendDocument"
elif [ -n "$IMAGE_FILE" ]; then
    check_file "$IMAGE_FILE"
    CURL_OPTIONS="$CURL_OPTIONS --form photo=@$IMAGE_FILE"
    CURL_OPTIONS="$CURL_OPTIONS --form caption=<-"
    METHOD="sendPhoto"
else
    CURL_OPTIONS="$CURL_OPTIONS --form text=<-"
    [ -n "$PARSE_MODE" ] && CURL_OPTIONS="$CURL_OPTIONS --form-string parse_mode=$PARSE_MODE"
    [ "$DISABLE_WEB_PAGE_PREVIEW" = true ] && CURL_OPTIONS="$CURL_OPTIONS --form-string disable_web_page_preview=true"
    METHOD="sendMessage"
fi

[ "$DISABLE_NOTIFICATION" = true ] && CURL_OPTIONS="$CURL_OPTIONS --form-string disable_notification=true"

for id in "${CHATS[@]}"; do
    MY_CURL_OPTIONS="$CURL_OPTIONS --form-string chat_id=$id $URL$TOKEN/$METHOD"
    if [ "$DRY_RUN" = true ]; then
        echo "Executing: curl $MY_CURL_OPTIONS"
        echo "     Text: $TEXT"
        echo
        status=0
        response='{"ok": true}'
    else
        response=`curl $MY_CURL_OPTIONS <<< "$TEXT"`
        status=$?
    fi
    log "Response was: $response"
    if [ $status -ne 0 ]; then
        echo "curl reported an error. Exit code was: $status."
        echo "Response was: $response"
        echo "Quitting."
        exit $status
    fi

    if [ "$HAS_JQ" = true ]; then
        if [ "`jq -r '.ok' <<< "$response"`" != "true" ]; then
            echo "Telegram reported following error:"
            jq -r '"\(.error_code): \(.description)"' <<< "$response"
            echo "Quitting."
            exit 1
        fi
    else
        if [[ "$response" != '{"ok":true'* ]]; then
            echo "Telegram reported an error:"
            echo $response
            echo "Quitting."
            exit 1
        fi
    fi
done

newarticle_bot.sh

LAST_ARTICLE_URL=`cat /root/telegram_bot/lastarticleurl`
NEW_ARTICLE_URL=`curl -s https://你的域名 |grep "<li><h3><a" | awk -F "[\"\"]" '{print $2}'`
if [ "$LAST_ARTICLE_URL"x = "$NEW_ARTICLE_URL"x ];then
echo "no new article!"
else
echo $NEW_ARTICLE_URL |/root/telegram_bot/telegram.sh -
echo $NEW_ARTICLE_URL >/root/telegram_bot/lastarticleurl
fi

crontab -l

* * * * * /root/telegram_bot/newarticle_bot.sh



Routeros配置openvpn服务器和客户端

网上很多教程了,但是Routeros使用openvpn over shadowsocks避免使用机场安全隐患中需要一些内容,所以在此简单写一下。

Routeros配置openvpn服务器步骤如下
1,创建根证书
routeros网络配置地址(下同):/webfig/#System:Certificates.Certificates.new
以下信息根据自己更改,注意证书有效期
证书配置
保存后点击上图中sign按钮签发证书,成功后会如下图,证书前显示KAT
证书KAT标记
2,配置openvpn客户端地址池
/webfig/#IP:Pool.Pools.new
如图openvpn地址池
3,配置openvpn配置profile
/webfig/#PPP.Profiles.new
填写前三项,其他默认,注意本地地址和地址池尽量在同一网段,远程地址选择上面地址池名
Screen Shot 2020-04-06 at 01.46.04.png
4,配置openvpn服务器
/webfig/#PPP.OVPN_Server
配置以下红圈位置
openvpn服务器配置
5,配置openvpn账号
/webfig/#PPP.Secrets.new
openvpn账号
需要几个就添加几个

Routeros配置openvpn客户端
/webfig/#PPP.Interface.new.OVPN_Client
openvpn客户端

openvpn over shadowsocks避免机场安全隐患

2019年底开始翻墙变得越来越难,基础宽带是联通,穿墙线路一直使用的阿里云香港线路在晚上高峰期qos限速,延迟增加,本来200m带宽严重的时候甚至掉到各位数。

这时候不得不考虑一直续费而没用的备用机场了,由于没有出口的控制权,机场理论上来说可以看到所有非加密流量内容,个人对此比较在意,但是又觊觎机场的iplc线路(阿里云cen线路),不受墙影响,不受时间段qos,曾经询问过价格,被每月几万劝退了。

那么退而求其次,利用机场的shadowsocks作外层隧道,使用openvpn或其他tcp的vpn连接gcp的vps,保证本地到出口vps的链路是加密,就既能利用iplc线路又保证安全,唯一问题是tcp over tcp受网络波动影响很大,具体请谷歌之。

下面是使用环境,本地routeros v6,本地openwrt with ShadowSocksR Plus+ ,某阿里云中转机场,gcp香港routeros v6

1,在gcp香港routeros开启openvpn服务器,并打开gcp防火墙相关端口,参考Routeros配置openvpn服务器和客户端
2,配置好openwrt上机场信息,保证能通过机场翻墙
3,本地routeros配置gcp香港服务器ip路由通过openwrt,参考Routeros配置使用openwrt作旁路由
4,配置本地routeros的openvpn客户端,参考Routeros配置openvpn服务器和客户端
5,配置gcp香港routeros通过openvpn回程路由,如果本地routeros使用masquerade等nat时不需要配置,关于masquerade相关参考routeros的默认nat尽量不要用masquerade
6,配置本地routeros的ip分流,参考routeros配置vpn分流大陆ip