Move subscription checker ownership and implementation to SubCheck
This commit is contained in:
hellcat
parent
c06b292139
commit
4778ca17d1
@ -2,4 +2,4 @@ # fnb 项目维护入口
|
||||
|
||||
项目工作先读 GemDocs AI版项目总览、维护协议及 `/Users/hellcat/Documents/GemDocs/AI版/10-项目/fnb.md`。线上事务优先经白色机 SSH；网页与订阅 HTTP 访问须符合 GemDocs 的反代规则。
|
||||
|
||||
免费订阅调查及检测范围见 [CFN_SUBSCRIPTIONS.md](docs/maintenance/CFN_SUBSCRIPTIONS.md)。该记录只覆盖已注明的证据，不代表已部署检测器或完成客户端验收。代码、凭据、业务数据及并行修改按当前任务授权处理。
|
||||
订阅检测已独立为 SubCheck，入口见 [SubCheck](/Users/hellcat/Documents/GemCodes/subcheck/AGENTS.md)；cfn来源适配与调查见 [CFN.md](/Users/hellcat/Documents/GemCodes/subcheck/docs/CFN.md)。fnb不再保存检测实现。代码、凭据、业务数据及并行修改按当前任务授权处理。
|
||||
|
||||
@ -1,48 +0,0 @@
|
||||
# cfn 免费订阅发布与检测调查
|
||||
|
||||
调查日期：2026-10-18。范围：用户授权只读调查；经白色机跳转读取线上文件、配置，并以只读事务查询最近三篇已发布文章；未发起线上 HTTP 请求、节点连通测试、客户端核心加载、安装、定时任务或生产变更。
|
||||
|
||||
## 当前发布链路
|
||||
|
||||
- cfn：bgp-jp01，Nginx `/etc/nginx/conf.d/clashfreenode.com.conf` 的根目录为 `/www/wwwroot/clashfreenode-new.com/public`。另有 `/www/wwwroot/clashfreenode.com` 旧目录及旧 `public/sub`；不能将旧目录的文件认作当前发布结果。
|
||||
- 当前文章使用 `articles` / `categories` 数据；免费节点分类 `freenode`。最近已发布文章 slug 为 `fn20261018`、`fn20261017`、`fn20261016`，分别包含对应日期的三类订阅链接。
|
||||
- 订阅文件由 nodehub 线上版本 nodebus.net 托管：`/cfn/<每日变化目录>/<YYYYMMDD>-v2ray.txt`、`-singbox.txt`、`-clash.yaml`。线上磁盘 `/www/wwwroot/nodebus.net/public/cfn/`。不复制每日目录值；每天从已发布文章的最终链接发现，不能仅替换日期。
|
||||
- nodehub 的 `routes/web.php` 注册 `POST /api/freenode/file/put`，处理器 `app/Http/Controllers/Api/v1/FreenodeController.php::filePut` 接收 `site_code_short` 与各格式 `raw`，生成按日期变化的目录并写文件。所读处理器检查空内容和写入结果，未执行 YAML/JSON/节点字段/客户端核心校验。本次没有追查上游爬取转换实现，不能据此认定全链路均无校验。
|
||||
- cfn `GithubController::getFnToday` 是向配置的 master 站 `/api/v1/freenode/url_feed_today/cfn` 转发请求的代码入口，不是本地订阅文件读取。本次未调用；清单返回的实时性和内容尚未验收。
|
||||
|
||||
## 线上文件检查结果
|
||||
|
||||
以下是直接读取文件后的结构解析，不等同于 HTTP 下载或客户端导入验收。
|
||||
|
||||
| 日期 | Clash proxies | Sing-box outbounds（含3个控制项） | V2Ray非空链接行 |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| 2026-10-18 | 1875 | 1947 | 2021 |
|
||||
| 2026-10-17 | 1318 | 1263 | 1326 |
|
||||
| 2026-10-16 | 1292 | 1241 | 1308 |
|
||||
|
||||
- 九个文件的 YAML / JSON / 外层 Base64 基础解析成功；Sing-box 文件后缀是 `.txt`，实际内容是 JSON。格式应由声明和内容共同确定。
|
||||
- 今日 Clash：基础字段齐全、端口范围检查通过，未发现重复名称、空服务器及策略组内缺失引用；未执行核心兼容检查。
|
||||
- 今日 V2Ray：解码后第4、78个非空链接行在协议名前含 U+FEFF BOM；近两天也分别发现2行。这是实际编码异常；是否跳过节点或导致整体失败取决于客户端，未实测。不可把外层Base64解码成功视为订阅全部正确。
|
||||
- 今日 Sing-box：`outbounds[6]`（从0计数，type=vmess）包含空 `server` 和 `server_port: 0`，是需要拦截的节点字段异常；未运行核心，未确认客户端实际报错文案。
|
||||
- 不同格式节点数量不同；协议覆盖和转换筛选原因尚未追查，不直接把数量差认定为丢节点故障。
|
||||
- 线上 PATH 中未发现 mihomo / sing-box / xray；本次不安装。语法检查通过不能替代对应版本核心的配置校验。
|
||||
|
||||
## 检测目标与后续边界
|
||||
|
||||
用户要求的是爬取、加工后的最终订阅能否被客户端接受，不要求逐节点连通或测速。后续检测应区分：文章与订阅地址发现、最终文件下载、外层编码与结构、节点字段、客户端核心兼容。错误报告隐藏节点密码、UUID和令牌。每日执行时间、通知方式及可用 HTTP 访问路线留待后续；手动检测器部署状态见下节。
|
||||
|
||||
## 手动检测点部署与首轮验收（2026-10-18）
|
||||
|
||||
用户授权搭建并手动运行，明确暂不做定时和通知。已按Git提交/推送/拉取流程部署到白色机，检测代码版本 `dce13098426ac238dd8a774c81c228a3e9214657`；入口见 [检测器使用与回滚](../../tools/subscription-checker/README.md)。白色机路径和依赖关系由GemDocs的 `TOOL-CFN-CHECKER` 唯一登记，不修改现有Nginx、代理、订阅或源站数据。
|
||||
|
||||
本机与白色机9项针对性测试通过。固定核心 Mihomo v1.19.32、Sing-box v1.14.3，以及独立离线GeoIP数据库，下载SHA256均核对官方发布摘要。无常驻服务/代理、定时器、通知；源站SSH凭据仅随手动调用临时传入并清理，不长期保存。
|
||||
|
||||
最终手动报告：白色机 `/var/lib/cfn-subscription-checker/20261018-183812-043270/report.json`（北京时间18:38开始）；文章 `fn20261018`，日期检查通过。报告总状态 fail 是订阅内容异常，不是检测器部署失败。
|
||||
|
||||
| 格式 | 实测结论 | 范围 |
|
||||
| --- | --- | --- |
|
||||
| Clash | 1875个节点；静态检查通过，Mihomo完整配置检查通过；核心峰值RSS 76984 KiB（约75MiB） | 未测节点连通或GUI导入 |
|
||||
| Sing-box | 1944个代理出站，另3个控制项；outbounds[6]缺server、uuid且端口为0；1.14.3明确拒绝旧DNS servers格式 | 只代表固定版本兼容性；核心在首次错误停止，尚无后续错误全量清单；峰值RSS 64920 KiB（约63MiB） |
|
||||
| V2Ray | 2021个非空链接行；2处BOM、1处VMess载荷解析失败、84条非标准直接JSON VMess（需客户端导入器核验）、9处端口问题、1处URI authority问题 | 静态发现共97项（不是97个唯一坏节点）；未执行GUI导入；不能将84条变体直接断言为所有客户端不支持 |
|
||||
|
||||
首次运行因离线缺GeoIP导致Mihomo检查blocked_dependency，补齐独立校验数据后复跑通过；保留首轮报告以区分环境问题和订阅问题。核心在PrivateNetwork与资源限制内运行，未产生节点连接；上述RSS不包含主Python脚本，也不是整机总峰值。HTTP下载因未登记用途匹配反代明确未测；服务器文件检查不能替代用户实际下载链路验收。
|
||||
2
tools/subscription-checker/.gitignore
vendored
2
tools/subscription-checker/.gitignore
vendored
@ -1,2 +0,0 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
@ -1,19 +0,0 @@
|
||||
# cfn 手动订阅检测
|
||||
|
||||
范围：从 cfn 最新已发布文章发现三类最终链接，经白色机 SSH 读取 nodehub 对应文件，验证编码/结构/字段，再串行执行固定版本 Mihomo 与 Sing-box 的配置检查。V2Ray URI 列表仅做静态导入格式检查，不冒充客户端导入成功。不连接节点，不启用 TUN、代理服务、定时器或通知。
|
||||
|
||||
代码通过 fn_b Git 提交/推送，白色机稀疏检出本目录到 `/opt/cfn-subscription-checker`，只运行明确提交。`install_cores.py` 从官方发布下载 `cores.json` 固定版本并校验官方API公布的SHA256；独立目录 `/opt/cfn-checker-cores`。不覆盖系统核心。依赖系统 Python3/PyYAML、SSH、systemd-run、GNU time。
|
||||
|
||||
手动入口（本机）：`python3 tools/subscription-checker/run_from_mac.py`。脚本仅按需读取既有私有资源，凭据不进参数、Git或报告；通过SSH标准输入传给白色机，源站问密临时文件在退出时清理，SSH主机密钥使用本机已核实记录严格校验。未配置服务器长期认证，后续定时方案另行处理。
|
||||
|
||||
报告和当次样本：`/var/lib/cfn-subscription-checker/<北京时间戳>/report.json`，目录700、文件600。样本包含公开节点凭据，仅限root私有检查，不提交Git。错误仅记录字段、序号及脱敏核心日志；报告记录源站文件读取状态，HTTP下载因反代入口缺失明确未测。文章不是当天时标记stale，发现缺失/多义链接会失败，不猜路径、不回退旧文件。
|
||||
|
||||
核心检查使用独立临时systemd服务：PrivateNetwork、NoNewPrivileges、ProtectSystem=strict，只允许写当次核心工作目录；MemoryMax=256M、MemorySwapMax=0、CPUQuota=50%、RuntimeMaxSec=45；校验完退出，不开放监听。若配置需远端规则/Geo数据而离线缺失，报告blocked_dependency，不伪报配置通过。256M为核心进程的限制，主Python解析进程另占内存。
|
||||
|
||||
返回值：0=服务器文件结构及可执行核心检查通过（仍未HTTP/GUI验收），1=发现异常、检查未完成或获取失败。检查失败是检测结果，不等于部署失败。
|
||||
|
||||
回滚：本工具无常驻服务，停止当前手动命令即可；确认无人运行后移除独立代码/核心目录，报告按需保留或另行清理。不改源站业务文件、白色机Nginx、路由或现有容器。依赖包未改时不涉及系统回滚。
|
||||
|
||||
测试：`python3 -m unittest discover -s tools/subscription-checker -v`。
|
||||
|
||||
已验收：2026-10-18，9项测试通过；今日实际文件手动运行完成，Clash核心通过，Sing-box旧DNS配置被1.14.3拒绝，V2Ray静态检测发现异常；细节见 [调查与验收记录](../../docs/maintenance/CFN_SUBSCRIPTIONS.md)。GeoIP数据URL为官方滚动发布，摘要固定；后续上游替换文件时安装器会安全失败，需要显式更新摘要，不能自动忽略校验。
|
||||
@ -1,183 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Manual cfn subscription validator. No timers, notifications, or proxy traffic."""
|
||||
import argparse, base64, collections, datetime, html, json, os, re, shlex, shutil, subprocess, sys, tempfile
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit, unquote
|
||||
import yaml
|
||||
|
||||
HOSTS = {'article': '45.153.247.44', 'files': '45.142.166.119'}
|
||||
FORMATS = {'clash': '.yaml', 'singbox': '.txt', 'v2ray': '.txt'}
|
||||
MAX_BYTES = 20 * 1024 * 1024
|
||||
|
||||
def command(args, **kw):
|
||||
return subprocess.run(args, capture_output=True, timeout=kw.pop('timeout', 45), **kw)
|
||||
|
||||
def ssh(host, code, env):
|
||||
r = command(['ssh', '-o', 'StrictHostKeyChecking=yes', '-o', 'UserKnownHostsFile='+env['CFN_KNOWN_HOSTS'], '-o', 'ConnectTimeout=12', '-o', 'NumberOfPasswordPrompts=1', 'root@'+host, 'python3 -c '+shlex.quote(code)], env=env, timeout=60)
|
||||
if r.returncode:
|
||||
raise RuntimeError('source_ssh_failed:'+host+':exit_'+str(r.returncode))
|
||||
if len(r.stdout) > MAX_BYTES: raise RuntimeError('source_response_too_large')
|
||||
return r.stdout
|
||||
|
||||
def collect(root, secrets):
|
||||
root.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryDirectory(prefix='auth-', dir=root) as auth:
|
||||
a = Path(auth); (a/'credentials.json').write_text(json.dumps(secrets)); (a/'known_hosts').write_text(secrets['known_hosts'])
|
||||
ask = a/'askpass.py'
|
||||
ask.write_text('#!/usr/bin/python3\nimport os,sys,json\np=" ".join(sys.argv[1:])\nd=json.load(open(os.environ["CFN_AUTH_FILE"]))\nh=next((h for h in d["passwords"] if h in p),None)\nif not h:sys.exit(1)\nprint(d["passwords"][h])\n')
|
||||
ask.chmod(0o700)
|
||||
for f in [a/'credentials.json', a/'known_hosts']: f.chmod(0o600)
|
||||
env = dict(os.environ, SSH_ASKPASS=str(ask), SSH_ASKPASS_REQUIRE='force', DISPLAY='1', CFN_AUTH_FILE=str(a/'credentials.json'), CFN_KNOWN_HOSTS=str(a/'known_hosts'))
|
||||
php = r'''chdir('/www/wwwroot/clashfreenode-new.com');require 'vendor/autoload.php';Dotenv\Dotenv::createImmutable(getcwd())->load();try{$p=new PDO('mysql:host='.$_ENV['DB_HOST'].';port='.($_ENV['DB_PORT']??3306).';dbname='.$_ENV['DB_DATABASE'].';charset=utf8mb4',$_ENV['DB_USERNAME'],$_ENV['DB_PASSWORD'],[PDO::ATTR_ERRMODE=>PDO::ERRMODE_EXCEPTION]);$p->exec('SET TRANSACTION READ ONLY');$p->beginTransaction();$r=$p->query("SELECT a.id,a.sSlug,a.sContent,a.updated_at FROM articles a JOIN categories c ON c.id=a.iCategoryId WHERE c.sSlug='freenode' AND a.iStatus=1 ORDER BY a.id DESC LIMIT 1")->fetch(PDO::FETCH_ASSOC);echo json_encode($r);$p->rollBack();}catch(Throwable $e){echo json_encode(['error'=>'article_query_failed']);exit(1);}'''
|
||||
code = 'import subprocess,sys\nr=subprocess.run(["php","-r",'+repr(php)+'],capture_output=True)\nsys.stdout.buffer.write(r.stdout)\nsys.exit(r.returncode)'
|
||||
row = json.loads(ssh(HOSTS['article'], code, env)); content = row.pop('sContent')
|
||||
links = list(dict.fromkeys(html.unescape(x) for x in re.findall(r'''(?:href|data-clipboard-text)=["']([^"']+)["']''', content)))
|
||||
found = {}
|
||||
for link in links:
|
||||
u=urlsplit(link)
|
||||
if u.scheme != 'https' or u.netloc != 'nodebus.net' or u.query or u.fragment: continue
|
||||
m=re.fullmatch(r'/cfn/[a-f0-9]{16}/(\d{8})-(v2ray|singbox|clash)(\.txt|\.yaml)',u.path)
|
||||
if not m or m[3] != FORMATS[m[2]]: continue
|
||||
if m[2] in found and found[m[2]]['path'] != u.path: raise RuntimeError('ambiguous_subscription_links')
|
||||
found[m[2]]={'path':u.path,'date':m[1]}
|
||||
if set(found) != set(FORMATS): raise RuntimeError('missing_or_unrecognized_subscription_links')
|
||||
article_date = re.fullmatch(r'fn(\d{8})', row['sSlug'])
|
||||
if not article_date or any(v['date'] != article_date[1] for v in found.values()): raise RuntimeError('article_file_date_mismatch')
|
||||
for kind,item in found.items():
|
||||
code='from pathlib import Path\nimport sys\np=Path('+repr('/www/wwwroot/nodebus.net/public'+item['path'])+')\nr=p.resolve()\nassert r.is_relative_to(Path("/www/wwwroot/nodebus.net/public/cfn"))\nassert 0<p.stat().st_size<='+str(MAX_BYTES)+'\nsys.stdout.buffer.write(p.read_bytes())'
|
||||
data=ssh(HOSTS['files'], code, env); f=root/(kind+FORMATS[kind]);f.write_bytes(data);f.chmod(0o600)
|
||||
manifest={'article_id':row['id'],'article_slug':row['sSlug'],'article_updated_at_raw':row['updated_at'],'date':article_date[1], 'acquisition':'ssh-server-file', 'http_download':'not_tested_missing_registered_proxy', 'formats':list(found)}
|
||||
(root/'manifest.json').write_text(json.dumps(manifest,ensure_ascii=False,indent=2));return manifest
|
||||
|
||||
def b64(s):
|
||||
s=''.join(s.split());return base64.b64decode(s+'='*((-len(s))%4),validate=True)
|
||||
|
||||
def validate(kind, data):
|
||||
issues=[]; result={'format':kind,'bytes':len(data),'issues':issues}; private=[]
|
||||
def issue(code, **where): issues.append(dict(code=code,**where))
|
||||
def remember(node):
|
||||
for k in ['name','tag','server','password','uuid','id','username','sni','servername']:
|
||||
v=node.get(k)
|
||||
if isinstance(v,str) and v:private.append(v)
|
||||
try:
|
||||
s=data.decode('utf-8')
|
||||
if not s.strip(): raise ValueError('empty')
|
||||
if s.lstrip().lower().startswith(('<!doctype html','<html')): raise ValueError('html')
|
||||
if kind=='clash':
|
||||
d=yaml.safe_load(s)
|
||||
if not isinstance(d,dict) or not isinstance(d.get('proxies'),list):raise ValueError('shape')
|
||||
nodes=d['proxies']; result['nodes']=len(nodes);result['types']=dict(collections.Counter(str(n.get('type')) for n in nodes if isinstance(n,dict)))
|
||||
names=[]
|
||||
for i,n in enumerate(nodes):
|
||||
if not isinstance(n,dict):issue('node_not_object',index=i);continue
|
||||
remember(n);names.append(n.get('name'))
|
||||
for k in ['name','type','server']:
|
||||
if not isinstance(n.get(k),str) or not n[k].strip():issue('missing_or_empty_field',index=i,field=k)
|
||||
check_port(n.get('port'),issue,i,'port')
|
||||
required={'vmess':['uuid'],'vless':['uuid'],'trojan':['password'],'ss':['cipher','password'],'ssr':['cipher','password','protocol','obfs']}.get(n.get('type'),[])
|
||||
for k in required:
|
||||
if not n.get(k):issue('missing_or_empty_field',index=i,field=k)
|
||||
dup=len(names)-len(set(str(x) for x in names))
|
||||
if dup:issue('duplicate_node_names',count=dup)
|
||||
groups=d.get('proxy-groups',[]);targets={str(n) for n in names}|{str(g.get('name')) for g in groups if isinstance(g,dict)}|{'DIRECT','REJECT','REJECT-DROP','PASS','COMPATIBLE','GLOBAL'}
|
||||
for i,g in enumerate(groups):
|
||||
for j,n in enumerate(g.get('proxies',[])):
|
||||
if str(n) not in targets:issue('missing_group_reference',group_index=i,reference_index=j)
|
||||
elif kind=='singbox':
|
||||
d=json.loads(s)
|
||||
if not isinstance(d,dict) or not isinstance(d.get('outbounds'),list):raise ValueError('shape')
|
||||
nodes=d['outbounds'];result['outbounds']=len(nodes);result['nodes']=sum(n.get('type') not in ['selector','urltest','direct','block','dns'] for n in nodes if isinstance(n,dict));result['types']=dict(collections.Counter(str(n.get('type')) for n in nodes if isinstance(n,dict)))
|
||||
tags=[]
|
||||
for i,n in enumerate(nodes):
|
||||
if not isinstance(n,dict):issue('node_not_object',index=i);continue
|
||||
remember(n);tags.append(n.get('tag'))
|
||||
if n.get('type') in ['selector','urltest','direct','block','dns']:continue
|
||||
if not isinstance(n.get('server'),str) or not n['server'].strip():issue('missing_or_empty_field',index=i,field='server')
|
||||
check_port(n.get('server_port'),issue,i,'server_port')
|
||||
for k in {'vmess':['uuid'],'vless':['uuid'],'trojan':['password'],'shadowsocks':['method','password']}.get(n.get('type'),[]):
|
||||
if not n.get(k):issue('missing_or_empty_field',index=i,field=k)
|
||||
if len(tags)!=len(set(str(x) for x in tags)):issue('duplicate_outbound_tags')
|
||||
else:
|
||||
s=b64(s).decode('utf-8') if '://' not in s else s
|
||||
lines=[x.strip() for x in s.splitlines() if x.strip()];result['nodes']=len(lines);schemes=collections.Counter()
|
||||
for i,line in enumerate(lines,1):
|
||||
if '\ufeff' in line:issue('unexpected_BOM',line=i)
|
||||
line=line.lstrip('\ufeff');scheme,sep,payload=line.partition('://');schemes[scheme]+=1
|
||||
if not sep or scheme not in ['vmess','vless','ss','ssr','trojan','hysteria','hysteria2','hy2','tuic','anytls','socks','socks5','http','https']:issue('unsupported_or_missing_scheme',line=i);continue
|
||||
if not payload:issue('empty_payload',line=i);continue
|
||||
if scheme=='vmess':
|
||||
if unquote(payload).lstrip().startswith('{'):
|
||||
issue('nonstandard_vmess_json_payload_requires_client_check',line=i)
|
||||
continue
|
||||
payload=payload.split('#',1)[0].split('?',1)[0]
|
||||
try:
|
||||
n=json.loads(b64(payload).decode());remember(n)
|
||||
if not all(str(n.get(k,'')).strip() for k in ['add','id']):issue('vmess_required_field',line=i)
|
||||
check_port(n.get('port'),issue,i,'port',location='line')
|
||||
except Exception:issue('vmess_payload_not_supported_or_invalid',line=i)
|
||||
elif scheme not in ['ss','ssr']:
|
||||
try:
|
||||
u=urlsplit(line)
|
||||
if not u.hostname:issue('missing_uri_host',line=i)
|
||||
check_port(u.port,issue,i,'port',location='line')
|
||||
except Exception:issue('invalid_uri_authority',line=i)
|
||||
result['schemes']=dict(schemes);result['core_check']={'status':'not_applicable','reason':'URI list requires client importer; no whole-file core check claimed'}
|
||||
if not result.get('nodes'):issue('empty_node_list')
|
||||
result['parse']='pass'
|
||||
except Exception as e:
|
||||
result['parse']='fail';issue('parse_error',error_type=type(e).__name__)
|
||||
if hasattr(e,'lineno'):issues[-1]['line']=e.lineno
|
||||
if hasattr(e,'problem_mark') and e.problem_mark:issues[-1]['line']=e.problem_mark.line+1
|
||||
result['structure_status']='fail' if issues else 'pass';result['issue_count']=len(issues);return result,private
|
||||
|
||||
def check_port(v, issue, index, field, location='index'):
|
||||
try:ok=not isinstance(v,bool) and str(v).isdigit() and 1<=int(v)<=65535
|
||||
except Exception:ok=False
|
||||
if not ok:issue('invalid_port',**{location:index,'field':field})
|
||||
|
||||
def core_check(kind, path, root, binaries, private):
|
||||
core=binaries/('mihomo' if kind=='clash' else 'sing-box')
|
||||
if not core.exists():return {'status':'not_tested','reason':'core_missing'}
|
||||
home=root/('core-'+kind);home.mkdir(mode=0o700,exist_ok=True)
|
||||
if kind=='clash':
|
||||
for name in ['Country.mmdb','geoip.metadb']:
|
||||
if (binaries/name).exists():shutil.copyfile(binaries/name,home/name)
|
||||
args=[str(core),'-t','-d',str(home),'-f',str(path)] if kind=='clash' else [str(core),'check','-D',str(home),'-c',str(path)]
|
||||
# A private network blocks rule/provider downloads and all node traffic. No proxy is started.
|
||||
wrapper=['systemd-run','--quiet','--wait','--pipe','--collect','--service-type=exec','-p','MemoryMax=256M','-p','MemorySwapMax=0','-p','CPUQuota=50%','-p','RuntimeMaxSec=45','-p','PrivateNetwork=yes','-p','NoNewPrivileges=yes','-p','ProtectSystem=strict','-p','ReadWritePaths='+str(home),'-p','TasksMax=64','/usr/bin/time','-f','CHECKER_MAX_RSS_KB=%M']
|
||||
try:
|
||||
r=command(wrapper+args,timeout=60);text=(r.stdout+r.stderr).decode(errors='replace')
|
||||
mem=re.search(r'CHECKER_MAX_RSS_KB=(\d+)',text)
|
||||
for v in sorted(set(private),key=len,reverse=True):
|
||||
if len(v)>2:text=text.replace(v,'[redacted]')
|
||||
text=re.sub(r'(?:https?|ss|vmess|vless|trojan)://[^\s"\']+','[redacted-url]',text)
|
||||
text=re.sub(r'\x1b\[[0-9;]*m','',text)
|
||||
status='pass' if r.returncode==0 else 'fail'
|
||||
if r.returncode and any(x in text.lower() for x in ['network is unreachable','network is down','no such host','download','geosite.dat','geoip.dat','country.mmdb']):status='blocked_dependency'
|
||||
return {'status':status,'exit_code':r.returncode,'peak_rss_kb':int(mem[1]) if mem else None,'message':text[-3500:]}
|
||||
except subprocess.TimeoutExpired:return {'status':'timeout'}
|
||||
|
||||
def main():
|
||||
os.umask(0o077)
|
||||
a=argparse.ArgumentParser();a.add_argument('--state-dir',default='/var/lib/cfn-subscription-checker');a.add_argument('--binaries',default='/opt/cfn-checker-cores');a.add_argument('--snapshot-dir');a.add_argument('--credentials-stdin',action='store_true');args=a.parse_args()
|
||||
now=datetime.datetime.now(datetime.timezone(datetime.timedelta(hours=8)));root=Path(args.state_dir)/now.strftime('%Y%m%d-%H%M%S-%f');root.mkdir(mode=0o700,parents=True)
|
||||
report={'checked_at':now.isoformat(),'http_download':'not_tested_missing_registered_proxy','node_connectivity':'not_tested','results':[]}
|
||||
try:
|
||||
if args.snapshot_dir:
|
||||
source=Path(args.snapshot_dir);manifest=json.loads((source/'manifest.json').read_text())
|
||||
for k,suffix in FORMATS.items():shutil.copyfile(source/(k+suffix),root/(k+suffix))
|
||||
elif args.credentials_stdin:manifest=collect(root,json.load(sys.stdin))
|
||||
else:raise RuntimeError('credentials_stdin_or_snapshot_dir_required')
|
||||
report['source']=manifest;report['freshness']='pass' if manifest['date']==now.strftime('%Y%m%d') else 'stale'
|
||||
for kind,suffix in FORMATS.items():
|
||||
f=root/(kind+suffix);f.chmod(0o600);result,private=validate(kind,f.read_bytes())
|
||||
if kind!='v2ray':result['core_check']=core_check(kind,f,root,Path(args.binaries),private)
|
||||
report['results'].append(result)
|
||||
failed=report['freshness']!='pass' or any(r['structure_status']=='fail' or r['core_check']['status'] in ['fail','timeout'] for r in report['results'])
|
||||
incomplete=any(r['core_check']['status'] in ['blocked_dependency','not_tested'] for r in report['results'])
|
||||
report['status']='fail' if failed else 'incomplete' if incomplete else 'pass_server_files_only'
|
||||
except Exception as e:
|
||||
report['status']='acquisition_failed';report['error_type']=type(e).__name__;report['error']=str(e) if isinstance(e,RuntimeError) else 'see source or format stage'
|
||||
output=root/'report.json';output.write_text(json.dumps(report,ensure_ascii=False,indent=2));print(output);print(json.dumps(report,ensure_ascii=False,indent=2));return 0 if report['status']=='pass_server_files_only' else 1
|
||||
|
||||
if __name__=='__main__':sys.exit(main())
|
||||
@ -1,22 +0,0 @@
|
||||
{
|
||||
"mihomo": {
|
||||
"version": "v1.19.32",
|
||||
"url": "https://github.com/MetaCubeX/mihomo/releases/download/v1.19.32/mihomo-linux-amd64-compatible-v1.19.32.gz",
|
||||
"sha256": "ba3ce607747a07f948fc35780e108a4a7c7f552a38b9bd4d115f313ebcb89c20"
|
||||
},
|
||||
"sing-box": {
|
||||
"version": "v1.14.3",
|
||||
"url": "https://github.com/SagerNet/sing-box/releases/download/v1.14.3/sing-box-1.14.3-linux-amd64.tar.gz",
|
||||
"sha256": "e2bdf179c15a3652955dc44867e33ca0965c9c9b91b34267dcc5a5d639a5feee"
|
||||
},
|
||||
"Country.mmdb": {
|
||||
"version": "snapshot-20261018",
|
||||
"url": "https://github.com/MetaCubeX/meta-rules-dat/releases/download/latest/country.mmdb",
|
||||
"sha256": "64809d264be87ecfb7ac3ad19b7678893b6275038a7d4bf9eec72464a6c4f9fe"
|
||||
},
|
||||
"geoip.metadb": {
|
||||
"version": "snapshot-20261018",
|
||||
"url": "https://github.com/MetaCubeX/meta-rules-dat/releases/download/latest/geoip.metadb",
|
||||
"sha256": "ec6701e428b7efe610fe40b187a69fb03324892a56792b033adc080d2b6c273e"
|
||||
}
|
||||
}
|
||||
@ -1,20 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
import gzip,hashlib,json,os,tarfile,tempfile,urllib.request,io
|
||||
from pathlib import Path
|
||||
os.umask(0o077)
|
||||
root=Path('/opt/cfn-checker-cores');root.mkdir(exist_ok=True,mode=0o700)
|
||||
manifest=json.loads(Path(__file__).with_name('cores.json').read_text())
|
||||
for name,info in manifest.items():
|
||||
req=urllib.request.Request(info['url'],headers={'User-Agent':'cfn-subscription-checker'})
|
||||
with urllib.request.urlopen(req,timeout=90) as r: data=r.read(100*1024*1024)
|
||||
if hashlib.sha256(data).hexdigest()!=info['sha256']:raise RuntimeError('checksum mismatch: '+name)
|
||||
if name in ['Country.mmdb','geoip.metadb']:binary=data
|
||||
elif name=='mihomo':binary=gzip.decompress(data)
|
||||
else:
|
||||
with tarfile.open(fileobj=io.BytesIO(data),mode='r:gz') as t:
|
||||
members=[m for m in t.getmembers() if m.isfile() and Path(m.name).name=='sing-box']
|
||||
if len(members)!=1:raise RuntimeError('unexpected archive')
|
||||
binary=t.extractfile(members[0]).read()
|
||||
f=root/(name+'.new');f.write_bytes(binary);f.chmod(0o700);f.replace(root/name)
|
||||
print(name,info['version'],'SHA256 verified')
|
||||
(root/'versions.json').write_text(json.dumps(manifest,indent=2))
|
||||
@ -1,19 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Manual launcher: secret values go only over SSH stdin, never argv/Git."""
|
||||
import json,os,subprocess,sys,tempfile
|
||||
from pathlib import Path
|
||||
HOSTS={'104.168.30.40':'VPS-001','45.153.247.44':'VPS-003','45.142.166.119':'VPS-005'}
|
||||
base=Path('/Users/hellcat/Documents/GemDocs/AI版/30-私有资源/凭据')
|
||||
passwords={host:(base/(name+'.md')).read_text().strip().split('：',1)[1].strip() for host,name in HOSTS.items()}
|
||||
known=[]
|
||||
for host in HOSTS:
|
||||
r=subprocess.run(['ssh-keygen','-F',host],capture_output=True,text=True,check=True)
|
||||
known.extend(x for x in r.stdout.splitlines() if not x.startswith('#'))
|
||||
with tempfile.TemporaryDirectory(prefix='cfn-manual-') as tmp:
|
||||
os.chmod(tmp,0o700);p=Path(tmp)/'white-password';p.write_text(passwords['104.168.30.40']);p.chmod(0o600)
|
||||
a=Path(tmp)/'askpass.py';a.write_text('#!/usr/local/bin/python3\nimport os\nfrom pathlib import Path\nprint(Path(os.environ["CFN_WHITE_PASSWORD"]).read_text())\n');a.chmod(0o700)
|
||||
env=dict(os.environ,SSH_ASKPASS=str(a),SSH_ASKPASS_REQUIRE='force',DISPLAY='1',CFN_WHITE_PASSWORD=str(p))
|
||||
payload={'passwords':{h:v for h,v in passwords.items() if h!='104.168.30.40'},'known_hosts':'\n'.join(known)+'\n'}
|
||||
cmd='python3 /opt/cfn-subscription-checker/tools/subscription-checker/check.py --credentials-stdin'
|
||||
r=subprocess.run(['ssh','-o','StrictHostKeyChecking=yes','-o','ConnectTimeout=15','root@104.168.30.40',cmd],input=json.dumps(payload).encode(),env=env)
|
||||
sys.exit(r.returncode)
|
||||
@ -1,26 +0,0 @@
|
||||
import base64,json,unittest
|
||||
from check import validate
|
||||
class ValidationTests(unittest.TestCase):
|
||||
def codes(self,kind,s):return {x['code'] for x in validate(kind,s.encode())[0]['issues']}
|
||||
def test_bom_inside_decoded_subscription(self):
|
||||
s=base64.b64encode('\ufefftrojan://secret@example.com:443#name'.encode()).decode()
|
||||
self.assertIn('unexpected_BOM',self.codes('v2ray',s))
|
||||
def test_vmess_fragment_is_not_part_of_base64(self):
|
||||
s='vmess://'+base64.b64encode(json.dumps({'add':'example.com','port':443,'id':'test-id'}).encode()).decode()+'#node'
|
||||
self.assertEqual(self.codes('v2ray',s),set())
|
||||
def test_plain_json_vmess_is_reported_as_client_compatibility(self):
|
||||
s='vmess://'+json.dumps({'add':'example.com','port':443,'id':'test-id'})
|
||||
self.assertEqual(self.codes('v2ray',s),{'nonstandard_vmess_json_payload_requires_client_check'})
|
||||
def test_empty_server_and_port_zero_in_valid_json(self):
|
||||
codes=self.codes('singbox',json.dumps({'outbounds':[{'type':'vmess','tag':'bad','server':'','server_port':0,'uuid':'test'}]}))
|
||||
self.assertEqual(codes,{'missing_or_empty_field','invalid_port'})
|
||||
def test_html_instead_of_subscription(self):self.assertIn('parse_error',self.codes('clash','<!doctype html><html>error</html>'))
|
||||
def test_missing_group_target(self):
|
||||
self.assertIn('missing_group_reference',self.codes('clash','proxies: [{name: ok, type: http, server: example.com, port: 80}]\nproxy-groups: [{name: choose, type: select, proxies: [missing]}]'))
|
||||
def test_empty_list_is_not_success(self):self.assertIn('empty_node_list',self.codes('clash','proxies: []'))
|
||||
def test_malformed_base64(self):self.assertIn('parse_error',self.codes('v2ray','@@@bad@@@'))
|
||||
def test_reports_do_not_contain_secrets(self):
|
||||
r,_=validate('singbox',json.dumps({'outbounds':[{'type':'trojan','tag':'private-name','server':'secret.host','server_port':0,'password':'test-private-password'}]}).encode())
|
||||
text=json.dumps(r)
|
||||
for secret in ['test-private-password','secret.host','private-name']:self.assertNotIn(secret,text)
|
||||
if __name__=='__main__':unittest.main()
|
||||
Loading…
Reference in New Issue
Block a user