do
This commit is contained in:
355
README.md
355
README.md
@@ -1,39 +1,48 @@
|
||||
# ProxyRotator
|
||||
|
||||
代理池管理系统 - 支持代理导入、测试、轮询分发和结果上报。
|
||||
代理池轮换管理系统,支持代理导入、健康检测、智能轮换和 Telegram Bot 管理。
|
||||
|
||||
## 功能特性
|
||||
|
||||
- **代理导入** - 支持文本和文件批量导入,自动解析多种代理格式
|
||||
- **健康检测** - 并发测试代理可用性,记录延迟和成功率
|
||||
- **智能轮换** - 基于评分的代理选择,支持租约机制防止并发冲突
|
||||
- **站点隔离** - 同一站点不会分配到相同代理,避免被封禁
|
||||
- **Telegram Bot** - 通过 Telegram 管理代理池
|
||||
- **REST API** - 完整的 HTTP API 接口
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1. 安装依赖
|
||||
### 环境要求
|
||||
|
||||
- Go 1.22+
|
||||
- PostgreSQL 14+
|
||||
|
||||
### 安装
|
||||
|
||||
```bash
|
||||
go mod tidy
|
||||
```
|
||||
# 克隆仓库
|
||||
git clone <repo-url>
|
||||
cd proxyrotator
|
||||
|
||||
### 2. 初始化数据库
|
||||
# 安装依赖
|
||||
make deps
|
||||
|
||||
```bash
|
||||
# 创建 PostgreSQL 数据库
|
||||
createdb proxyrotator
|
||||
|
||||
# 执行迁移脚本
|
||||
export DATABASE_URL="postgres://user:pass@localhost:5432/proxyrotator?sslmode=disable"
|
||||
# 初始化数据库
|
||||
export DATABASE_URL="postgres://postgres:password@localhost:5432/proxyrotator"
|
||||
make migrate
|
||||
|
||||
# 构建
|
||||
make build
|
||||
```
|
||||
|
||||
### 3. 启动服务
|
||||
### 配置
|
||||
|
||||
复制 `envexmaple` 为 `.env` 并修改配置:
|
||||
|
||||
```bash
|
||||
make run
|
||||
```
|
||||
|
||||
## 配置
|
||||
|
||||
通过环境变量配置,或创建 `.env` 文件:
|
||||
|
||||
```env
|
||||
DATABASE_URL=postgres://user:pass@localhost:5432/proxyrotator?sslmode=disable
|
||||
LISTEN_ADDR=:8080
|
||||
DATABASE_URL=postgres://postgres:password@localhost:5432/proxyrotator
|
||||
LISTEN_ADDR=0.0.0.0:9987
|
||||
API_KEY=your-secret-key
|
||||
RETURN_SECRET=true
|
||||
MAX_CONCURRENCY=200
|
||||
@@ -41,261 +50,125 @@ MAX_TEST_LIMIT=2000
|
||||
LEASE_TTL=60s
|
||||
```
|
||||
|
||||
| 变量 | 说明 | 默认值 |
|
||||
|------|------|--------|
|
||||
| `DATABASE_URL` | PostgreSQL 连接串 | - |
|
||||
| `LISTEN_ADDR` | 监听地址 | `:8080` |
|
||||
| `API_KEY` | API 密钥(空则不鉴权) | - |
|
||||
| `RETURN_SECRET` | 是否返回代理密码 | `true` |
|
||||
| `MAX_CONCURRENCY` | 测试最大并发数 | `200` |
|
||||
| `MAX_TEST_LIMIT` | 单次测试上限 | `2000` |
|
||||
| `LEASE_TTL` | 租约有效期 | `60s` |
|
||||
| 配置项 | 说明 | 默认值 |
|
||||
|--------|------|--------|
|
||||
| DATABASE_URL | PostgreSQL 连接字符串 | - |
|
||||
| LISTEN_ADDR | 监听地址 | 0.0.0.0:9987 |
|
||||
| API_KEY | API 认证密钥 | - |
|
||||
| RETURN_SECRET | 返回代理密码 | true |
|
||||
| MAX_CONCURRENCY | 测试最大并发数 | 200 |
|
||||
| MAX_TEST_LIMIT | 单次测试最大数量 | 2000 |
|
||||
| LEASE_TTL | 代理租约有效期 | 60s |
|
||||
|
||||
---
|
||||
### 运行
|
||||
|
||||
## 代理格式
|
||||
```bash
|
||||
# 直接运行
|
||||
make run
|
||||
|
||||
### 支持的格式
|
||||
# 或使用编译后的二进制
|
||||
./bin/proxyrotator
|
||||
|
||||
| 格式 | 示例 |
|
||||
|------|------|
|
||||
| `host:port` | `192.168.1.1:8080` |
|
||||
| `user:pass@host:port` | `admin:123456@192.168.1.1:8080` |
|
||||
| `http://host:port` | `http://192.168.1.1:8080` |
|
||||
| `http://user:pass@host:port` | `http://admin:123456@192.168.1.1:8080` |
|
||||
| `https://host:port` | `https://192.168.1.1:443` |
|
||||
| `socks5://host:port` | `socks5://192.168.1.1:1080` |
|
||||
| `socks5://user:pass@host:port` | `socks5://admin:123456@192.168.1.1:1080` |
|
||||
|
||||
### TXT 文件格式
|
||||
|
||||
每行一个代理,支持 `#` 注释:
|
||||
|
||||
```txt
|
||||
# HTTP 代理
|
||||
http://user:pass@1.2.3.4:8080
|
||||
http://5.6.7.8:3128
|
||||
|
||||
# SOCKS5 代理
|
||||
socks5://admin:pwd@10.0.0.1:1080
|
||||
socks5://10.0.0.2:1080
|
||||
|
||||
# 简写格式(自动推断协议)
|
||||
192.168.1.100:8080
|
||||
user:pass@192.168.1.101:8888
|
||||
# 开发模式(热重载)
|
||||
make dev
|
||||
```
|
||||
|
||||
### CSV 文件格式
|
||||
|
||||
列名:`protocol,host,port,username,password,group,tags`
|
||||
|
||||
```csv
|
||||
protocol,host,port,username,password,group,tags
|
||||
http,1.2.3.4,8080,user,pass,default,tag1;tag2
|
||||
socks5,5.6.7.8,1080,,,default,
|
||||
http,9.9.9.9,3128,admin,secret,vip,premium;fast
|
||||
```
|
||||
|
||||
- `tags` 使用分号 `;` 分隔多个标签
|
||||
- `username` 和 `password` 可留空
|
||||
|
||||
### 协议自动推断
|
||||
|
||||
当使用 `host:port` 或 `user:pass@host:port` 格式时,系统根据端口自动推断协议:
|
||||
|
||||
| 端口 | 推断协议 |
|
||||
|------|----------|
|
||||
| 443 | `https` |
|
||||
| 1080 | `socks5` |
|
||||
| 其他 | `http` |
|
||||
|
||||
可通过 `protocol_hint` 参数强制指定协议。
|
||||
|
||||
---
|
||||
|
||||
## API 接口
|
||||
|
||||
### 鉴权
|
||||
所有 API 请求需要在 Header 中携带 `X-API-Key`。
|
||||
|
||||
如果配置了 `API_KEY`,请求时需携带:
|
||||
### 代理导入
|
||||
|
||||
```bash
|
||||
# 方式一:Authorization Header
|
||||
-H "Authorization: Bearer your-api-key"
|
||||
|
||||
# 方式二:X-API-Key Header
|
||||
-H "X-API-Key: your-api-key"
|
||||
```
|
||||
|
||||
### 1. 文本导入
|
||||
|
||||
`POST /v1/proxies/import/text`
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/import/text" \
|
||||
# 文本导入
|
||||
curl -X POST http://localhost:9987/v1/proxies/import/text \
|
||||
-H "X-API-Key: your-secret-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"group": "default",
|
||||
"tags": ["batch-01"],
|
||||
"protocol_hint": "auto",
|
||||
"text": "http://user:pass@1.2.3.4:8080\nsocks5://5.6.7.8:1080\n9.9.9.9:3128"
|
||||
}'
|
||||
-d '{"text": "socks5://user:pass@1.2.3.4:1080\nhttp://5.6.7.8:8080"}'
|
||||
|
||||
# 文件导入
|
||||
curl -X POST http://localhost:9987/v1/proxies/import/file \
|
||||
-H "X-API-Key: your-secret-key" \
|
||||
-F "file=@proxies.txt"
|
||||
```
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"imported": 2,
|
||||
"duplicated": 1,
|
||||
"invalid": 0,
|
||||
"invalid_items": []
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 文件上传
|
||||
|
||||
`POST /v1/proxies/import/file`
|
||||
### 获取代理
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/import/file" \
|
||||
-F "file=@proxies.txt" \
|
||||
-F "group=default" \
|
||||
-F "tags=batch-01,imported" \
|
||||
-F "protocol_hint=auto" \
|
||||
-F "type=auto"
|
||||
# 获取下一个可用代理
|
||||
curl http://localhost:9987/v1/proxies/next \
|
||||
-H "X-API-Key: your-secret-key"
|
||||
|
||||
# 指定分组和站点
|
||||
curl "http://localhost:9987/v1/proxies/next?group=premium&site=example.com" \
|
||||
-H "X-API-Key: your-secret-key"
|
||||
```
|
||||
|
||||
| 参数 | 说明 |
|
||||
|------|------|
|
||||
| `file` | 上传的文件(必填) |
|
||||
| `group` | 分组名(默认 `default`) |
|
||||
| `tags` | 标签,逗号分隔 |
|
||||
| `type` | 文件类型:`auto`/`txt`/`csv` |
|
||||
| `protocol_hint` | 协议提示:`auto`/`http`/`https`/`socks5` |
|
||||
|
||||
### 3. 测试代理
|
||||
|
||||
`POST /v1/proxies/test`
|
||||
### 代理测试
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/test" \
|
||||
# 批量测试代理
|
||||
curl -X POST http://localhost:9987/v1/proxies/test \
|
||||
-H "X-API-Key: your-secret-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"group": "default",
|
||||
"filter": {
|
||||
"status": ["unknown", "alive"],
|
||||
"tags_any": ["batch-01"],
|
||||
"limit": 100
|
||||
},
|
||||
"test_spec": {
|
||||
"url": "https://httpbin.org/ip",
|
||||
"method": "GET",
|
||||
"timeout_ms": 5000,
|
||||
"expect_status": [200]
|
||||
},
|
||||
"concurrency": 50,
|
||||
"update_store": true,
|
||||
"write_log": true
|
||||
}'
|
||||
-d '{"group": "default", "limit": 100}'
|
||||
```
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": {
|
||||
"tested": 100,
|
||||
"alive": 75,
|
||||
"dead": 25
|
||||
},
|
||||
"results": [
|
||||
{"proxy_id": "uuid", "ok": true, "latency_ms": 340, "error": ""},
|
||||
{"proxy_id": "uuid", "ok": false, "latency_ms": 5000, "error": "timeout"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 获取代理(轮询)
|
||||
|
||||
`GET /v1/proxies/next`
|
||||
### 使用反馈
|
||||
|
||||
```bash
|
||||
curl "http://localhost:8080/v1/proxies/next?group=default&policy=round_robin&site=https://example.com"
|
||||
```
|
||||
|
||||
| 参数 | 说明 |
|
||||
|------|------|
|
||||
| `group` | 分组名(默认 `default`) |
|
||||
| `policy` | 策略:`round_robin`/`random`/`weighted` |
|
||||
| `site` | 目标站点(用于分组轮询) |
|
||||
| `tags_any` | 标签过滤,逗号分隔 |
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"proxy": {
|
||||
"id": "uuid",
|
||||
"protocol": "http",
|
||||
"host": "1.2.3.4",
|
||||
"port": 8080,
|
||||
"username": "user",
|
||||
"password": "pass"
|
||||
},
|
||||
"lease_id": "lease_abc123",
|
||||
"ttl_ms": 60000
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 上报结果
|
||||
|
||||
`POST /v1/proxies/report`
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/report" \
|
||||
# 报告代理使用结果
|
||||
curl -X POST http://localhost:9987/v1/proxies/report \
|
||||
-H "X-API-Key: your-secret-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"lease_id": "lease_abc123",
|
||||
"proxy_id": "uuid",
|
||||
"success": true,
|
||||
"latency_ms": 500
|
||||
}'
|
||||
-d '{"lease_id": "xxx", "success": true}'
|
||||
```
|
||||
|
||||
上报结果会影响代理的分数:
|
||||
- 成功:`score +1`
|
||||
- 失败:`score -3`
|
||||
|
||||
### 6. 健康检查
|
||||
|
||||
`GET /health`
|
||||
### 代理管理
|
||||
|
||||
```bash
|
||||
curl "http://localhost:8080/health"
|
||||
# 列出代理
|
||||
curl "http://localhost:9987/v1/proxies?group=default&status=alive" \
|
||||
-H "X-API-Key: your-secret-key"
|
||||
|
||||
# 获取统计信息
|
||||
curl http://localhost:9987/v1/proxies/stats \
|
||||
-H "X-API-Key: your-secret-key"
|
||||
|
||||
# 删除代理
|
||||
curl -X DELETE http://localhost:9987/v1/proxies/{id} \
|
||||
-H "X-API-Key: your-secret-key"
|
||||
```
|
||||
|
||||
---
|
||||
## 项目结构
|
||||
|
||||
## 分发策略
|
||||
```
|
||||
.
|
||||
├── server/ # 程序入口
|
||||
├── internal/
|
||||
│ ├── api/ # HTTP 路由和处理器
|
||||
│ ├── config/ # 配置加载
|
||||
│ ├── importer/ # 代理导入和解析
|
||||
│ ├── model/ # 数据模型
|
||||
│ ├── security/ # 安全相关
|
||||
│ ├── selector/ # 代理选择器
|
||||
│ ├── store/ # 数据库存储
|
||||
│ ├── telegram/ # Telegram Bot
|
||||
│ └── tester/ # 代理测试器
|
||||
└── migrations/ # 数据库迁移
|
||||
```
|
||||
|
||||
| 策略 | 说明 |
|
||||
|------|------|
|
||||
| `round_robin` | 轮询(默认),按顺序依次返回 |
|
||||
| `random` | 随机选择 |
|
||||
| `weighted` | 加权随机,分数高的代理被选中概率更大 |
|
||||
|
||||
---
|
||||
|
||||
## 构建命令
|
||||
## 开发
|
||||
|
||||
```bash
|
||||
make build # 构建二进制
|
||||
make run # 运行服务
|
||||
make test # 运行测试
|
||||
make migrate # 数据库迁移
|
||||
make fmt # 格式化代码
|
||||
make lint # 静态检查
|
||||
# 格式化代码
|
||||
make fmt
|
||||
|
||||
# 静态检查
|
||||
make lint
|
||||
|
||||
# 运行测试
|
||||
make test
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Reference in New Issue
Block a user