304 lines
5.9 KiB
Markdown
304 lines
5.9 KiB
Markdown
# ProxyRotator
|
||
|
||
代理池管理系统 - 支持代理导入、测试、轮询分发和结果上报。
|
||
|
||
## 快速开始
|
||
|
||
### 1. 安装依赖
|
||
|
||
```bash
|
||
go mod tidy
|
||
```
|
||
|
||
### 2. 初始化数据库
|
||
|
||
```bash
|
||
# 创建 PostgreSQL 数据库
|
||
createdb proxyrotator
|
||
|
||
# 执行迁移脚本
|
||
export DATABASE_URL="postgres://user:pass@localhost:5432/proxyrotator?sslmode=disable"
|
||
make migrate
|
||
```
|
||
|
||
### 3. 启动服务
|
||
|
||
```bash
|
||
make run
|
||
```
|
||
|
||
## 配置
|
||
|
||
通过环境变量配置,或创建 `.env` 文件:
|
||
|
||
```env
|
||
DATABASE_URL=postgres://user:pass@localhost:5432/proxyrotator?sslmode=disable
|
||
LISTEN_ADDR=:8080
|
||
API_KEY=your-secret-key
|
||
RETURN_SECRET=true
|
||
MAX_CONCURRENCY=200
|
||
MAX_TEST_LIMIT=2000
|
||
LEASE_TTL=60s
|
||
```
|
||
|
||
| 变量 | 说明 | 默认值 |
|
||
|------|------|--------|
|
||
| `DATABASE_URL` | PostgreSQL 连接串 | - |
|
||
| `LISTEN_ADDR` | 监听地址 | `:8080` |
|
||
| `API_KEY` | API 密钥(空则不鉴权) | - |
|
||
| `RETURN_SECRET` | 是否返回代理密码 | `true` |
|
||
| `MAX_CONCURRENCY` | 测试最大并发数 | `200` |
|
||
| `MAX_TEST_LIMIT` | 单次测试上限 | `2000` |
|
||
| `LEASE_TTL` | 租约有效期 | `60s` |
|
||
|
||
---
|
||
|
||
## 代理格式
|
||
|
||
### 支持的格式
|
||
|
||
| 格式 | 示例 |
|
||
|------|------|
|
||
| `host:port` | `192.168.1.1:8080` |
|
||
| `user:pass@host:port` | `admin:123456@192.168.1.1:8080` |
|
||
| `http://host:port` | `http://192.168.1.1:8080` |
|
||
| `http://user:pass@host:port` | `http://admin:123456@192.168.1.1:8080` |
|
||
| `https://host:port` | `https://192.168.1.1:443` |
|
||
| `socks5://host:port` | `socks5://192.168.1.1:1080` |
|
||
| `socks5://user:pass@host:port` | `socks5://admin:123456@192.168.1.1:1080` |
|
||
|
||
### TXT 文件格式
|
||
|
||
每行一个代理,支持 `#` 注释:
|
||
|
||
```txt
|
||
# HTTP 代理
|
||
http://user:pass@1.2.3.4:8080
|
||
http://5.6.7.8:3128
|
||
|
||
# SOCKS5 代理
|
||
socks5://admin:pwd@10.0.0.1:1080
|
||
socks5://10.0.0.2:1080
|
||
|
||
# 简写格式(自动推断协议)
|
||
192.168.1.100:8080
|
||
user:pass@192.168.1.101:8888
|
||
```
|
||
|
||
### CSV 文件格式
|
||
|
||
列名:`protocol,host,port,username,password,group,tags`
|
||
|
||
```csv
|
||
protocol,host,port,username,password,group,tags
|
||
http,1.2.3.4,8080,user,pass,default,tag1;tag2
|
||
socks5,5.6.7.8,1080,,,default,
|
||
http,9.9.9.9,3128,admin,secret,vip,premium;fast
|
||
```
|
||
|
||
- `tags` 使用分号 `;` 分隔多个标签
|
||
- `username` 和 `password` 可留空
|
||
|
||
### 协议自动推断
|
||
|
||
当使用 `host:port` 或 `user:pass@host:port` 格式时,系统根据端口自动推断协议:
|
||
|
||
| 端口 | 推断协议 |
|
||
|------|----------|
|
||
| 443 | `https` |
|
||
| 1080 | `socks5` |
|
||
| 其他 | `http` |
|
||
|
||
可通过 `protocol_hint` 参数强制指定协议。
|
||
|
||
---
|
||
|
||
## API 接口
|
||
|
||
### 鉴权
|
||
|
||
如果配置了 `API_KEY`,请求时需携带:
|
||
|
||
```bash
|
||
# 方式一:Authorization Header
|
||
-H "Authorization: Bearer your-api-key"
|
||
|
||
# 方式二:X-API-Key Header
|
||
-H "X-API-Key: your-api-key"
|
||
```
|
||
|
||
### 1. 文本导入
|
||
|
||
`POST /v1/proxies/import/text`
|
||
|
||
```bash
|
||
curl -X POST "http://localhost:8080/v1/proxies/import/text" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"group": "default",
|
||
"tags": ["batch-01"],
|
||
"protocol_hint": "auto",
|
||
"text": "http://user:pass@1.2.3.4:8080\nsocks5://5.6.7.8:1080\n9.9.9.9:3128"
|
||
}'
|
||
```
|
||
|
||
**响应:**
|
||
|
||
```json
|
||
{
|
||
"imported": 2,
|
||
"duplicated": 1,
|
||
"invalid": 0,
|
||
"invalid_items": []
|
||
}
|
||
```
|
||
|
||
### 2. 文件上传
|
||
|
||
`POST /v1/proxies/import/file`
|
||
|
||
```bash
|
||
curl -X POST "http://localhost:8080/v1/proxies/import/file" \
|
||
-F "file=@proxies.txt" \
|
||
-F "group=default" \
|
||
-F "tags=batch-01,imported" \
|
||
-F "protocol_hint=auto" \
|
||
-F "type=auto"
|
||
```
|
||
|
||
| 参数 | 说明 |
|
||
|------|------|
|
||
| `file` | 上传的文件(必填) |
|
||
| `group` | 分组名(默认 `default`) |
|
||
| `tags` | 标签,逗号分隔 |
|
||
| `type` | 文件类型:`auto`/`txt`/`csv` |
|
||
| `protocol_hint` | 协议提示:`auto`/`http`/`https`/`socks5` |
|
||
|
||
### 3. 测试代理
|
||
|
||
`POST /v1/proxies/test`
|
||
|
||
```bash
|
||
curl -X POST "http://localhost:8080/v1/proxies/test" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"group": "default",
|
||
"filter": {
|
||
"status": ["unknown", "alive"],
|
||
"tags_any": ["batch-01"],
|
||
"limit": 100
|
||
},
|
||
"test_spec": {
|
||
"url": "https://httpbin.org/ip",
|
||
"method": "GET",
|
||
"timeout_ms": 5000,
|
||
"expect_status": [200]
|
||
},
|
||
"concurrency": 50,
|
||
"update_store": true,
|
||
"write_log": true
|
||
}'
|
||
```
|
||
|
||
**响应:**
|
||
|
||
```json
|
||
{
|
||
"summary": {
|
||
"tested": 100,
|
||
"alive": 75,
|
||
"dead": 25
|
||
},
|
||
"results": [
|
||
{"proxy_id": "uuid", "ok": true, "latency_ms": 340, "error": ""},
|
||
{"proxy_id": "uuid", "ok": false, "latency_ms": 5000, "error": "timeout"}
|
||
]
|
||
}
|
||
```
|
||
|
||
### 4. 获取代理(轮询)
|
||
|
||
`GET /v1/proxies/next`
|
||
|
||
```bash
|
||
curl "http://localhost:8080/v1/proxies/next?group=default&policy=round_robin&site=https://example.com"
|
||
```
|
||
|
||
| 参数 | 说明 |
|
||
|------|------|
|
||
| `group` | 分组名(默认 `default`) |
|
||
| `policy` | 策略:`round_robin`/`random`/`weighted` |
|
||
| `site` | 目标站点(用于分组轮询) |
|
||
| `tags_any` | 标签过滤,逗号分隔 |
|
||
|
||
**响应:**
|
||
|
||
```json
|
||
{
|
||
"proxy": {
|
||
"id": "uuid",
|
||
"protocol": "http",
|
||
"host": "1.2.3.4",
|
||
"port": 8080,
|
||
"username": "user",
|
||
"password": "pass"
|
||
},
|
||
"lease_id": "lease_abc123",
|
||
"ttl_ms": 60000
|
||
}
|
||
```
|
||
|
||
### 5. 上报结果
|
||
|
||
`POST /v1/proxies/report`
|
||
|
||
```bash
|
||
curl -X POST "http://localhost:8080/v1/proxies/report" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"lease_id": "lease_abc123",
|
||
"proxy_id": "uuid",
|
||
"success": true,
|
||
"latency_ms": 500
|
||
}'
|
||
```
|
||
|
||
上报结果会影响代理的分数:
|
||
- 成功:`score +1`
|
||
- 失败:`score -3`
|
||
|
||
### 6. 健康检查
|
||
|
||
`GET /health`
|
||
|
||
```bash
|
||
curl "http://localhost:8080/health"
|
||
```
|
||
|
||
---
|
||
|
||
## 分发策略
|
||
|
||
| 策略 | 说明 |
|
||
|------|------|
|
||
| `round_robin` | 轮询(默认),按顺序依次返回 |
|
||
| `random` | 随机选择 |
|
||
| `weighted` | 加权随机,分数高的代理被选中概率更大 |
|
||
|
||
---
|
||
|
||
## 构建命令
|
||
|
||
```bash
|
||
make build # 构建二进制
|
||
make run # 运行服务
|
||
make test # 运行测试
|
||
make migrate # 数据库迁移
|
||
make fmt # 格式化代码
|
||
make lint # 静态检查
|
||
```
|
||
|
||
## License
|
||
|
||
MIT
|