frist
This commit is contained in:
dela
303
README.md
Normal file
303
README.md
Normal file
@@ -0,0 +1,303 @@
|
||||
# ProxyRotator
|
||||
|
||||
代理池管理系统 - 支持代理导入、测试、轮询分发和结果上报。
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1. 安装依赖
|
||||
|
||||
```bash
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
### 2. 初始化数据库
|
||||
|
||||
```bash
|
||||
# 创建 PostgreSQL 数据库
|
||||
createdb proxyrotator
|
||||
|
||||
# 执行迁移脚本
|
||||
export DATABASE_URL="postgres://user:pass@localhost:5432/proxyrotator?sslmode=disable"
|
||||
make migrate
|
||||
```
|
||||
|
||||
### 3. 启动服务
|
||||
|
||||
```bash
|
||||
make run
|
||||
```
|
||||
|
||||
## 配置
|
||||
|
||||
通过环境变量配置，或创建 `.env` 文件：
|
||||
|
||||
```env
|
||||
DATABASE_URL=postgres://user:pass@localhost:5432/proxyrotator?sslmode=disable
|
||||
LISTEN_ADDR=:8080
|
||||
API_KEY=your-secret-key
|
||||
RETURN_SECRET=true
|
||||
MAX_CONCURRENCY=200
|
||||
MAX_TEST_LIMIT=2000
|
||||
LEASE_TTL=60s
|
||||
```
|
||||
|
||||
| 变量 | 说明 | 默认值 |
|
||||
|------|------|--------|
|
||||
| `DATABASE_URL` | PostgreSQL 连接串 | - |
|
||||
| `LISTEN_ADDR` | 监听地址 | `:8080` |
|
||||
| `API_KEY` | API 密钥（空则不鉴权） | - |
|
||||
| `RETURN_SECRET` | 是否返回代理密码 | `true` |
|
||||
| `MAX_CONCURRENCY` | 测试最大并发数 | `200` |
|
||||
| `MAX_TEST_LIMIT` | 单次测试上限 | `2000` |
|
||||
| `LEASE_TTL` | 租约有效期 | `60s` |
|
||||
|
||||
---
|
||||
|
||||
## 代理格式
|
||||
|
||||
### 支持的格式
|
||||
|
||||
| 格式 | 示例 |
|
||||
|------|------|
|
||||
| `host:port` | `192.168.1.1:8080` |
|
||||
| `user:pass@host:port` | `admin:123456@192.168.1.1:8080` |
|
||||
| `http://host:port` | `http://192.168.1.1:8080` |
|
||||
| `http://user:pass@host:port` | `http://admin:123456@192.168.1.1:8080` |
|
||||
| `https://host:port` | `https://192.168.1.1:443` |
|
||||
| `socks5://host:port` | `socks5://192.168.1.1:1080` |
|
||||
| `socks5://user:pass@host:port` | `socks5://admin:123456@192.168.1.1:1080` |
|
||||
|
||||
### TXT 文件格式
|
||||
|
||||
每行一个代理，支持 `#` 注释：
|
||||
|
||||
```txt
|
||||
# HTTP 代理
|
||||
http://user:pass@1.2.3.4:8080
|
||||
http://5.6.7.8:3128
|
||||
|
||||
# SOCKS5 代理
|
||||
socks5://admin:pwd@10.0.0.1:1080
|
||||
socks5://10.0.0.2:1080
|
||||
|
||||
# 简写格式（自动推断协议）
|
||||
192.168.1.100:8080
|
||||
user:pass@192.168.1.101:8888
|
||||
```
|
||||
|
||||
### CSV 文件格式
|
||||
|
||||
列名：`protocol,host,port,username,password,group,tags`
|
||||
|
||||
```csv
|
||||
protocol,host,port,username,password,group,tags
|
||||
http,1.2.3.4,8080,user,pass,default,tag1;tag2
|
||||
socks5,5.6.7.8,1080,,,default,
|
||||
http,9.9.9.9,3128,admin,secret,vip,premium;fast
|
||||
```
|
||||
|
||||
- `tags` 使用分号 `;` 分隔多个标签
|
||||
- `username` 和 `password` 可留空
|
||||
|
||||
### 协议自动推断
|
||||
|
||||
当使用 `host:port` 或 `user:pass@host:port` 格式时，系统根据端口自动推断协议：
|
||||
|
||||
| 端口 | 推断协议 |
|
||||
|------|----------|
|
||||
| 443 | `https` |
|
||||
| 1080 | `socks5` |
|
||||
| 其他 | `http` |
|
||||
|
||||
可通过 `protocol_hint` 参数强制指定协议。
|
||||
|
||||
---
|
||||
|
||||
## API 接口
|
||||
|
||||
### 鉴权
|
||||
|
||||
如果配置了 `API_KEY`，请求时需携带：
|
||||
|
||||
```bash
|
||||
# 方式一：Authorization Header
|
||||
-H "Authorization: Bearer your-api-key"
|
||||
|
||||
# 方式二：X-API-Key Header
|
||||
-H "X-API-Key: your-api-key"
|
||||
```
|
||||
|
||||
### 1. 文本导入
|
||||
|
||||
`POST /v1/proxies/import/text`
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/import/text" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"group": "default",
|
||||
"tags": ["batch-01"],
|
||||
"protocol_hint": "auto",
|
||||
"text": "http://user:pass@1.2.3.4:8080\nsocks5://5.6.7.8:1080\n9.9.9.9:3128"
|
||||
}'
|
||||
```
|
||||
|
||||
**响应：**
|
||||
|
||||
```json
|
||||
{
|
||||
"imported": 2,
|
||||
"duplicated": 1,
|
||||
"invalid": 0,
|
||||
"invalid_items": []
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 文件上传
|
||||
|
||||
`POST /v1/proxies/import/file`
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/import/file" \
|
||||
-F "file=@proxies.txt" \
|
||||
-F "group=default" \
|
||||
-F "tags=batch-01,imported" \
|
||||
-F "protocol_hint=auto" \
|
||||
-F "type=auto"
|
||||
```
|
||||
|
||||
| 参数 | 说明 |
|
||||
|------|------|
|
||||
| `file` | 上传的文件（必填） |
|
||||
| `group` | 分组名（默认 `default`） |
|
||||
| `tags` | 标签，逗号分隔 |
|
||||
| `type` | 文件类型：`auto`/`txt`/`csv` |
|
||||
| `protocol_hint` | 协议提示：`auto`/`http`/`https`/`socks5` |
|
||||
|
||||
### 3. 测试代理
|
||||
|
||||
`POST /v1/proxies/test`
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/test" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"group": "default",
|
||||
"filter": {
|
||||
"status": ["unknown", "alive"],
|
||||
"tags_any": ["batch-01"],
|
||||
"limit": 100
|
||||
},
|
||||
"test_spec": {
|
||||
"url": "https://httpbin.org/ip",
|
||||
"method": "GET",
|
||||
"timeout_ms": 5000,
|
||||
"expect_status": [200]
|
||||
},
|
||||
"concurrency": 50,
|
||||
"update_store": true,
|
||||
"write_log": true
|
||||
}'
|
||||
```
|
||||
|
||||
**响应：**
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": {
|
||||
"tested": 100,
|
||||
"alive": 75,
|
||||
"dead": 25
|
||||
},
|
||||
"results": [
|
||||
{"proxy_id": "uuid", "ok": true, "latency_ms": 340, "error": ""},
|
||||
{"proxy_id": "uuid", "ok": false, "latency_ms": 5000, "error": "timeout"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 获取代理（轮询）
|
||||
|
||||
`GET /v1/proxies/next`
|
||||
|
||||
```bash
|
||||
curl "http://localhost:8080/v1/proxies/next?group=default&policy=round_robin&site=https://example.com"
|
||||
```
|
||||
|
||||
| 参数 | 说明 |
|
||||
|------|------|
|
||||
| `group` | 分组名（默认 `default`） |
|
||||
| `policy` | 策略：`round_robin`/`random`/`weighted` |
|
||||
| `site` | 目标站点（用于分组轮询） |
|
||||
| `tags_any` | 标签过滤，逗号分隔 |
|
||||
|
||||
**响应：**
|
||||
|
||||
```json
|
||||
{
|
||||
"proxy": {
|
||||
"id": "uuid",
|
||||
"protocol": "http",
|
||||
"host": "1.2.3.4",
|
||||
"port": 8080,
|
||||
"username": "user",
|
||||
"password": "pass"
|
||||
},
|
||||
"lease_id": "lease_abc123",
|
||||
"ttl_ms": 60000
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 上报结果
|
||||
|
||||
`POST /v1/proxies/report`
|
||||
|
||||
```bash
|
||||
curl -X POST "http://localhost:8080/v1/proxies/report" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"lease_id": "lease_abc123",
|
||||
"proxy_id": "uuid",
|
||||
"success": true,
|
||||
"latency_ms": 500
|
||||
}'
|
||||
```
|
||||
|
||||
上报结果会影响代理的分数：
|
||||
- 成功：`score +1`
|
||||
- 失败：`score -3`
|
||||
|
||||
### 6. 健康检查
|
||||
|
||||
`GET /health`
|
||||
|
||||
```bash
|
||||
curl "http://localhost:8080/health"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 分发策略
|
||||
|
||||
| 策略 | 说明 |
|
||||
|------|------|
|
||||
| `round_robin` | 轮询（默认），按顺序依次返回 |
|
||||
| `random` | 随机选择 |
|
||||
| `weighted` | 加权随机，分数高的代理被选中概率更大 |
|
||||
|
||||
---
|
||||
|
||||
## 构建命令
|
||||
|
||||
```bash
|
||||
make build # 构建二进制
|
||||
make run # 运行服务
|
||||
make test # 运行测试
|
||||
make migrate # 数据库迁移
|
||||
make fmt # 格式化代码
|
||||
make lint # 静态检查
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Reference in New Issue
Block a user