0131c8b408
- Add Free_Proxy_Website-style fpw_* plugins and register them - Per-plugin crawl timeout (crawl_timeout_seconds=120); remove global crawl_timeout setting - Validator: fix connect vs total timeout on save; SOCKS session LRU cache; drop redundant semaphore - Validation handler uses single DB connection; batch upsert after crawl; WorkerPool put_nowait - Remove unused max_retries from settings API/UI; settings maintenance SQL + init_db cleanup of deprecated keys - WebSocket dashboard stats; ProxyList pool_filter and API alignment - POST /api/proxies/delete-one for IPv6-safe deletes; task poll stops on 404 - pytest uses PROXYPOOL_DB_PATH=db/proxies.test.sqlite so tests do not wipe production DB - .gitignore: explicit proxies.test.sqlite patterns; fix plugin_service ValidationException import Made-with: Cursor
182 lines
4.8 KiB
Markdown
182 lines
4.8 KiB
Markdown
# 测试说明
|
||
|
||
## 测试结构
|
||
|
||
```
|
||
tests/
|
||
├── conftest.py # pytest 配置和 fixtures
|
||
├── task_utils.py # 异步任务轮询（集成/E2E 共用）
|
||
├── support/ # 测试专用插件类等（非 mock）
|
||
├── README.md # 本文件
|
||
├── unit/ # 单元测试
|
||
│ ├── test_models.py # 模型测试
|
||
│ └── test_repositories.py # 仓库层测试
|
||
├── integration/ # 集成测试
|
||
│ ├── test_proxies_api.py # 代理 API 测试
|
||
│ ├── test_plugins_api.py # 插件 API 测试
|
||
│ ├── test_plugins_live_crawl.py # 各插件真实爬取验收（须外网）
|
||
│ ├── test_scheduler_api.py # 调度器 API 测试
|
||
│ ├── test_settings_api.py # 设置 API 测试
|
||
│ └── test_health_api.py # 健康检查测试
|
||
└── e2e/ # 端到端测试
|
||
└── test_full_workflow.py # 完整工作流测试
|
||
```
|
||
|
||
## 网络与真实调用
|
||
|
||
集成测试与 E2E **不再 mock** `PluginRunner` / `ValidatorService`：会发起真实 HTTP 爬取与代理验证（视设置而定）。运行全量 `pytest` 需要 **可用的出站网络**，且含 `network` / `slow` 标记的用例可能耗时数分钟。
|
||
|
||
跳过需外网的用例（例如离线快速检查）：
|
||
|
||
```bash
|
||
pytest -m "not network"
|
||
```
|
||
|
||
**插件爬取验收**（`test_plugins_live_crawl.py`）：
|
||
|
||
- 核心 8 插件：必须至少 1 条代理且无 Runner 失败。
|
||
- `fpw_*`：对照 [Free_Proxy_Website](https://github.com/cyubuchen/Free_Proxy_Website) 的公开源，允许 0 条（国际网络差异），使用更长超时。
|
||
|
||
```bash
|
||
pytest tests/integration/test_plugins_live_crawl.py -v
|
||
```
|
||
|
||
## 运行测试
|
||
|
||
### 安装测试依赖
|
||
|
||
```bash
|
||
pip install pytest pytest-asyncio httpx
|
||
```
|
||
|
||
### 运行所有测试
|
||
|
||
```bash
|
||
pytest
|
||
```
|
||
|
||
### 运行特定类型的测试
|
||
|
||
```bash
|
||
# 仅运行单元测试
|
||
pytest tests/unit -v
|
||
|
||
# 仅运行集成测试
|
||
pytest tests/integration -v
|
||
|
||
# 仅运行 E2E 测试
|
||
pytest tests/e2e -v
|
||
```
|
||
|
||
### 运行特定测试文件
|
||
|
||
```bash
|
||
pytest tests/integration/test_proxies_api.py -v
|
||
```
|
||
|
||
### 运行特定测试函数
|
||
|
||
```bash
|
||
pytest tests/integration/test_proxies_api.py::TestProxiesAPI::test_get_stats -v
|
||
```
|
||
|
||
## 测试覆盖的 API
|
||
|
||
### 代理 API (`/api/proxies/*`)
|
||
|
||
- ✅ `GET /api/proxies/stats` - 获取统计信息
|
||
- ✅ `POST /api/proxies` - 列出代理
|
||
- ✅ `GET /api/proxies/random` - 获取随机代理
|
||
- ✅ `GET /api/proxies/export/{format}` - 导出代理 (csv, txt, json)
|
||
- ✅ `DELETE /api/proxies/{ip}/{port}` - 删除代理
|
||
- ✅ `POST /api/proxies/batch-delete` - 批量删除
|
||
- ✅ `DELETE /api/proxies/clean-invalid` - 清理无效代理
|
||
|
||
### 插件 API (`/api/plugins/*`)
|
||
|
||
- ✅ `GET /api/plugins` - 列出插件
|
||
- ✅ `PUT /api/plugins/{id}/toggle` - 切换插件状态
|
||
- ✅ `GET /api/plugins/{id}/config` - 获取插件配置
|
||
- ✅ `POST /api/plugins/{id}/config` - 更新插件配置
|
||
- ✅ `POST /api/plugins/{id}/crawl` - 触发单个插件爬取
|
||
- ✅ `POST /api/plugins/crawl-all` - 触发所有插件爬取
|
||
|
||
### 调度器 API (`/api/scheduler/*`)
|
||
|
||
- ✅ `GET /api/scheduler/status` - 获取调度器状态
|
||
- ✅ `POST /api/scheduler/start` - 启动调度器
|
||
- ✅ `POST /api/scheduler/stop` - 停止调度器
|
||
- ✅ `POST /api/scheduler/validate-now` - 立即验证
|
||
|
||
### 设置 API (`/api/settings`)
|
||
|
||
- ✅ `GET /api/settings` - 获取设置
|
||
- ✅ `POST /api/settings` - 保存设置
|
||
|
||
### 健康检查
|
||
|
||
- ✅ `GET /` - 根端点
|
||
- ✅ `GET /health` - 健康检查
|
||
|
||
## 测试 Fixtures
|
||
|
||
### `client`
|
||
|
||
异步 HTTP 客户端，用于发送请求到测试应用。
|
||
|
||
```python
|
||
async def test_example(client):
|
||
response = await client.get("/api/proxies/stats")
|
||
assert response.status_code == 200
|
||
```
|
||
|
||
### `db`
|
||
|
||
数据库连接 fixture。
|
||
|
||
```python
|
||
async def test_example(db, proxy_repo):
|
||
await proxy_repo.insert_or_update(db, "192.168.1.1", 8080, "http", 50)
|
||
```
|
||
|
||
### `sample_proxy`
|
||
|
||
创建一个测试代理并自动清理。
|
||
|
||
```python
|
||
async def test_example(client, sample_proxy):
|
||
# sample_proxy = {"ip": "192.168.1.1", "port": 8080, "protocol": "http", "score": 50}
|
||
response = await client.delete(f"/api/proxies/{sample_proxy['ip']}/{sample_proxy['port']}")
|
||
assert response.status_code == 200
|
||
```
|
||
|
||
## 编写新测试
|
||
|
||
### 单元测试示例
|
||
|
||
```python
|
||
# tests/unit/test_new_feature.py
|
||
import pytest
|
||
from app.models.domain import ProxyRaw
|
||
|
||
|
||
class TestProxyRaw:
|
||
def test_create(self):
|
||
proxy = ProxyRaw("192.168.1.1", 8080, "http")
|
||
assert proxy.ip == "192.168.1.1"
|
||
```
|
||
|
||
### 集成测试示例
|
||
|
||
```python
|
||
# tests/integration/test_new_api.py
|
||
import pytest
|
||
|
||
|
||
class TestNewAPI:
|
||
@pytest.mark.asyncio
|
||
async def test_new_endpoint(self, client):
|
||
response = await client.get("/api/new-endpoint")
|
||
assert response.status_code == 200
|
||
```
|