feat: fpw plugins, validation/crawl perf, WS stats, test DB isolation
- Add Free_Proxy_Website-style fpw_* plugins and register them - Per-plugin crawl timeout (crawl_timeout_seconds=120); remove global crawl_timeout setting - Validator: fix connect vs total timeout on save; SOCKS session LRU cache; drop redundant semaphore - Validation handler uses single DB connection; batch upsert after crawl; WorkerPool put_nowait - Remove unused max_retries from settings API/UI; settings maintenance SQL + init_db cleanup of deprecated keys - WebSocket dashboard stats; ProxyList pool_filter and API alignment - POST /api/proxies/delete-one for IPv6-safe deletes; task poll stops on 404 - pytest uses PROXYPOOL_DB_PATH=db/proxies.test.sqlite so tests do not wipe production DB - .gitignore: explicit proxies.test.sqlite patterns; fix plugin_service ValidationException import Made-with: Cursor
This commit is contained in:
@@ -5,6 +5,8 @@
|
||||
```
|
||||
tests/
|
||||
├── conftest.py # pytest 配置和 fixtures
|
||||
├── task_utils.py # 异步任务轮询(集成/E2E 共用)
|
||||
├── support/ # 测试专用插件类等(非 mock)
|
||||
├── README.md # 本文件
|
||||
├── unit/ # 单元测试
|
||||
│ ├── test_models.py # 模型测试
|
||||
@@ -12,6 +14,7 @@ tests/
|
||||
├── integration/ # 集成测试
|
||||
│ ├── test_proxies_api.py # 代理 API 测试
|
||||
│ ├── test_plugins_api.py # 插件 API 测试
|
||||
│ ├── test_plugins_live_crawl.py # 各插件真实爬取验收(须外网)
|
||||
│ ├── test_scheduler_api.py # 调度器 API 测试
|
||||
│ ├── test_settings_api.py # 设置 API 测试
|
||||
│ └── test_health_api.py # 健康检查测试
|
||||
@@ -19,6 +22,25 @@ tests/
|
||||
└── test_full_workflow.py # 完整工作流测试
|
||||
```
|
||||
|
||||
## 网络与真实调用
|
||||
|
||||
集成测试与 E2E **不再 mock** `PluginRunner` / `ValidatorService`:会发起真实 HTTP 爬取与代理验证(视设置而定)。运行全量 `pytest` 需要 **可用的出站网络**,且含 `network` / `slow` 标记的用例可能耗时数分钟。
|
||||
|
||||
跳过需外网的用例(例如离线快速检查):
|
||||
|
||||
```bash
|
||||
pytest -m "not network"
|
||||
```
|
||||
|
||||
**插件爬取验收**(`test_plugins_live_crawl.py`):
|
||||
|
||||
- 核心 8 插件:必须至少 1 条代理且无 Runner 失败。
|
||||
- `fpw_*`:对照 [Free_Proxy_Website](https://github.com/cyubuchen/Free_Proxy_Website) 的公开源,允许 0 条(国际网络差异),使用更长超时。
|
||||
|
||||
```bash
|
||||
pytest tests/integration/test_plugins_live_crawl.py -v
|
||||
```
|
||||
|
||||
## 运行测试
|
||||
|
||||
### 安装测试依赖
|
||||
|
||||
Reference in New Issue
Block a user