Compare commits

...
10 Commits
Author SHA1 Message Date
chenxin 73ad3c02b0 启动补丁:乾坤装配参数名、注册接口双前缀、/chat 残留链接清理
- main.py:Executor(sandbox, http_client=…) 参数名与构造签名不符(http=…),致乾坤沙箱装配降级 503——修正
- 注册页手写 fetch 多拼一层 v1(/api/proxy/v1/auth/register → 后端 /api/v1/v1/* 404);BACKEND_ROOT 已含 /api/v1,proxy 调用不应再带——去掉
- /chat 页 M2b 已随线性会话退役,对话主入口为 /trees(镜树);六处残留链接(首页/登录/注册跳转/错误页/404 页/管理壳)全部改指 /trees
- 回归:前端 type-check + 203 passed;后端百工+API 套件 45 passed
2026-08-28 10:31:36 +08:00
chenxin 3c050b923c M9 补丁:启动冒烟修复——镜树列表 str 属性访问、内置能力平台作用域与种子提交缺失
真实进程冒烟(灵台+神识起进程 → 登录 → 能力目录 → 授契 → 调用)抓到三处缺陷,全量测试因分支未覆盖而漏网:
① mirror_tree.list_trees:select(Character.id).scalars() 已返回 id 字符串,c.id 属性访问在有 character 的 owner 上必炸(M2a 引入,测试只走了无 character 分支)——改为直接收集
② 内置能力种子写死 owner_scope=local-user,真实用户(UUID owner)查不到能力目录——引入 PLATFORM_SCOPE:builtin 能力挂平台域全局可见可授契,自定义能力仍按 owner 隔离
③ ensure_governance_seed 缺 session.commit——种子数据未提交即随启动 session 回滚(直调误读为成功因同 session 可见未提交数据)
冒烟证据:未授契调用 403 HJ-1303 留痕 → 授行契 → 调用 ok 返回观察标证;能力目录 4 项(builtin.datetime/echo/qiankun.shell/web.fetch);全量回归 349 passed 不变
另:本地 .env 补入神识段(SHENSHI_BASE_URL=127.0.0.1:8080 接真神识)
2026-08-28 10:06:35 +08:00
chenxin 2483a25f63 M9:桌面壳与收口——镜匣最小壳、器谱登记、发布证据落位、重写计划功成身退
- 镜匣 desktop/(Tauri v2 最小壳):WebView 承载琉璃镜(frontend/out 或 devUrl);能力桥最小命令(通知);乾坤通道 HMAC 短期凭证构造(与 sandbox/src/auth.rs 验签同算法,密钥仅经 SANDBOX_AUTH_SECRET 环境注入——二进制零平台密钥,静态审计钉死);hard_stop/resume 本地硬急停原子断路(全部通道命令过 ensure_not_halted,本地最高优先级不经云端);tantivy 依赖与 capabilities/ 目录预留;updater 插件位预留;cargo check 通过
- 器谱域 domains/tianzai(迁移 010):devices/device_grants——一次配对(密钥指纹登记)、逐项授契(可授面显式枚举,未知能力键拒)、撤回立即生效、设备撤销连带全部授契失效且不可再授;/devices 器谱 API 组
- 发布收口:release-evidence.md 落位重写期 RW-* 证据包(M0–M9 十里程碑验收自动化套件与结果物);能力矩阵 M9 行收官;rewrite-plan.md 按计划删除(M0–M9 全部完成,能力矩阵与证据台账接棒),全库引用清理
- 验收(test_m9_jingxia 四用例):器谱未授契 403 留痕/撤回即效/设备撤销连带;镜匣零平台密钥(rg 审计源码与配置为空);硬急停语义钉死(通道命令全检查+resume 唯一恢复+HMAC 无密钥不出凭证)
- 五套件终检全绿:后端 349、前端 203、BFF 4、神识 5 包、镜匣 cargo check(2 个预存环境失败历次 HEAD 复验与本周期无关);Compose nginx profile 配置校验通过
2026-08-28 09:35:27 +08:00
chenxin c63b75152a M8:天工阁、叩心与辰正——受托任务后台并行、叩心四约束、时态感知投影
- 天工阁域 domains/tiangong:backstage_tasks(委托包+验收标准+结果证据);有限分解(parent 链)与受控并行(各自后台 Run);子任务全终态级联父任务进入待验收;按证据 accept/reject
- kernel.start_backstage_run(kind=background_task):loop 后台分支不占前台席位、与前台对话并行;断言放宽容纳无人物功能任务;结果只经回调回呈任务——无身份写入路径
- 叩心域 domains/kouxin:knocking_candidates(event/time_anchor/manual 触发 + 草稿 + 裁决快照);四约束裁决(总开关→静默窗口【本地时区判定】→24h 频率→预算),dispatch 未过 429 且快照留痕;候选默认不自动发送;expire_stale 过期推进不触碰缘契
- 辰正 domains/zhengchen:temporal_context() 复用 time_sense 计算核(快照/农历/季节),产出【时态感知】投影段接入镜树生成 prompt(客观时间语境,不附情绪臆测——守一纪律)
- 退役:modules/goal_orchestrator|planner|perception|big_cycle、application/planning、services/governance|goal_persist、candidates 表链(imports 候选抽取 API 面)、goals/plans/agent_roster/perception_channels/big_cycles/旧候选表(迁移 009);前端 admin goals/plans/agents/perception/world 页与六组件;governance 只留 GenerationPreferences;admin 治理面收敛为司命+百工两枢;pivots 健康以域表计数重建
- 验收(test_m8_tiangong 五用例):后台任务无身份写入(记忆/魂籍/缘契零新增);后台与前台并行;分解级联;叩心四约束逐项;不回应不降低关系(缘契维度/状态/版本零变化)
- 四套件:后端 345、前端 203、BFF 4、神识 5 包全绿(2 个预存环境失败不变);迁移 009 净库干净
2026-08-28 09:13:39 +08:00
chenxin b419e0afde M7:形仪与语音——真形锚点、候选定相、声音档案与克隆授权、ASR 接入
- 形仪域 domains/xingyi:form_anchors(稳定辨识特征/年龄表现/许可禁忌/可变范围——生成内容不可静默覆盖锚点);forms(主仪/化仪/文字形仪,candidate→adopt 定相治理:未定相不进呈现位、主仪唯一、新主仪自动退役旧主仪);voice_profiles(三来源,授权克隆必填用途与传播范围,revoke-grant 撤回立即生效,assert-usable 合成前置校验拒绝撤回/过期)
- ASR:元炁 SenseVoice /v1/audio/transcriptions 透传(Whisper 兼容 multipart);YUANQI_BASE_URL 未配置时 503 不降级不伪造——多模态关闭纪律
- 前端形仪工坊(人物详情页):真形锚点编辑(特征逐行解析)/候选登记定相废弃/声音档案建立与授权撤回;头像呈现位改走定相主仪(forms/primary);xingyi api 库 + mock + 组件 4 用例
- 退役:旧 avatar 模块(services/endpoints/avatar、avatar_dir 配置、avatar_url 字段、前端直传组件与测试)
- 验收(test_m7_xingyi 三用例):未定相不成为正式形仪(含主仪唯一与旧主仪 retired 断言);克隆授权可撤回(缺用途拒/撤回即拒/过期同理/预设不受约束);关闭全部多模态后文字相伴完整可用(ASR 503 时镜树缘契百工照常——回归)
- 四套件:后端 376、前端 203、BFF 4、神识 5 包全绿(2 个预存环境失败不变);迁移 008 净库+±1 干净
2026-08-27 20:26:52 +08:00
chenxin 7e23d6fa84 M6:缘契、人格与世界——契约版本链、铸魂分灵、河图境界与三界分流
- 缘契域 domains/yuanqi:covenants(当前版)+ covenant_events(不可变历史);提议—问心—铭契—撤回全链;边界变更立即生效;多维关系并存,服务层拒绝 affinity/好感度单一键;浪漫维度成人资格标记位
- 道体铸魂与分灵 domains/jingsheng/soul.py:soul_forgings 确认流(锚点变更 pending→confirmed 应用并签发新魂籍版本,确认前人物锚点不变);分灵创建新人物(继承资料与谱系 lineage 链、不继承原身份,魂籍从 v1 起)
- 河图域 domains/hetu:persistent_worlds 境界 + world_entities/world_relations 境界作用域重建 + 世谱挂接;境界正史视图(scope=persistent_world 归真集合);canonize 后置推进正史计数
- 三界分流:reality→息壤提取(不变);persistent_world→境界实录+正史推进;simulation→只记试演实录;factual_records 增 scope 列——境界/幻境实录与真界本相隔离
- 正影打通:引文 target_type=memory 时被引记忆 superseded → 返回 successor_memory_id 提示,不静默替换
- 前端:人物详情页缘契面板(提议维度选择/铭契/撤回/事件史)+ 世界页重建(境界目录/实体/正史概览)+ covenants/worlds api 库
- 退役:旧 personality/drive/evolution/world 模块、服务、端点、仓储、schema、前端五页与组件样式;迁移 007 退役九张旧表与 characters.current_personality_version_id
- 验收(test_m6_domains 六用例):多维无单一好感度/铭契版本链/事件不可抹除/撤回立即生效/边界即时/铸魂确认流+魂籍版本/分灵谱系/境界正史不进真界(实录 scope 隔离+真界候选=0)/引文正影后继
- 四套件:后端 379、前端 199、BFF 4、神识 5 包全绿(2 个预存环境失败不变);迁移 007 净库+±1 干净
2026-08-27 20:10:11 +08:00
chenxin 684261f4b4 M5:百工与司命——能力注册表、授契行契请命、禁域裁决与观察标证
- 百工域 domains/baigong:capabilities(统一能力契约:执行类型 builtin/qiankun/mcp/http + 四维风险标注 + 来源版本)、mandates(授契 grant / 行契 charter 预算时间窗,撤销立即生效)、tool_calls(净化摘要 + 观察标证);请命复用 pending_confirmations 加 expires_at/brief_json(超时=拒绝执行 HJ-1302,六要素说明)
- 司命裁决器 siming.py:四维取最高档(能力/数据/后果/情境);禁域为代码常量不可配置(递归删根/格式化/写裸设备/fork 炸弹、file:// 协议、密钥材料外发)——专项测试断言恶意标注与篡改对象仍拒 HJ-1301
- 执行链:禁域→裁决(HJ-1303 未授契拒且双留痕)→请命→执行(乾坤经 SandboxClient HMAC;MCP 经 call_tool;内置无副作用)→净化标证(观察非事实,敏感遮蔽+体量截断)→行动账
- MCP 登记/启停/刷新自动派生与撤下 capabilities(博闻登记接入点);内置能力种子(echo/datetime/qiankun.shell/web.fetch)
- 端点:/capabilities /mandates /petitions /tool-calls + /tools 兼容投影;前端百工页四区(能力目录四维标注/授契行契/请命批准拒绝/行动账观察标证)
- 退役:modules/safety|skills|tool_runtime|agent、application/tool_runtime、services/skills|safety、tool_policies/skills/model_endpoint_overrides 表(迁移 006)、前端 SkillManager/ToolsConsole/旧 tools 页
- 验收(test_baigong_siming 四用例):未授契被拒留痕、请命超时不执行、禁域不可关闭、结果为观察不入忆;后端 454、前端 220、BFF 4、神识 5 包全绿(2 个预存环境失败不变)
2026-08-27 19:17:21 +08:00
chenxin 144415247e M4 收口:神识用户面与计量闭环——BYOK/配额三段式/usage 落库/Anthropic+Gemini 适配器/神识阁
- 神识 Go:store 双实现(PG 自动建表+60s 就绪重试 / 内存回退)承载 usage_logs·byok_keys·quota_balances;BYOK 密钥库(AES-GCM 加密、API 脱敏、撤回即失效、主密钥未配置 fail-closed);配额三段式(预扣 max_tokens 钳位 1M→结算实际 token→全败幂等退款,HJ-2004);计量归属(IR user_id→usage_logs,/me/usage 用户维度聚合+配额总览,/admin/usage/sample 对账抽样);Anthropic/Gemini 出站适配器(SSE 解析、system 提升、用量映射);stub 端点补名修复计量空 endpoint
- IR 契约:user_id/byok_key_id 双端同步(Go types.go + Py types.py)
- 灵台:runs.owner_id(迁移 005,server_default 兼容存量)经 kernel 全链透传为 IR user_id;shenshi_client 扩展用户面方法(X-Hunjing-User-Id 代理,Echo 桩带内存实现);/me/byok-keys 登记·列表·撤回与 /me/usage 端点(4 用例)
- 前端神识阁(/models 重建):偏好(生成参数表单)/自带密钥(登记加密保存、脱敏列表、撤回二次确认)/用量(聚合表+配额进度条)三区;me API 库+mock+5 用例;侧栏恢复入口
- 配置与部署:.env.example 增 SHENSHI_DATABASE_URL/SHENSHI_BYOK_MASTER_KEY/SHENSHI_USER_BUDGET_TOKENS;compose 神识容器接自有库(init-shenshi.sql 预建 hunjing_shenshi,与灵台不共库共表)
- 证据:Go 34 用例(BYOK 独占/配额拒绝/结算退款/归属聚合)、后端 532、前端 210、BFF 4 全绿;跨进程 E2E 四场景(灵台 HttpShenshiClient→真 Go 神识:登记脱敏/BYOK 独占不可达整体失败/用量归属聚合/撤回即失效)通过;rewrite-plan M4 标记完成、能力矩阵神识行升已验证
2026-08-27 18:39:17 +08:00
chenxin 462d570751 M4b:ai_provider 删除手术①-⑥——神识全面接管模型出口
- ① 后端删除 app/ai_provider/(38 文件,含 anthropic/openai 格式、billing、audio、embedding)、services/ai_provider_admin.py、core/config/model_config.py、tests/ai_provider/(21 文件)
- ② 连带退役 application/agent/(旧 Agent 循环/steer/follow-up/确认注册表)、application/skill_executor.py、app/skills/(python skills 孤儿)、endpoints/audio.py、application/imports.py(ProviderManager 消费方);settings 端点收敛为生成偏好
- ③ runs.py 修剪:AgentRunRegistry/steer/follow-up/confirmations 端点退役,保留 list/get/events/cancel/stream;safety 确认改为直接落库裁决(pending→approved/denied)
- ④ 前端删除:models 页、ModelGatewayView、gateway.ts、GatewayPanel(玄关)、ModelGatewayPanel/ModelStrategyForm、audio.ts/useVoiceInput、网关类型与 mock;设置页收敛为外观/语言/通知
- ⑤ 配置收敛:config/models|providers|pricing.example.yaml 归档 shenshi/config/;.env.example 删 provider keys/MODEL_ENDPOINT_POOL/MODEL_CONFIG_PATH,收敛 SHENSHI_*
- ⑥ 验收:rg 审计灵台 Provider SDK import 与 ai_provider 引用归零;后端 528 passed(2 个预存失败经 HEAD 复跑验证与本手术无关)、前端 type-check+201 passed、神识 go vet+test、BFF 4 passed
2026-08-27 10:02:44 +08:00
chenxin 0d383725b5 更新相关的文档和配置 2026-08-27 09:37:35 +08:00
448 changed files with 25247 additions and 38452 deletions

No files matched your search

+15 -48
View File
@@ -88,7 +88,7 @@ LOG_ROTATION_SIZE=10 MB
# 本地单用户范围标识,当前阶段不实现多用户、组织、租户、OAuth 或 SSO # 本地单用户范围标识,当前阶段不实现多用户、组织、租户、OAuth 或 SSO
DEFAULT_OWNER_SCOPE=local-user DEFAULT_OWNER_SCOPE=local-user
# 默认逻辑模型名。必须指向 config/models.yaml 中真实远程 provider 的逻辑模型 # 默认逻辑模型名(提示值;实际路由由神识端点目录决定)
DEFAULT_MODEL_HINT=deepseek-v4-flash DEFAULT_MODEL_HINT=deepseek-v4-flash
# --- 管理端点和认证 --- # --- 管理端点和认证 ---
@@ -207,7 +207,7 @@ BIG_CYCLE_MEMORY_THRESHOLD=20
# 两次大周天最小间隔(秒),避免频繁触发 # 两次大周天最小间隔(秒),避免频繁触发
BIG_CYCLE_MIN_INTERVAL_SECONDS=300 BIG_CYCLE_MIN_INTERVAL_SECONDS=300
# 大周天候选生成所用模型(config/models.yaml 的逻辑模型名) # 大周天候选生成所用模型(逻辑模型名;路由由神识决定)
BIG_CYCLE_MODEL=deepseek-v4-flash BIG_CYCLE_MODEL=deepseek-v4-flash
# 单次大周天生成的演化候选上限 # 单次大周天生成的演化候选上限
@@ -238,7 +238,7 @@ MCP_STDIO_CALL_TIMEOUT_SECONDS=30
# MCP stdio SIGTERM → SIGKILL 等待秒数 # MCP stdio SIGTERM → SIGKILL 等待秒数
MCP_STDIO_STOP_TIMEOUT_SECONDS=5 MCP_STDIO_STOP_TIMEOUT_SECONDS=5
# Embedding 模型逻辑名。指向 config/models.yaml 中已注册的 embedding 模型 # Embedding 模型逻辑名(路由由神识决定)
EMBEDDING_MODEL_HINT=bge-m3 EMBEDDING_MODEL_HINT=bge-m3
# --- 感知与导入 --- # --- 感知与导入 ---
@@ -259,53 +259,12 @@ IMPORT_ALLOWED_AUDIO_MIME_TYPES=audio/wav,audio/mpeg,audio/mp4,audio/flac
# ===================================================================== # =====================================================================
# 模型网关 · 神识(LLM Provider API Keys) # 模型网关 · 神识(Go 网关统一接管,灵台不经手任何 Provider 密钥)
# ===================================================================== # =====================================================================
# 模型名称、provider 映射、能力标签、超时、限流、价格等,统一放在 config/models.yaml 中。 # 模型端点目录(含 base_url / api_key / 上游模型 / 优先级)统一经神识环境变量 SHENSHI_ENDPOINTS 配置,
# .env 只保存各 provider 的 URL 与 API key,避免模型策略散落在环境变量中。 # 详见下方「神识 · 模型网关」段;旧 config/models|providers|pricing.yaml 与灵台侧各 provider
# API Key 已随 ai_provider 退役(归档于 shenshi/config/ 仅供参考)。
# --- DeepSeek ---
DEEPSEEK_API_KEY=
DEEPSEEK_BASE_URL=https://api.deepseek.com
# --- 火山方舟 / 豆包 ---
VOLCENGINE_ARK_API_KEY=
VOLCENGINE_ARK_BASE_URL=https://ark.cn-beijing.volces.com/api/v3
# --- 小米 MiMo ---
XIAOMI_MIMO_API_KEY=
XIAOMI_MIMO_BASE_URL=https://token-plan-cn.xiaomimimo.com/v1
# --- OpenAI ---
OPENAI_API_KEY=
OPENAI_BASE_URL=https://api.openai.com/v1
# --- Anthropic ---
ANTHROPIC_API_KEY=
ANTHROPIC_BASE_URL=https://api.anthropic.com
# --- 元炁(本地推理引擎)---
# 作为 OpenAI 兼容 provider 接入,供 bge-m3 embedding 等模型调用。
YUANQI_BASE_URL=http://localhost:11434
YUANQI_API_KEY=
# --- 其他兼容网关预留 ---
CUSTOM_LLM_API_KEY=
CUSTOM_LLM_BASE_URL=
# --- 模型 endpoint 池路由(高级调优项,保持默认即可正常工作)---
# 物理 endpoint 池的 failover 与重试策略
MODEL_ENDPOINT_POOL_ENABLED=true
MODEL_ENDPOINT_POOL_DRY_RUN_SELECTION=false
MODEL_ENDPOINT_POOL_EXPOSE_DEBUG_HEADERS=false
MODEL_ENDPOINT_POOL_AGENT_FAILOVER_ENABLED=false
MODEL_ENDPOINT_POOL_PER_ENDPOINT_MAX_RETRIES=1
MODEL_ENDPOINT_POOL_RETRY_BASE_DELAY_SECONDS=0.25
MODEL_ENDPOINT_POOL_RETRY_MAX_DELAY_SECONDS=2.0
# 模型配置文件路径
# 默认读取项目根目录 config/models.yaml;不存在时 fallback 到 config/models.example.yaml
MODEL_CONFIG_PATH=../config/models.yaml
# ===================================================================== # =====================================================================
@@ -680,6 +639,14 @@ SHENSHI_SERVICE_TOKEN=
SHENSHI_TIMEOUT_SECONDS=120.0 SHENSHI_TIMEOUT_SECONDS=120.0
# 神识服务监听端口(shenshi 服务自身) # 神识服务监听端口(shenshi 服务自身)
SHENSHI_PORT=8080 SHENSHI_PORT=8080
# 神识自有库(usage_logs/byok_keys/quota_balances;未配置时进程内存储,重启即失)
# 本地开发可留空;正式环境指向 hunjing-postgres(如 postgresql://hunjing:hunjing@hunjing-postgres:5432/hunjing_shenshi)
SHENSHI_DATABASE_URL=
# BYOK 主密钥(32 字节 hex 或 base64;未配置时 BYOK 功能整体禁用 fail-closed)
# 生成示例:python3 -c "import secrets; print(secrets.token_hex(32))"
SHENSHI_BYOK_MASTER_KEY=
# 用户级软配额(token 口径;0 = 不限制,P0/P1 只做预算熔断不做计费)
SHENSHI_USER_BUDGET_TOKENS=0
# 神识出站端点目录(JSON 数组;未配置时回声桩兜底)。 # 神识出站端点目录(JSON 数组;未配置时回声桩兜底)。
# type: openai_compatible(云厂商兼容面与元炁 http://hunjing-yuanqi:11434 共用) # type: openai_compatible(云厂商兼容面与元炁 http://hunjing-yuanqi:11434 共用)
# 示例: # 示例:
+1 -1
View File
@@ -244,7 +244,7 @@ make test # 全部测试(backend / shenshi / bff / fronten
```bash ```bash
# 先准备配置(只需一次) # 先准备配置(只需一次)
cp .env.example .env # 填写 POSTGRES_PASSWORD 与各 *_API_KEY cp .env.example .env # 填写 POSTGRES_PASSWORD 与各 *_API_KEY
cp config/models.example.yaml config/models.yaml # 按需填写真实模型 endpoint # 模型端点目录经 SHENSHI_ENDPOINTS 配置(见 .env.example 神识段),无需复制 yaml
# 1) 启动基础设施(首次创建 hunjing-net 网络;Milvus 会自动带起 hunjing-etcd / hunjing-minio) # 1) 启动基础设施(首次创建 hunjing-net 网络;Milvus 会自动带起 hunjing-etcd / hunjing-minio)
docker compose -f docker/docker-compose.infra.yml up -d docker compose -f docker/docker-compose.infra.yml up -d
+2 -2
View File
@@ -67,10 +67,10 @@ dev-sandbox:
cd sandbox && cargo run cd sandbox && cargo run
dev-tianmu: dev-tianmu:
cd tianmu && uv run python -m src.main cd tianmu && uv run tianmu
dev-yuanqi: dev-yuanqi:
cd yuanqi && uv run python -m src.main cd yuanqi && uv run uvicorn src.main:app
# 核心三服务并行:灵台 + 神识 + 照壁(M0 验收链路) # 核心三服务并行:灵台 + 神识 + 照壁(M0 验收链路)
dev-core: dev-core:
+1 -1
View File
@@ -74,7 +74,7 @@ flowchart LR
E --> F[代码与测试] E --> F[代码与测试]
``` ```
现有代码是迁移与审计对象,不反向覆盖已确认的产品定义。当前处于 rewrite/1.0 全量重写期,架构与契约层已按五层栈重建,范围与节奏见[重写执行计划](docs/operations/rewrite-plan.md)。 现有代码是迁移与审计对象,不反向覆盖已确认的产品定义。当前处于 rewrite/1.0 全量重写期,架构与契约层已按五层栈重建,范围与节奏见重写执行计划(已收口删除)。
## 文档入口 ## 文档入口
@@ -0,0 +1,40 @@
"""运行归属用户列_runs_owner_id_用量计量透传神识
Revision ID: 005
Revises: 004
Create Date: 2026-08-27 10:31:10.225717
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '005'
down_revision: Union[str, Sequence[str], None] = '004'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_index(op.f('ix_factual_records_canonization_id'), 'factual_records', ['canonization_id'], unique=False)
op.create_index(op.f('ix_factual_records_character_id'), 'factual_records', ['character_id'], unique=False)
op.create_index(op.f('ix_memory_triggers_memory_id'), 'memory_triggers', ['memory_id'], unique=False)
op.add_column('runs', sa.Column('owner_id', sa.String(length=80), server_default='', nullable=False))
op.create_index(op.f('ix_runs_owner_id'), 'runs', ['owner_id'], unique=False)
# ### end Alembic commands ###
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index(op.f('ix_runs_owner_id'), table_name='runs')
op.drop_column('runs', 'owner_id')
op.drop_index(op.f('ix_memory_triggers_memory_id'), table_name='memory_triggers')
op.drop_index(op.f('ix_factual_records_character_id'), table_name='factual_records')
op.drop_index(op.f('ix_factual_records_canonization_id'), table_name='factual_records')
# ### end Alembic commands ###
@@ -0,0 +1,153 @@
"""百工域_能力注册表_授契行契_调用记录_请命超时改造_退役旧策略与端点覆盖表
Revision ID: 006
Revises: 005
Create Date: 2026-08-27 18:52:47.720303
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '006'
down_revision: Union[str, Sequence[str], None] = '005'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('capabilities',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('name', sa.String(length=120), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('description', sa.Text(), nullable=False),
sa.Column('executor_type', sa.String(length=20), nullable=False),
sa.Column('executor_ref', sa.String(length=200), nullable=False),
sa.Column('params_schema_json', sa.Text(), nullable=False),
sa.Column('risk_dims_json', sa.Text(), nullable=False),
sa.Column('data_scope', sa.Text(), nullable=False),
sa.Column('destinations', sa.Text(), nullable=False),
sa.Column('source', sa.String(length=20), nullable=False),
sa.Column('version', sa.String(length=40), nullable=False),
sa.Column('consent_mode', sa.String(length=10), nullable=False),
sa.Column('enabled', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_capabilities_owner_scope'), 'capabilities', ['owner_scope'], unique=False)
op.create_index('uq_capabilities_owner_name', 'capabilities', ['owner_scope', 'name'], unique=True)
op.create_table('mandates',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=True),
sa.Column('capability_id', sa.String(length=80), nullable=False),
sa.Column('kind', sa.String(length=10), nullable=False),
sa.Column('scope_json', sa.JSON(), nullable=False),
sa.Column('expires_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('budget_total', sa.Integer(), nullable=False),
sa.Column('budget_used', sa.Integer(), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('granted_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['capability_id'], ['capabilities.id'], ),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_mandates_capability_id'), 'mandates', ['capability_id'], unique=False)
op.create_index(op.f('ix_mandates_character_id'), 'mandates', ['character_id'], unique=False)
op.create_index(op.f('ix_mandates_owner_scope'), 'mandates', ['owner_scope'], unique=False)
op.create_index(op.f('ix_mandates_status'), 'mandates', ['status'], unique=False)
op.create_table('tool_calls',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('run_id', sa.String(length=80), nullable=True),
sa.Column('character_id', sa.String(length=80), nullable=True),
sa.Column('capability_id', sa.String(length=80), nullable=False),
sa.Column('trace_id', sa.String(length=80), nullable=True),
sa.Column('decision', sa.String(length=40), nullable=False),
sa.Column('mandate_id', sa.String(length=80), nullable=True),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('error_code', sa.String(length=40), nullable=True),
sa.Column('input_digest', sa.Text(), nullable=False),
sa.Column('output_digest', sa.Text(), nullable=False),
sa.Column('evidence_json', sa.JSON(), nullable=False),
sa.Column('duration_ms', sa.Integer(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.ForeignKeyConstraint(['run_id'], ['runs.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_tool_calls_capability_id'), 'tool_calls', ['capability_id'], unique=False)
op.create_index(op.f('ix_tool_calls_character_id'), 'tool_calls', ['character_id'], unique=False)
op.create_index(op.f('ix_tool_calls_created_at'), 'tool_calls', ['created_at'], unique=False)
op.create_index(op.f('ix_tool_calls_run_id'), 'tool_calls', ['run_id'], unique=False)
op.create_index(op.f('ix_tool_calls_trace_id'), 'tool_calls', ['trace_id'], unique=False)
op.drop_index(op.f('ix_skills_status'), table_name='skills')
op.drop_table('skills')
op.drop_table('tool_policies')
op.drop_table('model_endpoint_overrides')
# ### end Alembic commands ###
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('model_endpoint_overrides',
sa.Column('logical_model_name', sa.VARCHAR(length=120), nullable=False),
sa.Column('endpoint_name', sa.VARCHAR(length=120), nullable=False),
sa.Column('base_url', sa.VARCHAR(length=500), nullable=True),
sa.Column('api_key', sa.VARCHAR(length=500), nullable=True),
sa.Column('status', sa.VARCHAR(length=40), nullable=True),
sa.Column('weight', sa.INTEGER(), nullable=True),
sa.Column('priority', sa.INTEGER(), nullable=True),
sa.Column('created_at', sa.DATETIME(), nullable=False),
sa.Column('updated_at', sa.DATETIME(), nullable=False),
sa.PrimaryKeyConstraint('logical_model_name', 'endpoint_name')
)
op.create_table('tool_policies',
sa.Column('name', sa.VARCHAR(length=80), nullable=False),
sa.Column('description', sa.TEXT(), nullable=False),
sa.Column('risk_level', sa.VARCHAR(length=20), nullable=False),
sa.Column('permission_policy', sa.VARCHAR(length=20), nullable=False),
sa.Column('side_effect', sa.BOOLEAN(), nullable=False),
sa.Column('audit_required', sa.BOOLEAN(), nullable=False),
sa.Column('source', sa.VARCHAR(length=20), nullable=False),
sa.Column('enabled', sa.BOOLEAN(), nullable=False),
sa.Column('created_at', sa.DATETIME(), nullable=False),
sa.Column('updated_at', sa.DATETIME(), nullable=False),
sa.PrimaryKeyConstraint('name')
)
op.create_table('skills',
sa.Column('id', sa.VARCHAR(length=80), nullable=False),
sa.Column('name', sa.VARCHAR(length=120), nullable=False),
sa.Column('description', sa.TEXT(), nullable=False),
sa.Column('trigger', sa.TEXT(), nullable=False),
sa.Column('prompt_fragment', sa.TEXT(), nullable=False),
sa.Column('tool_bindings_json', sa.TEXT(), nullable=False),
sa.Column('risk_level', sa.VARCHAR(length=8), nullable=False),
sa.Column('status', sa.VARCHAR(length=40), nullable=False),
sa.Column('created_at', sa.DATETIME(), nullable=False),
sa.Column('updated_at', sa.DATETIME(), nullable=False),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('name')
)
op.create_index(op.f('ix_skills_status'), 'skills', ['status'], unique=False)
op.drop_index(op.f('ix_tool_calls_trace_id'), table_name='tool_calls')
op.drop_index(op.f('ix_tool_calls_run_id'), table_name='tool_calls')
op.drop_index(op.f('ix_tool_calls_created_at'), table_name='tool_calls')
op.drop_index(op.f('ix_tool_calls_character_id'), table_name='tool_calls')
op.drop_index(op.f('ix_tool_calls_capability_id'), table_name='tool_calls')
op.drop_table('tool_calls')
op.drop_index(op.f('ix_mandates_status'), table_name='mandates')
op.drop_index(op.f('ix_mandates_owner_scope'), table_name='mandates')
op.drop_index(op.f('ix_mandates_character_id'), table_name='mandates')
op.drop_index(op.f('ix_mandates_capability_id'), table_name='mandates')
op.drop_table('mandates')
op.drop_index('uq_capabilities_owner_name', table_name='capabilities')
op.drop_index(op.f('ix_capabilities_owner_scope'), table_name='capabilities')
op.drop_table('capabilities')
# ### end Alembic commands ###
@@ -0,0 +1,170 @@
"""缘契域_契约与事件_铸魂分灵_河图境界与实体_实录作用域_退役旧人格驱动演化世界表
Revision ID: 007
Revises: 006
Create Date: 2026-08-27 19:32:08.478484
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '007'
down_revision: Union[str, Sequence[str], None] = '006'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# 旧世界表退役重建(世界权威移交河图域 hetu;开发数据丢弃)
op.drop_table('world_entities')
op.drop_table('character_world_entities')
op.drop_table('world_relations')
# 缘契域
op.create_table('covenants',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('dimensions_json', sa.Text(), nullable=False),
sa.Column('primary_title', sa.String(length=120), nullable=True),
sa.Column('boundaries_json', sa.Text(), nullable=False),
sa.Column('adult_consent', sa.Boolean(), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('version', sa.Integer(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_covenants_character_id'), 'covenants', ['character_id'], unique=False)
op.create_index(op.f('ix_covenants_owner_scope'), 'covenants', ['owner_scope'], unique=False)
op.create_index(op.f('ix_covenants_status'), 'covenants', ['status'], unique=False)
op.create_table('covenant_events',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('covenant_id', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=False),
sa.Column('type', sa.String(length=20), nullable=False),
sa.Column('payload_json', sa.Text(), nullable=False),
sa.Column('actor', sa.String(length=20), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['covenant_id'], ['covenants.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_covenant_events_character_id'), 'covenant_events', ['character_id'], unique=False)
op.create_index(op.f('ix_covenant_events_covenant_id'), 'covenant_events', ['covenant_id'], unique=False)
op.create_index(op.f('ix_covenant_events_created_at'), 'covenant_events', ['created_at'], unique=False)
op.create_index(op.f('ix_covenant_events_type'), 'covenant_events', ['type'], unique=False)
# 铸魂
op.create_table('soul_forgings',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=False),
sa.Column('proposed_anchor_json', sa.Text(), nullable=False),
sa.Column('reason', sa.Text(), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('confirmed_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_soul_forgings_character_id'), 'soul_forgings', ['character_id'], unique=False)
op.create_index(op.f('ix_soul_forgings_status'), 'soul_forgings', ['status'], unique=False)
# 河图境界
op.create_table('persistent_worlds',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('name', sa.String(length=120), nullable=False),
sa.Column('description', sa.Text(), nullable=False),
sa.Column('rules_json', sa.Text(), nullable=False),
sa.Column('lorebook_id', sa.String(length=80), nullable=True),
sa.Column('canon_count', sa.Integer(), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_persistent_worlds_lorebook_id'), 'persistent_worlds', ['lorebook_id'], unique=False)
op.create_index(op.f('ix_persistent_worlds_owner_scope'), 'persistent_worlds', ['owner_scope'], unique=False)
op.create_table('world_entities',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('world_id', sa.String(length=80), nullable=False),
sa.Column('kind', sa.String(length=30), nullable=False),
sa.Column('name', sa.String(length=120), nullable=False),
sa.Column('description', sa.Text(), nullable=False),
sa.Column('attrs_json', sa.Text(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['world_id'], ['persistent_worlds.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_world_entities_world_id'), 'world_entities', ['world_id'], unique=False)
op.create_table('world_relations',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('world_id', sa.String(length=80), nullable=False),
sa.Column('source_entity_id', sa.String(length=80), nullable=False),
sa.Column('target_entity_id', sa.String(length=80), nullable=False),
sa.Column('relation', sa.String(length=80), nullable=False),
sa.Column('note', sa.Text(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['source_entity_id'], ['world_entities.id'], ),
sa.ForeignKeyConstraint(['target_entity_id'], ['world_entities.id'], ),
sa.ForeignKeyConstraint(['world_id'], ['persistent_worlds.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_world_relations_source_entity_id'), 'world_relations', ['source_entity_id'], unique=False)
op.create_index(op.f('ix_world_relations_target_entity_id'), 'world_relations', ['target_entity_id'], unique=False)
op.create_index(op.f('ix_world_relations_world_id'), 'world_relations', ['world_id'], unique=False)
# 实录作用域
with op.batch_alter_table('factual_records') as batch:
batch.add_column(sa.Column('scope', sa.String(length=40), nullable=False, server_default='reality'))
batch.create_index('ix_factual_records_scope', ['scope'])
# 旧人格/驱动/演化域退役(人格基线=道体锚点 identity_baseline_json + 魂籍版本链)
with op.batch_alter_table('characters') as batch:
batch.drop_column('current_personality_version_id')
op.drop_table('personality_versions')
op.drop_table('personality_proposals')
op.drop_table('curiosity_questions')
op.drop_table('exploratory_subgoals')
op.drop_table('evolution_candidates')
def downgrade() -> None:
"""Downgrade schema."""
op.drop_table('evolution_candidates')
op.drop_table('exploratory_subgoals')
op.drop_table('curiosity_questions')
op.drop_table('personality_proposals')
op.drop_table('personality_versions')
with op.batch_alter_table('characters') as batch:
batch.add_column(sa.Column('current_personality_version_id', sa.String(length=80), nullable=True))
with op.batch_alter_table('factual_records') as batch:
batch.drop_index('ix_factual_records_scope')
batch.drop_column('scope')
op.drop_index(op.f('ix_world_relations_world_id'), table_name='world_relations')
op.drop_index(op.f('ix_world_relations_target_entity_id'), table_name='world_relations')
op.drop_index(op.f('ix_world_relations_source_entity_id'), table_name='world_relations')
op.drop_table('world_relations')
op.drop_index(op.f('ix_world_entities_world_id'), table_name='world_entities')
op.drop_table('world_entities')
op.drop_index(op.f('ix_persistent_worlds_owner_scope'), table_name='persistent_worlds')
op.drop_index(op.f('ix_persistent_worlds_lorebook_id'), table_name='persistent_worlds')
op.drop_table('persistent_worlds')
op.drop_index(op.f('ix_soul_forgings_status'), table_name='soul_forgings')
op.drop_index(op.f('ix_soul_forgings_character_id'), table_name='soul_forgings')
op.drop_table('soul_forgings')
op.drop_index(op.f('ix_covenant_events_type'), table_name='covenant_events')
op.drop_index(op.f('ix_covenant_events_created_at'), table_name='covenant_events')
op.drop_index(op.f('ix_covenant_events_covenant_id'), table_name='covenant_events')
op.drop_index(op.f('ix_covenant_events_character_id'), table_name='covenant_events')
op.drop_table('covenant_events')
op.drop_index(op.f('ix_covenants_status'), table_name='covenants')
op.drop_index(op.f('ix_covenants_owner_scope'), table_name='covenants')
op.drop_index(op.f('ix_covenants_character_id'), table_name='covenants')
op.drop_table('covenants')
@@ -0,0 +1,90 @@
"""形仪域_真形锚点_主仪化仪定相_声音档案与克隆授权
Revision ID: 008
Revises: 007
Create Date: 2026-08-27 20:15:34.665285
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '008'
down_revision: Union[str, Sequence[str], None] = '007'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('form_anchors',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=False),
sa.Column('features_json', sa.Text(), nullable=False),
sa.Column('age_presentation', sa.String(length=40), nullable=False),
sa.Column('permissions_json', sa.Text(), nullable=False),
sa.Column('mutable_scope_json', sa.Text(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_form_anchors_character_id'), 'form_anchors', ['character_id'], unique=False)
op.create_table('forms',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=False),
sa.Column('kind', sa.String(length=20), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('source', sa.String(length=20), nullable=False),
sa.Column('description', sa.Text(), nullable=False),
sa.Column('media_ref', sa.String(length=500), nullable=True),
sa.Column('live2d_json', sa.Text(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('adopted_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_forms_character_id'), 'forms', ['character_id'], unique=False)
op.create_index(op.f('ix_forms_kind'), 'forms', ['kind'], unique=False)
op.create_index(op.f('ix_forms_status'), 'forms', ['status'], unique=False)
op.create_table('voice_profiles',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=False),
sa.Column('source', sa.String(length=30), nullable=False),
sa.Column('name', sa.String(length=120), nullable=False),
sa.Column('params_json', sa.Text(), nullable=False),
sa.Column('grant_purpose', sa.String(length=200), nullable=True),
sa.Column('grant_expires_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('grant_distribution', sa.String(length=200), nullable=True),
sa.Column('grant_revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('is_active', sa.Boolean(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_voice_profiles_character_id'), 'voice_profiles', ['character_id'], unique=False)
op.create_index(op.f('ix_voice_profiles_is_active'), 'voice_profiles', ['is_active'], unique=False)
op.add_column('pending_confirmations', sa.Column('expires_at', sa.DateTime(timezone=True), nullable=True))
op.add_column('pending_confirmations', sa.Column('brief_json', sa.Text(), server_default='{}', nullable=False))
# ### end Alembic commands ###
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.drop_column('pending_confirmations', 'brief_json')
op.drop_column('pending_confirmations', 'expires_at')
op.drop_index(op.f('ix_voice_profiles_is_active'), table_name='voice_profiles')
op.drop_index(op.f('ix_voice_profiles_character_id'), table_name='voice_profiles')
op.drop_table('voice_profiles')
op.drop_index(op.f('ix_forms_status'), table_name='forms')
op.drop_index(op.f('ix_forms_kind'), table_name='forms')
op.drop_index(op.f('ix_forms_character_id'), table_name='forms')
op.drop_table('forms')
op.drop_index(op.f('ix_form_anchors_character_id'), table_name='form_anchors')
op.drop_table('form_anchors')
# ### end Alembic commands ###
@@ -0,0 +1,87 @@
"""天工阁_受托任务_叩心候选与约束_退役旧目标计划感知大周天模块
Revision ID: 009
Revises: 008
Create Date: 2026-08-27 20:31:23.297111
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '009'
down_revision: Union[str, Sequence[str], None] = '008'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('backstage_tasks',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=True),
sa.Column('parent_task_id', sa.String(length=80), nullable=True),
sa.Column('title', sa.String(length=200), nullable=False),
sa.Column('brief', sa.Text(), nullable=False),
sa.Column('acceptance_criteria', sa.Text(), nullable=False),
sa.Column('status', sa.String(length=30), nullable=False),
sa.Column('result_digest', sa.Text(), nullable=False),
sa.Column('evidence_json', sa.Text(), nullable=False),
sa.Column('run_id', sa.String(length=80), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('completed_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.ForeignKeyConstraint(['parent_task_id'], ['backstage_tasks.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_backstage_tasks_character_id'), 'backstage_tasks', ['character_id'], unique=False)
op.create_index(op.f('ix_backstage_tasks_owner_scope'), 'backstage_tasks', ['owner_scope'], unique=False)
op.create_index(op.f('ix_backstage_tasks_parent_task_id'), 'backstage_tasks', ['parent_task_id'], unique=False)
op.create_index(op.f('ix_backstage_tasks_run_id'), 'backstage_tasks', ['run_id'], unique=False)
op.create_index(op.f('ix_backstage_tasks_status'), 'backstage_tasks', ['status'], unique=False)
op.create_table('knocking_candidates',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('character_id', sa.String(length=80), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('trigger', sa.String(length=20), nullable=False),
sa.Column('draft', sa.Text(), nullable=False),
sa.Column('channel', sa.String(length=20), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('ruling_json', sa.Text(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('dispatched_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_knocking_candidates_character_id'), 'knocking_candidates', ['character_id'], unique=False)
op.create_index(op.f('ix_knocking_candidates_owner_scope'), 'knocking_candidates', ['owner_scope'], unique=False)
op.create_index(op.f('ix_knocking_candidates_status'), 'knocking_candidates', ['status'], unique=False)
# ### end Alembic commands ###
# 旧域退役:太一(goals)/洛书(plans)/天工阁旧 roster/感知/大周天
op.drop_table('goals')
op.drop_table('plans')
op.drop_table('agent_roster')
op.drop_table('perception_channels')
op.drop_table('big_cycles')
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index(op.f('ix_knocking_candidates_status'), table_name='knocking_candidates')
op.drop_index(op.f('ix_knocking_candidates_owner_scope'), table_name='knocking_candidates')
op.drop_index(op.f('ix_knocking_candidates_character_id'), table_name='knocking_candidates')
op.drop_table('knocking_candidates')
op.drop_index(op.f('ix_backstage_tasks_status'), table_name='backstage_tasks')
op.drop_index(op.f('ix_backstage_tasks_run_id'), table_name='backstage_tasks')
op.drop_index(op.f('ix_backstage_tasks_parent_task_id'), table_name='backstage_tasks')
op.drop_index(op.f('ix_backstage_tasks_owner_scope'), table_name='backstage_tasks')
op.drop_index(op.f('ix_backstage_tasks_character_id'), table_name='backstage_tasks')
op.drop_table('backstage_tasks')
# ### end Alembic commands ###
@@ -0,0 +1,121 @@
"""器谱_设备配对与逐项授契
Revision ID: 010
Revises: 009
Create Date: 2026-08-28 09:25:57.705846
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '010'
down_revision: Union[str, Sequence[str], None] = '009'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('device_grants',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('device_id', sa.String(length=80), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('capability', sa.String(length=60), nullable=False),
sa.Column('scope_json', sa.Text(), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('granted_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('revoked_at', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_device_grants_capability'), 'device_grants', ['capability'], unique=False)
op.create_index(op.f('ix_device_grants_device_id'), 'device_grants', ['device_id'], unique=False)
op.create_index(op.f('ix_device_grants_owner_scope'), 'device_grants', ['owner_scope'], unique=False)
op.create_index(op.f('ix_device_grants_status'), 'device_grants', ['status'], unique=False)
op.create_table('devices',
sa.Column('id', sa.String(length=80), nullable=False),
sa.Column('owner_scope', sa.String(length=80), nullable=False),
sa.Column('kind', sa.String(length=30), nullable=False),
sa.Column('name', sa.String(length=120), nullable=False),
sa.Column('key_fingerprint', sa.String(length=128), nullable=False),
sa.Column('status', sa.String(length=20), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('last_seen_at', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_devices_owner_scope'), 'devices', ['owner_scope'], unique=False)
op.create_index(op.f('ix_devices_status'), 'devices', ['status'], unique=False)
op.drop_index(op.f('ix_character_profile_candidates_character_id'), table_name='character_profile_candidates')
op.drop_index(op.f('ix_character_profile_candidates_import_job_id'), table_name='character_profile_candidates')
op.drop_index(op.f('ix_character_profile_candidates_status'), table_name='character_profile_candidates')
op.drop_table('character_profile_candidates')
op.drop_index(op.f('ix_memory_candidates_character_id'), table_name='memory_candidates')
op.drop_index(op.f('ix_memory_candidates_cycle_id'), table_name='memory_candidates')
op.drop_index(op.f('ix_memory_candidates_import_job_id'), table_name='memory_candidates')
op.drop_index(op.f('ix_memory_candidates_status'), table_name='memory_candidates')
op.drop_table('memory_candidates')
# ### end Alembic commands ###
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('memory_candidates',
sa.Column('id', sa.VARCHAR(length=80), nullable=False),
sa.Column('character_id', sa.VARCHAR(length=80), nullable=False),
sa.Column('import_job_id', sa.VARCHAR(length=80), nullable=True),
sa.Column('cycle_id', sa.VARCHAR(length=80), nullable=True),
sa.Column('type', sa.VARCHAR(length=60), nullable=False),
sa.Column('title', sa.VARCHAR(length=200), nullable=False),
sa.Column('content', sa.TEXT(), nullable=False),
sa.Column('confidence', sa.FLOAT(), nullable=False),
sa.Column('importance', sa.FLOAT(), nullable=False),
sa.Column('sensitivity', sa.VARCHAR(length=40), nullable=False),
sa.Column('evidence_ids_json', sa.TEXT(), nullable=False),
sa.Column('status', sa.VARCHAR(length=40), nullable=False),
sa.Column('reason', sa.TEXT(), nullable=True),
sa.Column('reviewed_at', sa.DATETIME(), nullable=True),
sa.Column('created_at', sa.DATETIME(), nullable=False),
sa.Column('updated_at', sa.DATETIME(), nullable=False),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.ForeignKeyConstraint(['cycle_id'], ['big_cycles.id'], ),
sa.ForeignKeyConstraint(['import_job_id'], ['character_import_jobs.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_memory_candidates_status'), 'memory_candidates', ['status'], unique=False)
op.create_index(op.f('ix_memory_candidates_import_job_id'), 'memory_candidates', ['import_job_id'], unique=False)
op.create_index(op.f('ix_memory_candidates_cycle_id'), 'memory_candidates', ['cycle_id'], unique=False)
op.create_index(op.f('ix_memory_candidates_character_id'), 'memory_candidates', ['character_id'], unique=False)
op.create_table('character_profile_candidates',
sa.Column('id', sa.VARCHAR(length=80), nullable=False),
sa.Column('character_id', sa.VARCHAR(length=80), nullable=False),
sa.Column('import_job_id', sa.VARCHAR(length=80), nullable=True),
sa.Column('field', sa.VARCHAR(length=60), nullable=False),
sa.Column('value', sa.TEXT(), nullable=False),
sa.Column('confidence', sa.FLOAT(), nullable=False),
sa.Column('evidence_ids_json', sa.TEXT(), nullable=False),
sa.Column('status', sa.VARCHAR(length=40), nullable=False),
sa.Column('reason', sa.TEXT(), nullable=True),
sa.Column('reviewed_at', sa.DATETIME(), nullable=True),
sa.Column('created_at', sa.DATETIME(), nullable=False),
sa.Column('updated_at', sa.DATETIME(), nullable=False),
sa.ForeignKeyConstraint(['character_id'], ['characters.id'], ),
sa.ForeignKeyConstraint(['import_job_id'], ['character_import_jobs.id'], ),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_character_profile_candidates_status'), 'character_profile_candidates', ['status'], unique=False)
op.create_index(op.f('ix_character_profile_candidates_import_job_id'), 'character_profile_candidates', ['import_job_id'], unique=False)
op.create_index(op.f('ix_character_profile_candidates_character_id'), 'character_profile_candidates', ['character_id'], unique=False)
op.drop_index(op.f('ix_devices_status'), table_name='devices')
op.drop_index(op.f('ix_devices_owner_scope'), table_name='devices')
op.drop_table('devices')
op.drop_index(op.f('ix_device_grants_status'), table_name='device_grants')
op.drop_index(op.f('ix_device_grants_owner_scope'), table_name='device_grants')
op.drop_index(op.f('ix_device_grants_device_id'), table_name='device_grants')
op.drop_index(op.f('ix_device_grants_capability'), table_name='device_grants')
op.drop_table('device_grants')
# ### end Alembic commands ###
-63
View File
@@ -1,63 +0,0 @@
from app.ai_provider.config.loader import load_models_config, load_providers_config
from app.ai_provider.config.models import ModelConfig, ProviderConfig
# 嵌入 / 重排序
from app.ai_provider.embedding import (
EmbeddingProtocol,
EmbeddingRequest,
EmbeddingResponse,
EmbeddingResult,
EmbeddingService,
EmbeddingServiceError,
)
from app.ai_provider.manager import ProviderManager, ProviderManagerFactory
from app.ai_provider.reranker import (
RerankerProtocol,
RerankerService,
RerankerServiceError,
RerankRequest,
RerankResponse,
RerankResult,
)
from app.ai_provider.types.chat import ChatChunk, ChatResult, TokenUsage
from app.ai_provider.types.errors import AIProviderError
from app.ai_provider.types.messages import (
AIMessage,
BaseMessage,
HumanMessage,
SystemMessage,
ToolMessage,
)
__all__ = [
# LLM
"ProviderManager",
"ProviderManagerFactory",
"BaseMessage",
"SystemMessage",
"HumanMessage",
"AIMessage",
"ToolMessage",
"ChatChunk",
"ChatResult",
"TokenUsage",
"AIProviderError",
"ModelConfig",
"ProviderConfig",
"load_models_config",
"load_providers_config",
# Embedding
"EmbeddingProtocol",
"EmbeddingRequest",
"EmbeddingResponse",
"EmbeddingResult",
"EmbeddingService",
"EmbeddingServiceError",
# Reranker
"RerankerProtocol",
"RerankRequest",
"RerankResponse",
"RerankResult",
"RerankerService",
"RerankerServiceError",
]
-197
View File
@@ -1,197 +0,0 @@
"""音频服务 — ASR / TTS 适配器外观。
用法::
from app.ai_provider.audio import AudioService
service = AudioService(factory)
result = await service.transcribe(audio_bytes, format="wav")
result = await service.synthesize("你好,魂镜", voice="default")
``AudioService`` 通过注册表中带有 ``asr`` / ``tts`` 能力标签的模型来解析
使用哪个模型,然后将创建委托给 ``ModelFactory.create_audio()``。
"""
from __future__ import annotations
from app.ai_provider.audio.base import (
ASRProtocol,
ASRRequest,
ASRResponse,
AudioServiceError,
TTSProtocol,
TTSRequest,
TTSResponse,
)
class AudioService:
"""外观类,负责解析 ASR / TTS 模型并委托给 ProviderManager。
接受 ProviderManager 或实现了以下接口的适配器:
- ``.registry`` → 具有 ``list_capability_models(capability: str)`` 方法
- ``.create_audio(name: str)`` → 返回 ASRProtocol / TTSProtocol 实例
"""
def __init__(self, manager) -> None:
self._manager = manager
self._asr_adapter: ASRProtocol | None = None
self._tts_adapter: TTSProtocol | None = None
# ── Registry 兼容适配 ─────────────────────────────────────────
@property
def _registry(self):
"""按需获取 registry,兼容 adapter 和裸 ProviderManager/ModelRegistry。"""
mgr = self._manager
if hasattr(mgr, "registry"):
return mgr.registry
if hasattr(mgr, "model_registry"):
return mgr.model_registry
return mgr
def _list_capability_models(self, tag: str):
reg = self._registry
# 适配器接口(list_capability_models)vs ModelRegistry 接口(list_by_capability)
if hasattr(reg, "list_capability_models"):
return reg.list_capability_models(tag)
if hasattr(reg, "list_by_capability"):
return reg.list_by_capability(tag)
return []
@property
def _factory(self):
mgr = self._manager
if hasattr(mgr, "create_audio"):
return mgr
return None
# ── ASR ──────────────────────────────────────────────────────────────
@property
def asr(self) -> ASRProtocol:
"""解析并缓存 ASR 适配器(取注册表中第一个活跃 ASR 模型)。"""
if self._asr_adapter is None:
self._asr_adapter = self._resolve_asr()
return self._asr_adapter
def asr_for(self, model_name: str) -> ASRProtocol:
"""解析并缓存指定模型名的 ASR 适配器(覆盖默认第一个)。"""
self._asr_adapter = self._factory.create_audio(model_name)
return self._asr_adapter
async def transcribe(
self,
audio_data: bytes,
format: str = "wav",
language: str | None = None,
model_name: str | None = None,
) -> ASRResponse:
"""快捷方法:构造请求并调用 ASR 适配器。
Args:
model_name: 可选,指定使用哪个 ASR 模型(按注册表逻辑名)。
省略时取注册表中第一个活跃的 ASR 模型。
"""
if model_name is not None:
adapter = self.asr_for(model_name)
else:
adapter = self.asr
request = ASRRequest(
audio_data=audio_data,
format=format,
language=language,
)
return await adapter.transcribe(request)
# ── TTS ──────────────────────────────────────────────────────────────
@property
def tts(self) -> TTSProtocol:
"""解析并缓存 TTS 适配器。"""
if self._tts_adapter is None:
self._tts_adapter = self._resolve_tts()
return self._tts_adapter
async def synthesize(
self,
text: str,
voice: str = "default",
response_format: str = "wav",
extra: dict | None = None,
) -> TTSResponse:
"""快捷方法:构造请求并调用 TTS 适配器。"""
request = TTSRequest(
text=text,
voice=voice,
response_format=response_format,
extra=extra,
)
return await self.tts.synthesize(request)
# ── 解析 ───────────────────────────────────────────────────────
def _resolve_asr(self) -> ASRProtocol:
"""查找第一个活跃的 ASR 模型并通过工厂创建。"""
models = self._list_capability_models("asr")
if not models:
raise AudioServiceError(
status_code=503,
code="audio_asr_no_model",
message="No ASR model configured in the model registry.",
)
factory = self._factory
if factory is None:
raise AudioServiceError(
status_code=501,
code="audio_asr_no_factory",
message="Audio adapter factory not available (provider manager missing create_audio).",
)
return factory.create_audio(models[0].name)
def _resolve_tts(self) -> TTSProtocol:
"""查找第一个活跃的 TTS 模型并通过工厂创建。"""
models = self._list_capability_models("tts")
if not models:
raise AudioServiceError(
status_code=503,
code="audio_tts_no_model",
message="No TTS model configured in the model registry.",
)
factory = self._factory
if factory is None:
raise AudioServiceError(
status_code=501,
code="audio_tts_no_factory",
message="Audio adapter factory not available (provider manager missing create_audio).",
)
return factory.create_audio(models[0].name)
# ── 生命周期 ────────────────────────────────────────────────────────
async def close(self) -> None:
"""释放适配器持有的 HTTP 连接。"""
for adapter in (self._asr_adapter, self._tts_adapter):
if adapter is None:
continue
close = getattr(adapter, "close", None)
if close is not None:
await close()
async def __aenter__(self) -> AudioService:
return self
async def __aexit__(self, *exc_info: object) -> None:
await self.close()
__all__ = [
"ASRProtocol",
"ASRRequest",
"ASRResponse",
"AudioService",
"AudioServiceError",
"TTSProtocol",
"TTSRequest",
"TTSResponse",
]
-263
View File
@@ -1,263 +0,0 @@
"""MiMo ASR adapter — ``POST /v1/chat/completions`` with ``input_audio``.
MiMo 的 ASR 不走独立音频端点,而是通过聊天补全接口扩展实现:
POST {base_url}/chat/completions
Content-Type: application/json
Authorization: Bearer {api_key}
{
"model": "mimo-v2.5-asr",
"messages": [
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
"data": "data:audio/wav;base64,<base64_encoded_audio>"
}
}
]
}
],
"asr_options": {
"language": "auto"
}
}
响应::
{
"choices": [{
"message": {
"content": "转写后的文本"
}
}]
}
"""
from __future__ import annotations
import base64
import hashlib
import time
import httpx
from app.ai_provider.audio.base import (
ASRProtocol,
ASRRequest,
ASRResponse,
AudioAdapterBase,
AudioServiceError,
)
from app.core.logging import logger
_MIME_MAP = {
"wav": "audio/wav",
"mp3": "audio/mpeg",
"m4a": "audio/mp4",
"flac": "audio/flac",
}
# ASR 缓存:默认最多 128 条,TTL 1 小时
_ASR_CACHE_MAX_SIZE = 128
_ASR_CACHE_TTL_SECONDS = 3600
class _ASRCacheEntry:
__slots__ = ("text", "stored_at")
def __init__(self, text: str) -> None:
self.text = text
self.stored_at = time.monotonic()
_asr_cache: dict[str, _ASRCacheEntry] = {}
def _asr_cache_key(model: str, audio_data: bytes, language: str, fmt: str) -> str:
"""生成 ASR 缓存键:SHA256(model + audio_bytes + language + format)。"""
raw = f"{model}\0{language}\0{fmt}\0"
raw_bytes = raw.encode() + audio_data
return hashlib.sha256(raw_bytes).hexdigest()
def _asr_cache_get(key: str) -> str | None:
entry = _asr_cache.get(key)
if entry is None:
return None
if time.monotonic() - entry.stored_at > _ASR_CACHE_TTL_SECONDS:
del _asr_cache[key]
return None
return entry.text
def _asr_cache_set(key: str, text: str) -> None:
if len(_asr_cache) >= _ASR_CACHE_MAX_SIZE:
sorted_keys = sorted(_asr_cache.keys(), key=lambda k: _asr_cache[k].stored_at)
for old_key in sorted_keys[: _ASR_CACHE_MAX_SIZE // 2]:
del _asr_cache[old_key]
_asr_cache[key] = _ASRCacheEntry(text)
class HunjingASROpenAI(AudioAdapterBase, ASRProtocol):
"""MiMo ASR 适配器,通过 ``POST /chat/completions`` 携带 ``input_audio`` 实现。
将音频字节包装为 chat 消息中的 ``input_audio`` 内容类型的 data URL,
然后从响应中提取转写文本。
"""
_adapter_type = "asr"
_default_timeout = 300
def __init__(
self,
*,
base_url: str,
api_key: str,
model: str = "mimo-v2.5-asr",
request_timeout_seconds: int = 300,
) -> None:
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._model = model
self._timeout = request_timeout_seconds
self._client: httpx.AsyncClient | None = None
# ── 客户端 ───────────────────────────────────────────────────────────
@property
def _http(self) -> httpx.AsyncClient:
if self._client is None or self._client.is_closed:
self._client = httpx.AsyncClient(timeout=httpx.Timeout(self._timeout))
return self._client
async def close(self) -> None:
if self._client is not None and not self._client.is_closed:
await self._client.aclose()
# ── ASRProtocol ──────────────────────────────────────────────────────
async def transcribe(self, request: ASRRequest) -> ASRResponse:
"""通过 ``POST /chat/completions`` 转写音频。
将音频编码为 data URL 并以 ``input_audio`` 内容块的形式发送到 chat 消息中。
"""
# ── 缓存查找 ──────────────────────────────────────────────────
cache_key = _asr_cache_key(
self._model, request.audio_data, request.language or "auto", request.format
)
cached = _asr_cache_get(cache_key)
if cached is not None:
logger.info("audio_asr.cache_hit", model=self._model, text_length=len(cached))
return ASRResponse(text=cached)
# ── 构造请求 ──────────────────────────────────────────────────
mime = _MIME_MAP.get(request.format, "audio/wav")
b64_audio = base64.b64encode(request.audio_data).decode()
data_url = f"data:{mime};base64,{b64_audio}"
payload: dict = {
"model": self._model,
"messages": [
{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": {
"data": data_url,
},
},
],
},
],
"asr_options": {
"language": request.language or "auto",
},
}
headers = {
"Authorization": f"Bearer {self._api_key}",
"Content-Type": "application/json",
}
url = f"{self._base_url}/chat/completions"
logger.info(
"audio_asr.request",
model=self._model,
format=request.format,
language=request.language,
url=url,
size_bytes=len(request.audio_data),
)
# ── API 调用 ────────────────────────────────────────────────
try:
response = await self._http.post(url, json=payload, headers=headers)
except httpx.TimeoutException as exc:
raise AudioServiceError(
status_code=504,
code="audio_asr_timeout",
message="ASR request timed out calling chat completions.",
) from exc
except httpx.RequestError as exc:
raise AudioServiceError(
status_code=503,
code="audio_asr_connection_error",
message=f"ASR connection error: {exc}",
) from exc
if response.status_code != 200:
body = response.text[:500]
raise AudioServiceError(
status_code=response.status_code,
code="audio_asr_api_error",
message=f"ASR API returned {response.status_code}: {body}",
)
# 解析响应,提取转写文本
text = self._extract_text(response.json())
if not text:
raise AudioServiceError(
status_code=500,
code="audio_asr_empty_response",
message="ASR API returned empty transcription.",
)
# ── 写入缓存 ────────────────────────────────────────────────
_asr_cache_set(cache_key, text)
logger.info(
"audio_asr.success",
model=self._model,
text_length=len(text),
cache_size=len(_asr_cache),
)
return ASRResponse(text=text)
# ── 辅助方法 ──────────────────────────────────────────────────────────
@staticmethod
def _extract_text(body: dict) -> str:
"""从 chat completion 响应中提取转写文本。"""
try:
choices = body.get("choices", [])
if not choices:
raise AudioServiceError(
status_code=500,
code="audio_asr_no_choices",
message="ASR response has no choices.",
)
return (choices[0].get("message") or {}).get("content") or ""
except AudioServiceError:
raise
except Exception as exc:
raise AudioServiceError(
status_code=500,
code="audio_asr_response_parse_error",
message=f"Failed to parse ASR response: {exc}",
) from exc
-245
View File
@@ -1,245 +0,0 @@
"""音频服务类型 — 请求/响应模型和协议定义。
ASR(自动语音识别)和 TTS(文本转语音)适配器**不是** LangChain 的
``BaseChatModel`` 子类 —— 它们是通过 HTTP 调用的服务,经由 ``AudioService``
或直接从 ``ModelFactory`` 消费。
用法::
service = AudioService(factory)
result = await service.transcribe(audio_data)
result = await service.synthesize("你好")
# 或通过工厂直接调用:
tts = factory.create_audio("mimo-v2.5-tts")
result = await tts.synthesize(TTSRequest(text="你好"))
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Protocol, runtime_checkable
from app.ai_provider.types.errors import AIProviderError
if TYPE_CHECKING:
from app.ai_provider.config.models import ModelConfig, ModelEndpointConfig
class AudioServiceError(AIProviderError):
"""ASR / TTS 服务失败的基础错误。"""
# ── ASR ───────────────────────────────────────────────────────────────────
class ASRRequest:
"""将音频转写为文本的请求。
属性:
audio_data: 原始音频字节。
format: 音频容器格式(如 ``"wav"``、``"mp3"``、``"m4a"``)。
language: 可选的 ISO-639-1 语言提示(如 ``"zh"``、``"en"``)。
"""
__slots__ = ("audio_data", "format", "language")
def __init__(
self,
audio_data: bytes,
format: str = "wav",
language: str | None = None,
) -> None:
self.audio_data = audio_data
self.format = format
self.language = language
@dataclass(frozen=True)
class ASRResponse:
"""成功转写的结果。"""
text: str
@runtime_checkable
class ASRProtocol(Protocol):
"""ASR 适配器的协议。"""
async def transcribe(self, request: ASRRequest) -> ASRResponse:
"""将音频字节转写为文本。
异常:
AudioServiceError: API 调用失败或响应异常时抛出。
"""
...
# ── TTS ───────────────────────────────────────────────────────────────────
class TTSRequest:
"""将文本合成为语音的请求。
属性:
text: 待朗读的文本。
voice: 音色标识符(模型决定可用音色)。
response_format: 期望的音频格式(``"wav"``、``"mp3"``、``"opus"``)。
extra: 厂商特定的额外参数(如音色克隆/设计参数)。
"""
__slots__ = ("text", "voice", "response_format", "extra")
def __init__(
self,
text: str,
voice: str = "default",
response_format: str = "wav",
extra: dict | None = None,
) -> None:
self.text = text
self.voice = voice
self.response_format = response_format
self.extra = extra or {}
@dataclass(frozen=True)
class TTSResponse:
"""成功合成的结果。"""
audio_data: bytes
format: str
@runtime_checkable
class TTSProtocol(Protocol):
"""TTS 适配器的协议。"""
async def synthesize(self, request: TTSRequest) -> TTSResponse:
"""将文本合成为音频字节。
异常:
AudioServiceError: API 调用失败或响应异常时抛出。
"""
...
# ── 共享工厂基类 ──────────────────────────────────────────────────────────
class AudioAdapterBase:
"""ASR/TTS 适配器的基类,提供共享的工厂方法。
子类必须设置 ``_adapter_type``(如 ``"asr"``、``"tts"``),并在
``__init__`` 中接受 ``base_url``、``api_key``、``model``、
``request_timeout_seconds`` 参数。
"""
_adapter_type: str = "audio"
_default_timeout: int = 120
@classmethod
def from_endpoint(
cls,
endpoint: ModelEndpointConfig,
*,
provider: ProviderConfig | None = None,
timeout_seconds: int | None = None,
) -> Any:
"""从已解析的端点配置构建适配器。
凭证解析优先级:
1. endpoint 直填的 ``base_url`` / ``api_key``
2. endpoint 的 ``base_url_env`` / ``api_key_env`` 指向的 settings 字段
3. provider 级凭证(调用方按需传入 ProviderConfig)
"""
base_url = _resolve_base_url(endpoint, provider)
api_key = _resolve_api_key(endpoint, provider)
if not base_url:
raise AudioServiceError(
status_code=500,
code=f"audio_{cls._adapter_type}_endpoint_invalid",
message=(
f"{cls._adapter_type.upper()} endpoint '{endpoint.name}' "
"has no resolved base_url."
),
)
return cls(
base_url=base_url,
api_key=api_key,
model=endpoint.model,
request_timeout_seconds=timeout_seconds
or endpoint.request_timeout_seconds
or cls._default_timeout,
)
@classmethod
def from_model_config(
cls,
config: ModelConfig,
*,
provider: ProviderConfig | None = None,
timeout_seconds: int | None = None,
) -> Any:
"""从逻辑模型配置构建适配器(使用第一个活跃端点)。"""
for ep in config.endpoints:
if ep.status == "active":
return cls.from_endpoint(
ep, provider=provider, timeout_seconds=timeout_seconds
)
raise AudioServiceError(
status_code=500,
code=f"audio_{cls._adapter_type}_no_endpoint",
message=f"{cls._adapter_type.upper()} model '{config.name}' has no active endpoint.",
)
def _resolve_base_url(
endpoint: ModelEndpointConfig,
provider: ProviderConfig | None,
) -> str:
"""端点 base_url → endpoint.base_url_env 对应 settings → provider.base_url(_env)。"""
if endpoint.base_url:
return endpoint.base_url
if endpoint.base_url_env:
val = _settings_get(endpoint.base_url_env)
if val:
return val
if provider is not None:
if provider.base_url:
return provider.base_url
if provider.base_url_env:
val = _settings_get(provider.base_url_env)
if val:
return val
return ""
def _resolve_api_key(
endpoint: ModelEndpointConfig,
provider: ProviderConfig | None,
) -> str:
"""端点 api_key → endpoint.api_key_env 对应 settings → provider 级凭证。"""
if endpoint.api_key:
return endpoint.api_key
if endpoint.api_key_env:
val = _settings_get(endpoint.api_key_env)
if val:
return val
if provider is not None:
# ProviderConfig 的 api_key_env / base_url_env 也走 settings 解析
val = _settings_get(provider.api_key_env)
if val:
return val
return ""
def _settings_get(env_name: str) -> str | None:
"""按大写 env 名查 settings 上的小写字段(与现有 _resolve_credentials 约定一致)。"""
from app.core.config import settings
key = env_name.lower()
if hasattr(settings, key):
value = getattr(settings, key)
return value if isinstance(value, str) else None
return None
-325
View File
@@ -1,325 +0,0 @@
"""MiMo TTS adapter — ``POST /v1/chat/completions`` with ``audio`` parameter.
MiMo 的 TTS 不走独立音频端点,而是通过聊天补全接口扩展实现:
POST {base_url}/chat/completions
Content-Type: application/json
Authorization: Bearer {api_key}
{
"model": "mimo-v2.5-tts",
"messages": [
{"role": "user", "content": "(可选)音色风格描述,voicedesign 必填"},
{"role": "assistant", "content": "要朗读的文本"}
],
"audio": {
"format": "wav",
"voice": "mimo_default"
}
}
响应::
{
"choices": [{
"message": {
"audio": {
"data": "<base64 编码的音频数据>",
"id": "...",
"expires_at": null,
"transcript": null
}
}
}]
}
模型差异:
- ``mimo-v2.5-tts`` — ``voice`` 可选,默认 ``mimo_default``,仅支持预置音色
- ``mimo-v2.5-tts-voiceclone`` — ``voice`` 必填,传入音频样本的 base64(mp3/wav)
- ``mimo-v2.5-tts-voicedesign`` — 不支持 ``voice``,\n ``user`` 消息必填(风格描述),支持 ``optimize_text_preview``
"""
from __future__ import annotations
import base64
import hashlib
import time
import httpx
from app.ai_provider.audio.base import (
AudioAdapterBase,
AudioServiceError,
TTSProtocol,
TTSRequest,
TTSResponse,
)
from app.core.logging import logger
# voicedesign 模型的默认风格描述(用户未提供时使用)
_DEFAULT_VOICE_DESIGN_PROMPT = "自然、清晰的朗读风格,语气平和,语速适中,适合日常对话场景。"
# TTS 音频缓存:默认最多 256 条,TTL 1 小时
_TTS_CACHE_MAX_SIZE = 256
_TTS_CACHE_TTL_SECONDS = 3600
class _TTSCacheEntry:
__slots__ = ("data", "format", "stored_at")
def __init__(self, data: bytes, format: str) -> None:
self.data = data
self.format = format
self.stored_at = time.monotonic()
_tts_cache: dict[str, _TTSCacheEntry] = {}
def _tts_cache_key(model: str, voice: str, fmt: str, text: str, extra_json: str) -> str:
"""生成 TTS 缓存键:SHA256(model + voice + format + text + extra)。"""
raw = f"{model}\0{voice}\0{fmt}\0{text}\0{extra_json}"
return hashlib.sha256(raw.encode()).hexdigest()
def _tts_cache_get(key: str) -> _TTSCacheEntry | None:
entry = _tts_cache.get(key)
if entry is None:
return None
if time.monotonic() - entry.stored_at > _TTS_CACHE_TTL_SECONDS:
del _tts_cache[key]
return None
return entry
def _tts_cache_set(key: str, entry: _TTSCacheEntry) -> None:
if len(_tts_cache) >= _TTS_CACHE_MAX_SIZE:
# 淘汰最旧的一半
sorted_keys = sorted(_tts_cache.keys(), key=lambda k: _tts_cache[k].stored_at)
for old_key in sorted_keys[: _TTS_CACHE_MAX_SIZE // 2]:
del _tts_cache[old_key]
_tts_cache[key] = entry
class HunjingTTSOpenAI(AudioAdapterBase, TTSProtocol):
"""MiMo TTS 适配器,通过 ``POST /chat/completions`` 携带 ``audio`` 参数实现。
调用兼容 OpenAI 格式的 chat completions 端点,在请求体中添加 ``audio`` 字段,
然后从响应中提取并解码 base64 音频数据。
"""
_adapter_type = "tts"
_default_timeout = 120
def __init__(
self,
*,
base_url: str,
api_key: str,
model: str = "mimo-v2.5-tts",
request_timeout_seconds: int = 120,
) -> None:
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._model = model
self._timeout = request_timeout_seconds
self._client: httpx.AsyncClient | None = None
# ── 客户端 ───────────────────────────────────────────────────────────
@property
def _http(self) -> httpx.AsyncClient:
if self._client is None or self._client.is_closed:
self._client = httpx.AsyncClient(timeout=httpx.Timeout(self._timeout))
return self._client
async def close(self) -> None:
if self._client is not None and not self._client.is_closed:
await self._client.aclose()
# ── TTSProtocol ──────────────────────────────────────────────────────
async def synthesize(self, request: TTSRequest) -> TTSResponse:
"""通过 ``POST /chat/completions`` 将文本合成为音频。
构造 chat 消息和 ``audio`` 参数,匹配 MiMo 兼容 OpenAI 格式的 TTS,
然后从响应中解码 base64 音频。
结果按 ``(model, voice, format, text)`` 在进程内缓存,避免重复 API 调用。
"""
# ── 缓存查找 ──────────────────────────────────────────────────────
cache_key = _tts_cache_key(
model=self._model,
voice=request.voice,
fmt=request.response_format,
text=request.text,
extra_json=str(sorted((request.extra or {}).items())),
)
cached = _tts_cache_get(cache_key)
if cached is not None:
logger.info("audio_tts.cache_hit", model=self._model, text_length=len(request.text))
return TTSResponse(audio_data=cached.data, format=cached.format)
# ── API 调用 ──────────────────────────────────────────────────────
url = f"{self._base_url}/chat/completions"
# 构造 messages
messages = self._build_messages(request)
# 构造 audio 参数
audio_params = self._build_audio_params(request)
payload: dict = {
"model": self._model,
"messages": messages,
"audio": audio_params,
"max_completion_tokens": 8192,
}
# 合并 extra(如 frequency_penalty, temperature 等)
if request.extra:
payload.update(request.extra)
headers = {
"Authorization": f"Bearer {self._api_key}",
"Content-Type": "application/json",
}
logger.info(
"audio_tts.request",
model=self._model,
voice=request.voice,
audio_format=request.response_format,
text_length=len(request.text),
url=url,
)
try:
response = await self._http.post(url, json=payload, headers=headers)
except httpx.TimeoutException as exc:
raise AudioServiceError(
status_code=504,
code="audio_tts_timeout",
message="TTS request timed out calling chat completions.",
) from exc
except httpx.RequestError as exc:
raise AudioServiceError(
status_code=503,
code="audio_tts_connection_error",
message=f"TTS connection error: {exc}",
) from exc
if response.status_code != 200:
body = response.text[:500]
raise AudioServiceError(
status_code=response.status_code,
code="audio_tts_api_error",
message=f"TTS API returned {response.status_code}: {body}",
)
# 解析响应,提取 base64 音频
audio_data = self._extract_audio(response.json())
# ── 写入缓存 ──────────────────────────────────────────────────────
_tts_cache_set(cache_key, _TTSCacheEntry(audio_data, request.response_format))
logger.info(
"audio_tts.success",
model=self._model,
audio_bytes=len(audio_data),
cache_size=len(_tts_cache),
)
return TTSResponse(audio_data=audio_data, format=request.response_format)
# ── 辅助方法 ──────────────────────────────────────────────────────────
def _build_messages(self, request: TTSRequest) -> list[dict]:
"""构造 chat messages 数组。
- voicedesign 模型:user 消息必填(风格描述)
- 其他模型:user 消息可选,assistant 消息存放待朗读文本
"""
is_voicedesign = "voicedesign" in self._model
if is_voicedesign:
# voicedesign:user 消息为风格提示,assistant 消息为朗读文本
style_prompt = (request.extra or {}).pop("style_prompt", _DEFAULT_VOICE_DESIGN_PROMPT)
return [
{"role": "user", "content": style_prompt},
{"role": "assistant", "content": request.text},
]
# 基础 TTS / voiceclone:只传 assistant 消息
return [
{"role": "assistant", "content": request.text},
]
def _build_audio_params(self, request: TTSRequest) -> dict:
"""构造 ``audio`` 参数字段,按模型处理 voice/format/optimize_text_preview 等。"""
params: dict = {
"format": _resolve_api_format(request.response_format),
}
is_voicedesign = "voicedesign" in self._model
if is_voicedesign:
# voicedesign 不支持 voice 字段
optimize = (request.extra or {}).pop("optimize_text_preview", None)
if optimize is not None:
params["optimize_text_preview"] = optimize
else:
# 基础 TTS / voiceclone:按需传 voice
if request.voice and request.voice != "default":
params["voice"] = request.voice
else:
params["voice"] = "mimo_default"
return params
@staticmethod
def _extract_audio(body: dict) -> bytes:
"""从 chat completion 响应中提取并解码 base64 音频数据。"""
try:
choices = body.get("choices", [])
if not choices:
raise AudioServiceError(
status_code=500,
code="audio_tts_no_choices",
message="TTS response has no choices.",
)
message = choices[0].get("message", {})
audio_info = message.get("audio")
if not audio_info:
raise AudioServiceError(
status_code=500,
code="audio_tts_no_audio_field",
message="TTS response missing audio field in message.",
)
raw = audio_info.get("data")
if not raw:
raise AudioServiceError(
status_code=500,
code="audio_tts_empty_audio_data",
message="TTS response has empty audio.data.",
)
return base64.b64decode(raw)
except AudioServiceError:
raise
except Exception as exc:
raise AudioServiceError(
status_code=500,
code="audio_tts_response_parse_error",
message=f"Failed to parse TTS response: {exc}",
) from exc
def _resolve_api_format(user_format: str) -> str:
"""将用户层面的格式名映射为 API 期望的格式。
MiMo TTS API 接受:``wav``、``mp3``、``pcm``、``pcm16``。
"""
fmt = user_format.lower()
if fmt in ("pcm", "pcm16"):
return "pcm16"
if fmt in ("wav", "mp3"):
return fmt
return "wav"
-231
View File
@@ -1,231 +0,0 @@
"""yuanqi 本地 ASR 适配器 — ``POST /v1/audio/transcriptions`` (multipart)。
与 MiMo 走 ``/chat/completions`` + ``input_audio`` 不同,元炁本地推理引擎
(yuanqi)对外暴露的是 OpenAI Whisper 兼容的 multipart 端点::
POST {base_url}/v1/audio/transcriptions
Content-Type: multipart/form-data
Authorization: Bearer {api_key}
file: <audio bytes>
language: zh (可选)
响应::
{"text": "转写后的文本"}
模型实际为 FunASR SenseVoiceSmall,转写文本已在 yuanqi 侧剥离了
``<|zh|><|NEUTRAL|><|Speech|><|withitn|>`` 等内部标签,此处无需二次清洗。
"""
from __future__ import annotations
import hashlib
import time
import httpx
from app.ai_provider.audio.base import (
ASRProtocol,
ASRRequest,
ASRResponse,
AudioAdapterBase,
AudioServiceError,
)
from app.core.logging import logger
# multipart 使用的 MIME 映射(与 yuanqi _SUPPORTED_FORMATS 白名单对齐)
_MIME_MAP = {
"wav": "audio/wav",
"mp3": "audio/mpeg",
"m4a": "audio/mp4",
"flac": "audio/flac",
"ogg": "audio/ogg",
"opus": "audio/opus",
"aac": "audio/aac",
"webm": "audio/webm",
}
# yuanqi ASR 路径前缀(与 src/endpoints/asr.py 的 router 前缀一致)
_ASR_PATH = "/v1/audio/transcriptions"
# ASR 缓存:默认最多 128 条,TTL 1 小时(与 MiMo ASR 适配器一致)
_ASR_CACHE_MAX_SIZE = 128
_ASR_CACHE_TTL_SECONDS = 3600
class _ASRCacheEntry:
__slots__ = ("text", "stored_at")
def __init__(self, text: str) -> None:
self.text = text
self.stored_at = time.monotonic()
_asr_cache: dict[str, _ASRCacheEntry] = {}
def _asr_cache_key(model: str, audio_data: bytes, language: str, fmt: str) -> str:
"""生成 ASR 缓存键:SHA256(model + audio_bytes + language + format)。"""
raw = f"{model}\0{language}\0{fmt}\0"
raw_bytes = raw.encode() + audio_data
return hashlib.sha256(raw_bytes).hexdigest()
def _asr_cache_get(key: str) -> str | None:
entry = _asr_cache.get(key)
if entry is None:
return None
if time.monotonic() - entry.stored_at > _ASR_CACHE_TTL_SECONDS:
del _asr_cache[key]
return None
return entry.text
def _asr_cache_set(key: str, text: str) -> None:
if len(_asr_cache) >= _ASR_CACHE_MAX_SIZE:
sorted_keys = sorted(_asr_cache.keys(), key=lambda k: _asr_cache[k].stored_at)
for old_key in sorted_keys[: _ASR_CACHE_MAX_SIZE // 2]:
del _asr_cache[old_key]
_asr_cache[key] = _ASRCacheEntry(text)
class YuanqiASRWhisper(AudioAdapterBase, ASRProtocol):
"""yuanqi 本地 ASR 适配器,走 OpenAI Whisper 兼容的 multipart 端点。
将音频字节以 ``multipart/form-data`` 的 ``file`` 字段上传到 yuanqi 的
``/v1/audio/transcriptions``,从 JSON 响应中提取转写文本。
"""
_adapter_type = "asr"
_default_timeout = 300
def __init__(
self,
*,
base_url: str,
api_key: str,
model: str = "SenseVoiceSmall",
request_timeout_seconds: int = 300,
) -> None:
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._model = model
self._timeout = request_timeout_seconds
self._client: httpx.AsyncClient | None = None
# ── 客户端 ───────────────────────────────────────────────────────────
@property
def _http(self) -> httpx.AsyncClient:
if self._client is None or self._client.is_closed:
self._client = httpx.AsyncClient(timeout=httpx.Timeout(self._timeout))
return self._client
async def close(self) -> None:
if self._client is not None and not self._client.is_closed:
await self._client.aclose()
# ── ASRProtocol ──────────────────────────────────────────────────────
async def transcribe(self, request: ASRRequest) -> ASRResponse:
"""通过 ``POST /v1/audio/transcriptions`` 转写音频。"""
# ── 缓存查找 ──────────────────────────────────────────────────
cache_key = _asr_cache_key(
self._model, request.audio_data, request.language or "auto", request.format
)
cached = _asr_cache_get(cache_key)
if cached is not None:
logger.info("audio_asr.cache_hit", model=self._model, text_length=len(cached))
return ASRResponse(text=cached)
# ── 构造 multipart 请求 ───────────────────────────────────────
mime = _MIME_MAP.get(request.format, "audio/wav")
filename = f"audio.{request.format}"
files = {"file": (filename, request.audio_data, mime)}
data: dict[str, str] = {}
if request.language:
data["language"] = request.language
# 本地 yuanqi 引擎通常无需鉴权;仅当配置了 api_key 才带 Authorization 头
headers: dict[str, str] = {}
if self._api_key:
headers["Authorization"] = f"Bearer {self._api_key}"
url = f"{self._base_url}{_ASR_PATH}"
logger.info(
"audio_asr.request",
model=self._model,
format=request.format,
language=request.language,
url=url,
size_bytes=len(request.audio_data),
)
# ── API 调用 ────────────────────────────────────────────────
try:
response = await self._http.post(url, files=files, data=data, headers=headers)
except httpx.TimeoutException as exc:
raise AudioServiceError(
status_code=504,
code="audio_asr_timeout",
message="ASR request timed out calling yuanqi transcriptions.",
) from exc
except httpx.RequestError as exc:
raise AudioServiceError(
status_code=503,
code="audio_asr_connection_error",
message=f"ASR connection error: {exc}",
) from exc
if response.status_code != 200:
body = response.text[:500]
raise AudioServiceError(
status_code=response.status_code,
code="audio_asr_api_error",
message=f"ASR API returned {response.status_code}: {body}",
)
# 解析响应,提取转写文本
text = self._extract_text(response.json())
if not text:
raise AudioServiceError(
status_code=500,
code="audio_asr_empty_response",
message="ASR API returned empty transcription.",
)
# ── 写入缓存 ────────────────────────────────────────────────
_asr_cache_set(cache_key, text)
logger.info(
"audio_asr.success",
model=self._model,
text_length=len(text),
cache_size=len(_asr_cache),
)
return ASRResponse(text=text)
# ── 辅助方法 ──────────────────────────────────────────────────────────
@staticmethod
def _extract_text(body: dict) -> str:
"""从 Whisper 兼容响应中提取转写文本。"""
try:
text = body.get("text")
if text is None:
raise AudioServiceError(
status_code=500,
code="audio_asr_no_text",
message="ASR response has no 'text' field.",
)
return text or ""
except AudioServiceError:
raise
except Exception as exc:
raise AudioServiceError(
status_code=500,
code="audio_asr_response_parse_error",
message=f"Failed to parse ASR response: {exc}",
) from exc
@@ -1,31 +0,0 @@
from app.ai_provider.config.loader import (
get_models_config_path,
get_pricing_config_path,
get_providers_config_path,
load_models_config,
load_pricing_config,
load_providers_config,
)
from app.ai_provider.config.models import (
ModelConfig,
ModelEndpointConfig,
ParameterMapping,
ProviderConfig,
derive_capabilities,
)
from app.ai_provider.core.billing import PricingInfo
__all__ = [
"ModelConfig",
"ModelEndpointConfig",
"PricingInfo",
"ProviderConfig",
"ParameterMapping",
"derive_capabilities",
"load_models_config",
"load_providers_config",
"load_pricing_config",
"get_models_config_path",
"get_providers_config_path",
"get_pricing_config_path",
]
-80
View File
@@ -1,80 +0,0 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
import yaml
from pydantic import BaseModel
from app.ai_provider.config.models import ModelConfig, ProviderConfig
from app.core.config import settings
class _ModelsFile(BaseModel):
version: int = 1
models: list[ModelConfig]
class _ProvidersFile(BaseModel):
version: int = 1
providers: dict[str, ProviderConfig]
class _PricingFile(BaseModel):
"""pricing.yaml 顶层结构。"""
version: int = 1
pricing: dict[str, dict] = {}
def get_models_config_path() -> Path:
"""返回已解析的 models.yaml 路径(project_root/config/models.yaml)。"""
return settings.project_root / "config" / "models.yaml"
def get_providers_config_path() -> Path:
"""返回已解析的 providers.yaml 路径(project_root/config/providers.yaml)。"""
return settings.project_root / "config" / "providers.yaml"
def load_models_config(path: Path | None = None) -> list[ModelConfig]:
"""从 YAML 文件加载模型配置。如果未指定路径,使用配置中的默认路径。"""
if path is None:
path = get_models_config_path()
if not path.exists():
raise FileNotFoundError(f"Models config not found: {path}")
raw: dict[str, Any] = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
parsed = _ModelsFile.model_validate(raw)
return parsed.models
def load_providers_config(path: Path | None = None) -> dict[str, ProviderConfig]:
"""从 YAML 文件加载提供商配置。如果未指定路径,使用配置中的默认路径。"""
if path is None:
path = get_providers_config_path()
if not path.exists():
raise FileNotFoundError(f"Providers config not found: {path}")
raw: dict[str, Any] = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
# 验证前将字典键注入为 provider 的 name 字段
providers_raw = raw.get("providers", {})
for name, cfg in providers_raw.items():
if isinstance(cfg, dict) and "name" not in cfg:
cfg["name"] = name
parsed = _ProvidersFile.model_validate(raw)
return parsed.providers
def get_pricing_config_path() -> Path:
"""返回 pricing.yaml 路径(project_root/config/pricing.yaml)。"""
return settings.project_root / "config" / "pricing.yaml"
def load_pricing_config(path: Path | None = None) -> dict[str, dict]:
"""从 YAML 文件加载定价配置。"""
if path is None:
path = get_pricing_config_path()
if not path.exists():
return {}
raw: dict[str, Any] = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
parsed = _PricingFile.model_validate(raw)
return parsed.pricing
-158
View File
@@ -1,158 +0,0 @@
from __future__ import annotations
from enum import StrEnum
from typing import Any
from pydantic import ( # noqa: TC002 — Pydantic 模型
BaseModel,
Field,
field_validator,
model_validator,
)
class SupportedFormats(StrEnum):
OPENAI = "openai"
ANTHROPIC = "anthropic"
_BASE_CAPABILITIES = {
("openai", "gpt"): {"chat", "tool_use", "structured_output"},
("openai", "gpt-4"): {"chat", "tool_use", "structured_output", "vision", "reasoning"},
("anthropic", "claude"): {
"chat",
"tool_use",
"structured_output",
"vision",
"reasoning",
"long_context",
},
("deepseek", "deepseek"): {"chat", "tool_use", "structured_output", "reasoning"},
("deepseek", "deepseek-v4"): {
"chat",
"tool_use",
"structured_output",
"reasoning",
"long_context",
},
}
def derive_capabilities(provider: str, model: str) -> set[str]:
"""根据 provider + model 前缀自动推导能力。
调用者可以在调用此函数后,在 ModelConfig 上覆盖个别标记。
"""
for (prov_prefix, model_prefix), caps in _BASE_CAPABILITIES.items():
if provider.startswith(prov_prefix) and model.startswith(model_prefix):
return set(caps)
return {"chat"}
class ModelEndpointConfig(BaseModel):
name: str
model: str
base_url: str | None = None
base_url_env: str | None = None
api_key: str | None = None
api_key_env: str | None = None
weight: int = 100
priority: int = 100
rpm_limit: int | None = None
tpm_limit: int | None = None
max_concurrency: int | None = None
request_timeout_seconds: int | None = None
status: str = "active"
extra_params: dict[str, Any] = Field(default_factory=dict)
class ModelConfig(BaseModel):
name: str
display_name: str | None = None
provider: str
model: str
format: SupportedFormats # "openai" | "anthropic"
chat_format: str | None = None
tags: list[str] = Field(default_factory=list)
supports_tools: bool | None = None
supports_vision: bool | None = None
supports_reasoning_effort: bool | None = None
supports_thinking: bool | None = None
supports_streaming: bool | None = True
supports_structured_output: bool | None = None
context_window: int = 32000
response_reserve: int = 4096
max_output_tokens: int | None = None
max_thinking_tokens: int | None = None
reasoning_effort: str | None = None
timeout_seconds: int | None = 60
dimensions: int | None = None # 向量维度(embedding 模型专用)
routing_strategy: str = "weighted_round_robin"
endpoint_pool_enabled: bool = False
weight: int = 100
endpoints: list[ModelEndpointConfig] = Field(default_factory=list)
@model_validator(mode="after")
def _validate_embedding_dimensions(self) -> ModelConfig:
"""embedding 模型必须声明 dimensions。"""
if "embedding" in self.tags and self.dimensions is None:
raise ValueError(
f"Embedding 模型 '{self.name}' 必须设置 dimensions(如 dimensions: 1024)"
)
return self
def resolved_capabilities(self) -> dict[str, bool]:
base = derive_capabilities(self.provider, self.model)
return {
"chat": True,
"tool_use": self.supports_tools
if self.supports_tools is not None
else ("tool_use" in base),
"vision": self.supports_vision
if self.supports_vision is not None
else ("vision" in base),
"reasoning": self.supports_reasoning_effort
if self.supports_reasoning_effort is not None
else ("reasoning" in base),
"thinking": self.supports_thinking
if self.supports_thinking is not None
else ("reasoning" in base),
"streaming": self.supports_streaming if self.supports_streaming is not None else True,
"structured_output": self.supports_structured_output
if self.supports_structured_output is not None
else ("structured_output" in base),
}
class ParameterMapping(BaseModel):
provider_param: str
min: float | None = None
max: float | None = None
default: Any = None
transform: str | None = None # "multiply_1000", "int", "str"
alias_for: str | None = None # map from another param name
handler: str | None = None # named handler for complex param transform
class ProviderConfig(BaseModel):
name: str
format: SupportedFormats
base_url_env: str
api_key_env: str
base_url: str | None = None
timeout_seconds: int = 60
max_retries: int = 1
parameter_mappings: dict[str, ParameterMapping] = Field(default_factory=dict)
parameter_handlers: list[str] = Field(
default_factory=list
) # ordered global handlers per provider
extra_headers: dict[str, str] = Field(default_factory=dict)
supports_streaming: bool = True
@field_validator("format")
@classmethod
def _validate_format(cls, v: str) -> str:
allowed = {e.value for e in SupportedFormats}
if v not in allowed:
raise ValueError(f"format must be one of {allowed}, got '{v}'")
return v
-17
View File
@@ -1,17 +0,0 @@
from app.ai_provider.core.billing import PricingFactory, PricingInfo, PricingRegistry
from app.ai_provider.core.registry import (
ModelRegistry,
ProviderRegistry,
RegistryFactory,
RegistryHolder,
)
__all__ = [
"ModelRegistry",
"ProviderRegistry",
"RegistryFactory",
"RegistryHolder",
"PricingInfo",
"PricingRegistry",
"PricingFactory",
]
-253
View File
@@ -1,253 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import date, datetime
from pydantic import BaseModel
from app.ai_provider.types.chat import TokenUsage
@dataclass
class CostBreakdown:
input_cost: float = 0.0
output_cost: float = 0.0
total: float = 0.0
currency: str = "CNY"
class CostCalculator:
"""根据价格信息和 token 数量计算成本。
保持厂商原始单位 —— 不做跨币种换算。
- 普通 input token 按 ``input_per_1m`` 计费。
- 缓存命中的 prompt token 优先按 ``input_cache_read_per_1m`` 计费
(若有配置),剩余按普通 input 费率。
- 若未配置 cache 费率,所有 prompt token 统一按 input_per_1m 计费。
"""
@staticmethod
def calculate(
pricing: PricingInfo | None,
prompt_tokens: int,
completion_tokens: int,
prompt_tokens_cached: int = 0,
) -> CostBreakdown:
if pricing is None:
return CostBreakdown(total=0.0)
# 缓存 token:优先走缓存费率
cached = min(prompt_tokens_cached, prompt_tokens)
uncached = prompt_tokens - cached
input_cost = 0.0
if uncached > 0 and pricing.input_per_1m is not None:
input_cost += (uncached / 1_000_000) * pricing.input_per_1m
if cached > 0 and pricing.input_cache_read_per_1m is not None:
input_cost += (cached / 1_000_000) * pricing.input_cache_read_per_1m
elif cached > 0 and pricing.input_per_1m is not None:
# 无缓存费率时统一按普通 input 计
input_cost += (cached / 1_000_000) * pricing.input_per_1m
output_cost = 0.0
if pricing.output_per_1m is not None and completion_tokens > 0:
output_cost = (completion_tokens / 1_000_000) * pricing.output_per_1m
return CostBreakdown(
input_cost=round(input_cost, 8),
output_cost=round(output_cost, 8),
total=round(input_cost + output_cost, 8),
currency=pricing.currency,
)
@dataclass
class UsageRecord:
model_name: str
provider: str
prompt_tokens: int = 0
completion_tokens: int = 0
total_tokens: int = 0
currency: str = "CNY"
cost: float = 0.0
timestamp: datetime = field(default_factory=datetime.now)
endpoint_name: str | None = None
status: str = "success"
error_message: str | None = None
class BillingTracker:
"""记录用量并按模型计算成本。"""
def __init__(self) -> None:
self._records: list[UsageRecord] = []
def record(
self,
model_name: str,
provider: str,
usage: TokenUsage | None,
pricing: PricingInfo | None = None,
endpoint_name: str | None = None,
status: str = "success",
error_message: str | None = None,
) -> UsageRecord:
prompt_tokens = usage.prompt_tokens if usage else 0
completion_tokens = usage.completion_tokens if usage else 0
total = usage.total_tokens if usage else 0
cached = usage.prompt_tokens_cached if usage else 0
cost_info = CostCalculator.calculate(pricing, prompt_tokens, completion_tokens, cached)
record = UsageRecord(
model_name=model_name,
provider=provider,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
total_tokens=total,
currency=cost_info.currency,
cost=cost_info.total,
endpoint_name=endpoint_name,
status=status,
error_message=error_message,
)
self._records.append(record)
return record
def get_records(
self,
since: datetime | None = None,
model_name: str | None = None,
) -> list[UsageRecord]:
records = self._records
if since:
records = [r for r in records if r.timestamp >= since]
if model_name:
records = [r for r in records if r.model_name == model_name]
return records
def total_cost(self, currency: str | None = None) -> float:
if currency:
return sum(r.cost for r in self._records if r.currency == currency)
return sum(r.cost for r in self._records)
class BudgetController:
"""强制执行每日和每月预算限制。"""
def __init__(
self,
daily_budget_cny: float = 0.0,
monthly_budget_cny: float = 0.0,
) -> None:
self.daily_budget_cny = daily_budget_cny
self.monthly_budget_cny = monthly_budget_cny
self._daily_cost_cny: float = 0.0
self._monthly_cost_cny: float = 0.0
self._current_date: date = date.today()
def add_cost_cny(self, cost: float) -> None:
self._daily_cost_cny += cost
self._monthly_cost_cny += cost
def allow_request(self) -> bool:
return not (
(self.daily_budget_cny > 0 and self._daily_cost_cny >= self.daily_budget_cny)
or (self.monthly_budget_cny > 0 and self._monthly_cost_cny >= self.monthly_budget_cny)
)
def reset_daily(self) -> None:
self._daily_cost_cny = 0.0
self._current_date = date.today()
def reset_monthly(self) -> None:
self._monthly_cost_cny = 0.0
class BillingFactory:
"""计费组件工厂。"""
@staticmethod
def create_tracker() -> BillingTracker:
return BillingTracker()
@staticmethod
def create_budget_controller(
daily_budget_cny: float = 0.0,
monthly_budget_cny: float = 0.0,
) -> BudgetController:
return BudgetController(
daily_budget_cny=daily_budget_cny,
monthly_budget_cny=monthly_budget_cny,
)
# ── 定价信息 ──────────────────────────────────────────────────────────
class PricingInfo(BaseModel):
"""模型定价信息。保持厂商原始单位,不转换不混合。
各厂商按各自币种/单位记录,展示时原生显示。
"""
currency: str = "CNY"
input_per_1m: float | None = None
output_per_1m: float | None = None
input_cache_read_per_1m: float | None = None
class PricingRegistry:
"""模型定价注册表。以模型逻辑名为键查找定价。
与 ModelRegistry 解耦 —— 改价不影响模型注册,无定价的模型
也可以正常使用(不计费)。
"""
def __init__(self, pricing_map: dict[str, PricingInfo] | None = None) -> None:
self._pricing: dict[str, PricingInfo] = dict(pricing_map or {})
def get(self, model_name: str) -> PricingInfo | None:
"""获取指定模型的定价信息。无定价时返回 None。"""
return self._pricing.get(model_name)
def list_all(self) -> dict[str, PricingInfo]:
"""返回全部定价数据的副本。"""
return dict(self._pricing)
class PricingFactory:
"""定价注册表工厂。"""
@staticmethod
def from_dict(data: dict[str, dict]) -> PricingRegistry:
"""从 YAML 解析后的 dict 构建 PricingRegistry。
YAML 按厂商嵌套组织(自文档化),工厂将其摊平为按模型逻辑名索引:
```yaml
pricing:
deepseek:
deepseek-v4-flash:
currency: CNY
input_per_1m: 1.0
anthropic:
claude-sonnet:
currency: USD
input_per_1m: 3.0
```
"""
pricing_map: dict[str, PricingInfo] = {}
for _provider_name, models_data in data.items():
if not isinstance(models_data, dict):
continue
for model_name, raw in models_data.items():
if raw is None:
continue
# 跳过所有字段均为 null 的条目(如 mimo-v2.5-asr)
values = [v for v in raw.values() if not isinstance(v, bool)]
if all(v is None for v in values):
continue
pricing_map[model_name] = PricingInfo(**raw)
return PricingRegistry(pricing_map)
-164
View File
@@ -1,164 +0,0 @@
from __future__ import annotations
import random
from app.ai_provider.config.models import ModelConfig, ProviderConfig
from app.ai_provider.core.billing import PricingFactory, PricingRegistry
from app.ai_provider.types.errors import AIProviderError
class ModelRegistry:
"""内存模型索引。支持逻辑名和别名查找。"""
def __init__(
self,
models: list[ModelConfig],
aliases: dict[str, str] | None = None,
) -> None:
self._models: dict[str, ModelConfig] = {}
self._aliases: dict[str, str] = dict(aliases or {})
self._default: str | None = None
self._fallback: str | None = None
self._capability_index: dict[str, list[ModelConfig]] = {}
for m in models:
self._models[m.name] = m
if "default" in m.tags and self._default is None:
self._default = m.name
if "fallback" in m.tags and self._fallback is None:
self._fallback = m.name
caps = m.resolved_capabilities()
for cap, supported in caps.items():
if supported:
self._capability_index.setdefault(cap, []).append(m)
# 索引 tags(如 tts / asr / voice_clone)——这些不在
# resolved_capabilities() 中,但 AudioService 等模块依赖它们查找模型。
for tag in m.tags:
self._capability_index.setdefault(tag, []).append(m)
if self._default is None and models:
self._default = models[0].name
def get(self, name: str) -> ModelConfig:
model = self._models.get(name)
if model is None:
raise AIProviderError(
message=f"Model '{name}' not found in registry",
code="MODEL_NOT_FOUND",
status_code=404,
)
return model
def resolve(self, name_or_alias: str | None = None) -> ModelConfig:
"""解析逻辑名或别名为 ModelConfig。
无参数时返回默认模型。
"""
if name_or_alias is None:
return self._models[self.get_default_name()]
if name_or_alias in self._models:
return self._models[name_or_alias]
if name_or_alias in self._aliases:
logical = self._aliases[name_or_alias]
return self._models[logical]
raise AIProviderError(
message=f"Model or alias '{name_or_alias}' not found",
code="MODEL_NOT_FOUND",
status_code=404,
)
def resolve_name(self, name_or_alias: str) -> str:
"""解析为规范逻辑名(不返回完整配置)。"""
if name_or_alias in self._models:
return name_or_alias
if name_or_alias in self._aliases:
return self._aliases[name_or_alias]
raise AIProviderError(
message=f"Model or alias '{name_or_alias}' not found",
code="MODEL_NOT_FOUND",
status_code=404,
)
def get_default_name(self) -> str:
if self._default is None:
raise AIProviderError(
message="No default model configured",
code="NO_DEFAULT_MODEL",
status_code=500,
)
return self._default
def get_fallback_name(self) -> str | None:
return self._fallback
def contains(self, name: str) -> bool:
return name in self._models
def list_all(self) -> list[ModelConfig]:
return list(self._models.values())
def list_by_capability(self, capability: str) -> list[ModelConfig]:
return list(self._capability_index.get(capability, []))
def resolve_by_capability(self, capability: str) -> str:
candidates = self._capability_index.get(capability)
if not candidates:
raise AIProviderError(
message=f"No active models with capability '{capability}'",
code="CAPABILITY_NOT_FOUND",
status_code=404,
)
weights = [max(c.weight, 1) for c in candidates]
chosen = random.choices(candidates, weights=weights, k=1)[0]
return chosen.name
class ProviderRegistry:
"""内存提供商配置索引。"""
def __init__(self, providers: dict[str, ProviderConfig]) -> None:
self._providers = dict(providers)
def get(self, name: str) -> ProviderConfig:
prov = self._providers.get(name)
if prov is None:
raise AIProviderError(
message=f"Provider '{name}' not configured",
code="PROVIDER_NOT_FOUND",
status_code=404,
)
return prov
def list_all(self) -> dict[str, ProviderConfig]:
return dict(self._providers)
class RegistryFactory:
"""工厂:从配置列表创建注册表。"""
@staticmethod
def create(
models: list[ModelConfig],
providers: dict[str, ProviderConfig],
aliases: dict[str, str] | None = None,
pricing_data: dict[str, dict] | None = None,
) -> RegistryHolder:
return RegistryHolder(
model_registry=ModelRegistry(models, aliases=aliases),
provider_registry=ProviderRegistry(providers),
pricing_registry=PricingFactory.from_dict(pricing_data or {}),
)
class RegistryHolder:
"""同时持有所有注册表。"""
def __init__(
self,
model_registry: ModelRegistry,
provider_registry: ProviderRegistry,
pricing_registry: PricingRegistry | None = None,
) -> None:
self.model_registry = model_registry
self.provider_registry = provider_registry
self.pricing_registry = pricing_registry or PricingRegistry()
-133
View File
@@ -1,133 +0,0 @@
from __future__ import annotations
import time
from enum import Enum
class CircuitBreakerState(Enum):
CLOSED = "closed"
OPEN = "open"
HALF_OPEN = "half_open"
class CircuitBreaker:
"""标准断路器,冷却后自动半开。"""
def __init__(
self,
name: str,
failure_threshold: int = 5,
cooldown_seconds: float = 30.0,
) -> None:
self.name = name
self.failure_threshold = failure_threshold
self.cooldown_seconds = cooldown_seconds
self._state = CircuitBreakerState.CLOSED
self._failure_count = 0
self._last_failure_time: float = 0.0
@property
def state(self) -> CircuitBreakerState:
if (
self._state == CircuitBreakerState.OPEN
and time.monotonic() - self._last_failure_time >= self.cooldown_seconds
):
self._state = CircuitBreakerState.HALF_OPEN
return self._state
def record_success(self) -> None:
self._failure_count = 0
self._state = CircuitBreakerState.CLOSED
def record_failure(self) -> None:
self._failure_count += 1
self._last_failure_time = time.monotonic()
if self._failure_count >= self.failure_threshold:
self._state = CircuitBreakerState.OPEN
def allow_request(self) -> bool:
return self.state != CircuitBreakerState.OPEN
class RetryStrategy:
"""指数退避重试。"""
def __init__(
self,
max_retries: int = 3,
base_delay: float = 1.0,
max_delay: float = 60.0,
multiplier: float = 2.0,
) -> None:
self.max_retries = max_retries
self.base_delay = base_delay
self.max_delay = max_delay
self.multiplier = multiplier
def should_retry(self, attempt_index: int) -> bool:
return attempt_index < self.max_retries
def get_delay(self, attempt_index: int) -> float:
delay = self.base_delay * (self.multiplier**attempt_index)
return min(delay, self.max_delay)
class DowngradeAction:
"""降级链中的一个步骤。"""
def __init__(self, type: str, target: str) -> None: # noqa: A002
self.type = type # "fallback_endpoint", "fallback_model", "cache", "degrade_quality"
self.target = target
class DowngradeChain:
"""当模型/端点失败时的降级动作链。"""
def __init__(self, actions: list[DowngradeAction]) -> None:
self._actions = actions
self._index = 0
def next_action(self) -> tuple[str, str] | None:
if self._index >= len(self._actions):
return None
action = self._actions[self._index]
return (action.type, action.target)
def advance(self) -> None:
self._index += 1
def is_exhausted(self) -> bool:
return self._index >= len(self._actions)
def reset(self) -> None:
self._index = 0
class ResilienceFactory:
"""弹性组件工厂。"""
@staticmethod
def create_circuit_breaker(
name: str,
failure_threshold: int = 5,
cooldown_seconds: float = 30.0,
) -> CircuitBreaker:
return CircuitBreaker(
name=name,
failure_threshold=failure_threshold,
cooldown_seconds=cooldown_seconds,
)
@staticmethod
def create_retry_strategy(
max_retries: int = 3,
base_delay: float = 1.0,
max_delay: float = 60.0,
multiplier: float = 2.0,
) -> RetryStrategy:
return RetryStrategy(
max_retries=max_retries,
base_delay=base_delay,
max_delay=max_delay,
multiplier=multiplier,
)
@@ -1,149 +0,0 @@
"""嵌入服务 — Embedding 适配器外观。
用法::
from app.ai_provider.embedding import EmbeddingService
service = EmbeddingService(manager)
result = await service.embed("要嵌入的文本")
results = await service.embed(["文本1", "文本2"])
"""
from __future__ import annotations
from app.ai_provider.embedding.base import (
EmbeddingProtocol,
EmbeddingRequest,
EmbeddingResponse,
EmbeddingResult,
EmbeddingServiceError,
)
from app.ai_provider.types.errors import AIProviderError
class EmbeddingService:
"""外观类,负责解析 Embedding 模型并委托给 ProviderManager。
接受 ProviderManager 或实现了以下接口的适配器:
- ``.model_registry`` → 具有 ``list_by_capability(capability)`` 方法
- ``.create_embedding(name)`` → 返回 EmbeddingProtocol 实例
"""
def __init__(self, manager) -> None:
self._manager = manager
self._adapter: EmbeddingProtocol | None = None
# ── Registry 兼容适配 ─────────────────────────────────────────
@property
def _registry(self):
mgr = self._manager
if hasattr(mgr, "model_registry"):
return mgr.model_registry
return mgr
@property
def _factory(self):
mgr = self._manager
if hasattr(mgr, "create_embedding"):
return mgr
return None
# ── Embedding ──────────────────────────────────────────────────
@property
def adapter(self) -> EmbeddingProtocol:
"""解析并缓存 Embedding 适配器。"""
if self._adapter is None:
self._adapter = self._resolve()
return self._adapter
async def embed(
self,
input: str | list[str],
model_hint: str | None = None,
dimensions: int | None = None,
) -> EmbeddingResponse:
"""快捷方法:构造请求并调用 Embedding 适配器。"""
model_name = self._resolve_model_name(model_hint)
# 优先从模型配置读 dimensions
effective_dimensions = dimensions
if effective_dimensions is None:
effective_dimensions = self._resolve_model_dimensions(model_hint)
request = EmbeddingRequest(
input=input if isinstance(input, list) else [input],
model=model_name,
dimensions=effective_dimensions,
)
return await self.adapter.embed(request)
# ── 解析 ───────────────────────────────────────────────────────
def _resolve(self) -> EmbeddingProtocol:
"""查找第一个活跃的 Embedding 模型并通过工厂创建。"""
models = self._list_capability_models("embedding")
if not models:
raise AIProviderError(
status_code=503,
code="embedding_no_model",
message="No embedding model configured in the model registry.",
)
factory = self._factory
if factory is None:
raise AIProviderError(
status_code=501,
code="embedding_no_factory",
message="Embedding adapter factory not available (provider manager missing create_embedding).",
)
return factory.create_embedding(models[0].name)
def _resolve_model_name(self, model_hint: str | None) -> str:
models = self._list_capability_models("embedding")
if model_hint:
return model_hint
if models:
return models[0].name
return ""
def _resolve_model_dimensions(self, model_hint: str | None) -> int | None:
"""从模型配置读取向量维度。"""
models = self._list_capability_models("embedding")
if model_hint:
for m in models:
if m.name == model_hint:
return m.dimensions
if models:
return models[0].dimensions
return None
def _list_capability_models(self, tag: str):
reg = self._registry
if hasattr(reg, "list_capability_models"):
return reg.list_capability_models(tag)
if hasattr(reg, "list_by_capability"):
return reg.list_by_capability(tag)
return []
# ── 生命周期 ────────────────────────────────────────────────────
async def close(self) -> None:
if self._adapter is not None:
close = getattr(self._adapter, "close", None)
if close is not None:
await close()
async def __aenter__(self) -> EmbeddingService:
return self
async def __aexit__(self, *exc_info: object) -> None:
await self.close()
__all__ = [
"EmbeddingProtocol",
"EmbeddingRequest",
"EmbeddingResponse",
"EmbeddingResult",
"EmbeddingService",
"EmbeddingServiceError",
]
-144
View File
@@ -1,144 +0,0 @@
"""嵌入服务类型 — 请求/响应模型和协议定义。
Embedding 适配器通过 HTTP 调用嵌入端点(通常是 OpenAI 兼容的 ``POST /v1/embeddings``),
经由 ``EmbeddingService`` 或直接从 ``ProviderManager`` 消费。
用法::
service = EmbeddingService(manager)
result = await service.embed("要嵌入的文本")
# 或通过工厂直接调用:
adapter = manager.create_embedding("text-embedding-3-small")
result = await adapter.embed(EmbeddingRequest(input=["文本"]))
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Protocol, runtime_checkable
from app.ai_provider.types.errors import AIProviderError
if TYPE_CHECKING:
from app.ai_provider.config.models import ModelConfig, ModelEndpointConfig
class EmbeddingServiceError(AIProviderError):
"""Embedding API 调用失败的基础错误。"""
# ── 请求 / 响应 ────────────────────────────────────────────────────────
@dataclass
class EmbeddingRequest:
"""将文本转换为向量的请求。
属性:
input: 待嵌入的文本字符串或字符串列表。
model: 模型标识符(如 ``"text-embedding-3-small"``)。
encoding_format: 返回编码格式(``"float"`` 或 ``"base64"``)。
dimensions: 输出向量的维度(仅部分模型支持)。
user: 终端用户标识(用于用量追踪)。
"""
input: str | list[str]
model: str = ""
encoding_format: str = "float"
dimensions: int | None = None
user: str | None = None
@dataclass
class EmbeddingResult:
"""单条文本的嵌入结果。"""
embedding: list[float]
index: int = 0
@dataclass
class EmbeddingResponse:
"""嵌入 API 的完整响应。"""
data: list[EmbeddingResult]
model: str = ""
usage: dict[str, int] | None = None
# ── Protocol ────────────────────────────────────────────────────────────
@runtime_checkable
class EmbeddingProtocol(Protocol):
"""Embedding 适配器的协议。"""
async def embed(self, request: EmbeddingRequest) -> EmbeddingResponse:
"""将文本转换为向量。
异常:
EmbeddingServiceError: API 调用失败或响应异常时抛出。
"""
...
async def close(self) -> None:
"""释放适配器持有的 HTTP 连接。"""
...
# ── 共享工厂基类 ──────────────────────────────────────────────────────
class EmbeddingAdapterBase:
"""Embedding 适配器基类,提供共享的工厂方法。
子类应在 ``__init__`` 中接受 ``base_url``、``api_key``、``model``、
``request_timeout_seconds`` 参数。
"""
_adapter_type: str = "embedding"
_default_timeout: int = 60
@classmethod
def from_endpoint(
cls,
endpoint: ModelEndpointConfig,
*,
timeout_seconds: int | None = None,
) -> Any:
"""从已解析的端点配置构建适配器。"""
base_url = endpoint.base_url
api_key = endpoint.api_key
if not base_url:
raise EmbeddingServiceError(
status_code=500,
code="embedding_endpoint_invalid",
message="Embedding endpoint has no resolved base_url.",
details={"endpoint": endpoint.name},
)
return cls(
base_url=base_url,
api_key=api_key,
model=endpoint.model,
request_timeout_seconds=timeout_seconds
or endpoint.request_timeout_seconds
or cls._default_timeout,
)
@classmethod
def from_model_config(
cls,
config: ModelConfig,
*,
timeout_seconds: int | None = None,
) -> Any:
"""从逻辑模型配置构建适配器(使用第一个活跃端点)。"""
for ep in config.endpoints:
if ep.status == "active":
return cls.from_endpoint(ep, timeout_seconds=timeout_seconds)
raise EmbeddingServiceError(
status_code=500,
code="embedding_no_endpoint",
message=f"Embedding model '{config.name}' has no active endpoint.",
)
@@ -1,169 +0,0 @@
"""OpenAI 兼容格式的 Embedding 适配器 — ``POST /v1/embeddings``。
覆盖 OpenAI、DeepSeek、以及任何兼容 OpenAI 嵌入端点的提供商。
请求::
POST {base_url}/v1/embeddings
Authorization: Bearer {api_key}
Content-Type: application/json
{
"model": "text-embedding-3-small",
"input": ["要嵌入的文本"],
"encoding_format": "float"
}
响应::
{
"object": "list",
"data": [
{"object": "embedding", "index": 0, "embedding": [0.xxx, ...]}
],
"model": "text-embedding-3-small",
"usage": {"prompt_tokens": 5, "total_tokens": 5}
}
"""
from __future__ import annotations
from typing import Any
import httpx
from app.ai_provider.embedding.base import (
EmbeddingAdapterBase,
EmbeddingProtocol,
EmbeddingRequest,
EmbeddingResponse,
EmbeddingResult,
EmbeddingServiceError,
)
from app.core.logging import logger
class OpenAIFormatEmbeddingAdapter(EmbeddingAdapterBase, EmbeddingProtocol):
"""OpenAI 兼容嵌入适配器,覆盖 95%+ 的嵌入服务提供商。"""
_adapter_type = "embedding"
_default_timeout = 60
def __init__(
self,
*,
base_url: str,
api_key: str,
model: str = "text-embedding-3-small",
request_timeout_seconds: int = 60,
) -> None:
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._model = model
self._timeout = request_timeout_seconds
self._client: httpx.AsyncClient | None = None
# ── 客户端 ───────────────────────────────────────────────────────────
@property
def _http(self) -> httpx.AsyncClient:
if self._client is None or self._client.is_closed:
self._client = httpx.AsyncClient(timeout=httpx.Timeout(self._timeout))
return self._client
async def close(self) -> None:
if self._client is not None and not self._client.is_closed:
await self._client.aclose()
# ── EmbeddingProtocol ────────────────────────────────────────────────
async def embed(self, request: EmbeddingRequest) -> EmbeddingResponse:
"""通过 ``POST /v1/embeddings`` 将文本转换为向量。"""
url = f"{self._base_url}/v1/embeddings"
payload: dict[str, Any] = {
"model": request.model or self._model,
"input": request.input,
"encoding_format": request.encoding_format,
}
if request.dimensions is not None:
payload["dimensions"] = request.dimensions
if request.user is not None:
payload["user"] = request.user
headers = {
"Content-Type": "application/json",
}
if self._api_key:
headers["Authorization"] = f"Bearer {self._api_key}"
logger.info(
"embedding.request",
model=payload["model"],
input_count=len(request.input) if isinstance(request.input, list) else 1,
dimensions=request.dimensions,
url=url,
)
try:
response = await self._http.post(url, json=payload, headers=headers)
except httpx.TimeoutException as exc:
raise EmbeddingServiceError(
status_code=504,
code="embedding_timeout",
message="Embedding request timed out.",
) from exc
except httpx.RequestError as exc:
raise EmbeddingServiceError(
status_code=503,
code="embedding_connection_error",
message=f"Embedding connection error: {exc}",
) from exc
if response.status_code != 200:
body = response.text[:500]
raise EmbeddingServiceError(
status_code=response.status_code,
code="embedding_api_error",
message=f"Embedding API returned {response.status_code}: {body}",
)
result = self._parse_response(response.json())
logger.info(
"embedding.success",
model=payload["model"],
total_vectors=len(result.data),
usage=result.usage,
)
return result
# ── 辅助方法 ──────────────────────────────────────────────────────────
@staticmethod
def _parse_response(body: dict) -> EmbeddingResponse:
"""从 OpenAI 格式的嵌入响应中提取结果。"""
try:
raw_data = body.get("data", [])
results = []
for item in raw_data:
results.append(
EmbeddingResult(
embedding=item.get("embedding", []),
index=item.get("index", 0),
)
)
# 按 index 排序确保顺序
results.sort(key=lambda r: r.index)
return EmbeddingResponse(
data=results,
model=body.get("model", ""),
usage=body.get("usage"),
)
except Exception as exc:
raise EmbeddingServiceError(
status_code=500,
code="embedding_response_parse_error",
message=f"Failed to parse embedding response: {exc}",
) from exc
-18
View File
@@ -1,18 +0,0 @@
from __future__ import annotations
from app.ai_provider.llm.anthropic_format import (
AnthropicFormatAdapter,
AnthropicFormatAdapterFactory,
)
from app.ai_provider.llm.openai_format import OpenAIFormatAdapter, OpenAIFormatAdapterFactory
from app.ai_provider.llm.stream import StreamFactory, StreamHandler, ToolCallAccumulator
__all__ = [
"AnthropicFormatAdapter",
"AnthropicFormatAdapterFactory",
"OpenAIFormatAdapter",
"OpenAIFormatAdapterFactory",
"StreamFactory",
"StreamHandler",
"ToolCallAccumulator",
]
@@ -1,433 +0,0 @@
from __future__ import annotations
import json
from collections.abc import AsyncIterator
from typing import Any
import httpx
from app.ai_provider.config.models import ModelConfig, ProviderConfig
from app.ai_provider.pipeline import ParameterPipeline, PipelineContext, PipelineFactory
from app.ai_provider.types.chat import ChatChunk, ChatResult, TokenUsage
from app.ai_provider.types.errors import AIProviderError
from app.ai_provider.types.messages import (
AIMessage,
BaseMessage,
SystemMessage,
ToolMessage,
)
from app.core.logging import logger
class AnthropicFormatAdapter:
"""兼容 Anthropic Messages API 格式的适配器。
覆盖通过 Anthropic API 访问的 Claude 模型,以及任何镜像
Anthropic 消息格式的提供商。
"""
API_VERSION = "2023-06-01"
def __init__(
self,
provider_config: ProviderConfig,
pipeline: ParameterPipeline | None = None,
) -> None:
self._provider = provider_config
self._pipeline = pipeline or PipelineFactory.create(provider_config)
self._client: httpx.AsyncClient | None = None
@property
def _http(self) -> httpx.AsyncClient:
if self._client is None:
self._client = httpx.AsyncClient(
base_url=self._provider.base_url or "",
timeout=self._provider.timeout_seconds,
)
return self._client
# ------------------------------------------------------------------
# 内部方法:消息转换
# ------------------------------------------------------------------
def _convert_message(self, msg: BaseMessage) -> dict[str, Any]:
"""将单条消息转换为 Anthropic 消息格式。
Anthropic 使用 content blocks 数组而非纯字符串。
System 消息**不在此处处理** —— 它们由 :meth:`build_request`
提取到顶层的 ``system`` 字段中。
"""
if isinstance(msg, ToolMessage):
return {
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": msg.tool_call_id,
"content": msg.content,
}
],
}
if isinstance(msg, AIMessage):
if msg.raw_content_blocks:
return {"role": "assistant", "content": msg.raw_content_blocks}
blocks: list[dict[str, Any]] = []
if msg.content:
blocks.append({"type": "text", "text": msg.content})
for tc in msg.tool_calls:
blocks.append(
{
"type": "tool_use",
"id": tc.get("id", f"tu_{tc['name']}"),
"name": tc["name"],
"input": tc.get("args", {}),
}
)
return {"role": "assistant", "content": blocks}
# HumanMessage
return {
"role": "user",
"content": [{"type": "text", "text": msg.content}],
}
@staticmethod
def _convert_tools(tools: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""将 OpenAI 兼容格式的工具转换为 Anthropic 格式。
OpenAI 格式::
{"type": "function", "function": {"name": ..., "parameters": ...}}
Anthropic 格式::
{"name": ..., "input_schema": ...}
"""
result: list[dict[str, Any]] = []
for tool in tools:
fn = tool.get("function", tool)
result.append(
{
"name": fn.get("name", ""),
"description": fn.get("description", ""),
"input_schema": fn.get("parameters", {"type": "object"}),
}
)
return result
# ------------------------------------------------------------------
# 请求构建
# ------------------------------------------------------------------
def build_request(
self,
model: ModelConfig,
messages: list[BaseMessage],
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
stream: bool = False,
) -> dict[str, Any]:
ctx = PipelineContext(
params=params or {},
model_config=model,
provider_config=self._provider,
)
pipe_result = self._pipeline.run(ctx)
# Extract system messages — Anthropic uses a top-level "system" field
system_parts: list[str] = []
non_system: list[BaseMessage] = []
for m in messages:
if isinstance(m, SystemMessage):
system_parts.append(m.content)
else:
non_system.append(m)
body: dict[str, Any] = {
"model": model.model,
"messages": [self._convert_message(m) for m in non_system],
"stream": stream,
}
if system_parts:
body["system"] = "\n".join(system_parts)
body.update(pipe_result.params)
if tools:
body["tools"] = self._convert_tools(tools)
if tool_choice:
body["tool_choice"] = tool_choice
# Anthropic requires explicit max_tokens
if "max_tokens" not in body:
body["max_tokens"] = model.max_output_tokens or 4096
return body
def bind_tools(
self,
request_body: dict[str, Any],
tools: list[dict[str, Any]],
tool_choice: str | dict[str, Any] | None = None,
) -> None:
request_body["tools"] = self._convert_tools(tools)
if tool_choice:
request_body["tool_choice"] = tool_choice
# ------------------------------------------------------------------
# 认证头
# ------------------------------------------------------------------
def _headers(self, api_key: str) -> dict[str, str]:
return {
"x-api-key": api_key,
"anthropic-version": self.API_VERSION,
"Content-Type": "application/json",
}
# ------------------------------------------------------------------
# 响应解析 —— 非流式
# ------------------------------------------------------------------
@staticmethod
def _parse_content_blocks(raw: dict[str, Any]) -> tuple[str, list[dict[str, Any]], str]:
"""解析 Anthropic content blocks 为 (text, tool_calls, reasoning)。"""
content_blocks = raw.get("content", [])
text_parts: list[str] = []
tool_calls: list[dict[str, Any]] = []
reasoning = ""
for block in content_blocks:
block_type = block.get("type", "")
if block_type == "text":
text_parts.append(block.get("text", ""))
elif block_type == "tool_use":
tool_calls.append(
{
"name": block.get("name", ""),
"args": block.get("input", {}),
"id": block.get("id", ""),
}
)
elif block_type == "thinking":
reasoning += block.get("thinking", "")
# signature 块是 thinking 的加密签名,不是 reasoning 内容,忽略
return "".join(text_parts), tool_calls, reasoning
def parse_nonstreaming_response(
self,
raw: dict[str, Any],
model_name: str,
) -> ChatResult:
content, tool_calls, reasoning = self._parse_content_blocks(raw)
result = ChatResult(
content=content,
model_name=model_name,
)
if tool_calls:
result.tool_calls = tool_calls
if reasoning:
result.reasoning = reasoning
usage = raw.get("usage")
if usage:
result.usage = TokenUsage(
prompt_tokens=usage.get("input_tokens", 0),
prompt_tokens_cached=usage.get("cache_read_input_tokens", 0),
completion_tokens=usage.get("output_tokens", 0),
total_tokens=usage.get("input_tokens", 0) + usage.get("output_tokens", 0),
)
return result
# ------------------------------------------------------------------
# 响应解析 —— 流式(SSE 事件行协议)
# ------------------------------------------------------------------
@staticmethod
def _parse_stream_event(event_type: str, data: dict[str, Any]) -> ChatChunk:
"""解析单条 SSE 流事件为 ChatChunk。"""
chunk = ChatChunk()
if event_type == "message_start":
message = data.get("message", {})
usage = message.get("usage")
if usage:
chunk.usage = TokenUsage(
prompt_tokens=usage.get("input_tokens", 0),
prompt_tokens_cached=usage.get("cache_read_input_tokens", 0),
completion_tokens=usage.get("output_tokens", 0),
)
elif event_type == "content_block_start":
block = data.get("content_block", {})
chunk.content_block_index = data.get("index", 0)
chunk.content_block_start = block
block_type = block.get("type", "")
if block_type == "text":
chunk.delta_content = block.get("text", "")
elif block_type == "thinking":
chunk.delta_reasoning = block.get("thinking", "")
elif block_type == "tool_use":
chunk.tool_call_chunks.append(
{
"index": data.get("index", 0),
"id": block.get("id"),
"name": block.get("name"),
"args": "",
}
)
elif event_type == "content_block_delta":
delta = data.get("delta", {})
chunk.content_block_index = data.get("index", 0)
chunk.content_block_delta = delta
delta_type = delta.get("type", "")
if delta_type == "text_delta":
chunk.delta_content = delta.get("text", "")
elif delta_type == "thinking_delta":
chunk.delta_reasoning = delta.get("thinking", "")
elif delta_type == "signature_delta":
# signature 不展示,但由 Agent Runtime 原样保留供下一轮回放。
pass
elif delta_type == "input_json_delta":
chunk.tool_call_chunks.append(
{
"index": data.get("index", 0),
"id": None,
"name": None,
"args": delta.get("partial_json", ""),
}
)
elif event_type == "message_delta":
delta = data.get("delta", {})
finish_reason_map = {
"end_turn": "stop",
"max_tokens": "length",
"stop_sequence": "stop",
"tool_use": "tool_calls",
}
stop_reason = delta.get("stop_reason")
if stop_reason:
chunk.finish_reason = finish_reason_map.get(stop_reason, stop_reason)
usage = data.get("usage")
if usage:
chunk.usage = TokenUsage(
prompt_tokens=0,
completion_tokens=usage.get("output_tokens", 0),
)
# content_block_stop, message_stop, ping — no content to extract
return chunk
# ------------------------------------------------------------------
# 公开 API:chat
# ------------------------------------------------------------------
async def chat(
self,
model: ModelConfig,
messages: list[BaseMessage],
api_key: str,
base_url: str | None = None,
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
) -> ChatResult:
url = f"{base_url or self._provider.base_url}/v1/messages"
body = self.build_request(
model,
messages,
params=params,
tools=tools,
tool_choice=tool_choice,
)
try:
response = await self._http.post(url, json=body, headers=self._headers(api_key))
response.raise_for_status()
data = response.json()
return self.parse_nonstreaming_response(data, model.name)
except httpx.HTTPStatusError as exc:
raise AIProviderError.from_provider(
message=f"Anthropic API error: {exc.response.status_code} {exc.response.text[:200]}",
code="PROVIDER_ERROR",
provider=self._provider.name,
original=exc,
) from exc
async def chat_stream(
self,
model: ModelConfig,
messages: list[BaseMessage],
api_key: str,
base_url: str | None = None,
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
) -> AsyncIterator[ChatChunk]:
url = f"{base_url or self._provider.base_url}/v1/messages"
body = self.build_request(
model,
messages,
params=params,
tools=tools,
tool_choice=tool_choice,
stream=True,
)
async with self._http.stream(
"POST",
url,
json=body,
headers=self._headers(api_key),
) as response:
if response.status_code != 200:
error_text = await response.aread()
raise AIProviderError.from_provider(
message=f"Anthropic stream error: {response.status_code} {error_text[:200]}",
code="PROVIDER_ERROR",
provider=self._provider.name,
)
current_event = ""
async for line in response.aiter_lines():
line = line.strip()
if not line:
continue
if line.startswith("event: "):
current_event = line[7:]
elif line.startswith("data: "):
data_str = line[6:]
if data_str and current_event:
try:
raw = json.loads(data_str)
yield self._parse_stream_event(current_event, raw)
except json.JSONDecodeError:
logger.warning(
"anthropic_stream_parse_error",
event=current_event,
raw=data_str[:100],
)
current_event = ""
async def close(self) -> None:
if self._client:
await self._client.aclose()
self._client = None
class AnthropicFormatAdapterFactory:
"""工厂:创建 Anthropic 格式适配器。"""
@staticmethod
def create(provider_config: ProviderConfig) -> AnthropicFormatAdapter:
return AnthropicFormatAdapter(provider_config=provider_config)
@@ -1,251 +0,0 @@
from __future__ import annotations
import json
from collections.abc import AsyncIterator
from typing import Any
import httpx
from app.ai_provider.config.models import ModelConfig, ProviderConfig
from app.ai_provider.pipeline import ParameterPipeline, PipelineContext, PipelineFactory
from app.ai_provider.types.chat import ChatChunk, ChatResult, TokenUsage
from app.ai_provider.types.errors import AIProviderError
from app.ai_provider.types.messages import AIMessage, BaseMessage, ToolMessage
from app.core.logging import logger
class OpenAIFormatAdapter:
"""兼容 OpenAI API 格式的适配器(覆盖 deepseek、xiaomi 等)。"""
def __init__(
self,
provider_config: ProviderConfig,
pipeline: ParameterPipeline | None = None,
) -> None:
self._provider = provider_config
self._pipeline = pipeline or PipelineFactory.create(provider_config)
self._client: httpx.AsyncClient | None = None
@property
def _http(self) -> httpx.AsyncClient:
if self._client is None:
self._client = httpx.AsyncClient(
base_url=self._provider.base_url or "",
timeout=self._provider.timeout_seconds,
)
return self._client
def _convert_message(self, msg: BaseMessage) -> dict[str, Any]:
if isinstance(msg, ToolMessage):
return {
"role": "tool",
"content": msg.content,
"tool_call_id": msg.tool_call_id,
}
if isinstance(msg, AIMessage):
d: dict[str, Any] = {"role": "assistant", "content": msg.content}
if msg.tool_calls:
d["tool_calls"] = [
{
"id": tc.get("id", f"call_{i}"),
"type": "function",
"function": {
"name": tc["name"],
"arguments": json.dumps(tc.get("args", {})),
},
}
for i, tc in enumerate(msg.tool_calls)
]
return d
return {"role": msg.role, "content": msg.content}
def build_request(
self,
model: ModelConfig,
messages: list[BaseMessage],
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
stream: bool = False,
) -> dict[str, Any]:
ctx = PipelineContext(
params=params or {},
model_config=model,
provider_config=self._provider,
)
pipe_result = self._pipeline.run(ctx)
body: dict[str, Any] = {
"model": model.model,
"messages": [self._convert_message(m) for m in messages],
"stream": stream,
}
body.update(pipe_result.params)
if tools:
body["tools"] = tools
if tool_choice:
body["tool_choice"] = tool_choice
return body
def bind_tools(
self,
request_body: dict[str, Any],
tools: list[dict[str, Any]],
tool_choice: str | dict[str, Any] | None = None,
) -> None:
request_body["tools"] = tools
if tool_choice:
request_body["tool_choice"] = tool_choice
def parse_chunk(self, raw: dict[str, Any]) -> ChatChunk:
chunk = ChatChunk()
choices = raw.get("choices", [])
if choices:
delta = choices[0].get("delta", {})
chunk.delta_content = delta.get("content") or ""
chunk.delta_reasoning = delta.get("reasoning_content") or ""
chunk.finish_reason = choices[0].get("finish_reason")
if delta.get("tool_calls"):
for tc in delta["tool_calls"]:
fn = tc.get("function", {})
chunk.tool_call_chunks.append(
{
"index": tc.get("index", 0),
"id": tc.get("id"),
"name": fn.get("name"),
"args": fn.get("arguments", ""),
}
)
usage = raw.get("usage")
if usage:
chunk.usage = TokenUsage(
prompt_tokens=usage.get("prompt_tokens", 0),
completion_tokens=usage.get("completion_tokens", 0),
total_tokens=usage.get("total_tokens", 0),
prompt_tokens_cached=usage.get("prompt_tokens_cached", 0),
)
return chunk
def parse_nonstreaming_response(
self,
raw: dict[str, Any],
model_name: str,
) -> ChatResult:
choice = raw.get("choices", [{}])[0]
message = choice.get("message", {})
content = message.get("content", "")
tool_calls_raw = message.get("tool_calls", [])
result = ChatResult(
content=content,
reasoning=message.get("reasoning_content", ""),
model_name=model_name,
)
if tool_calls_raw:
result.tool_calls = [
{
"name": tc["function"]["name"],
"args": json.loads(tc["function"].get("arguments", "{}")),
"id": tc["id"],
}
for tc in tool_calls_raw
]
usage = raw.get("usage")
if usage:
result.usage = TokenUsage(
prompt_tokens=usage.get("prompt_tokens", 0),
completion_tokens=usage.get("completion_tokens", 0),
total_tokens=usage.get("total_tokens", 0),
prompt_tokens_cached=usage.get("prompt_tokens_cached", 0),
)
return result
async def chat(
self,
model: ModelConfig,
messages: list[BaseMessage],
api_key: str,
base_url: str | None = None,
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
) -> ChatResult:
url = f"{base_url or self._provider.base_url}/chat/completions"
body = self.build_request(
model, messages, params=params, tools=tools, tool_choice=tool_choice
)
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
}
try:
response = await self._http.post(url, json=body, headers=headers)
response.raise_for_status()
data = response.json()
return self.parse_nonstreaming_response(data, model.name)
except httpx.HTTPStatusError as exc:
raise AIProviderError.from_provider(
message=f"OpenAI API error: {exc.response.status_code} {exc.response.text[:200]}",
code="PROVIDER_ERROR",
provider=self._provider.name,
original=exc,
) from exc
async def chat_stream(
self,
model: ModelConfig,
messages: list[BaseMessage],
api_key: str,
base_url: str | None = None,
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
) -> AsyncIterator[ChatChunk]:
url = f"{base_url or self._provider.base_url}/chat/completions"
body = self.build_request(
model, messages, params=params, tools=tools, tool_choice=tool_choice, stream=True
)
headers = {
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
}
async with self._http.stream("POST", url, json=body, headers=headers) as response:
if response.status_code != 200:
error_text = await response.aread()
raise AIProviderError.from_provider(
message=f"OpenAI stream error: {response.status_code} {error_text[:200]}",
code="PROVIDER_ERROR",
provider=self._provider.name,
)
async for line in response.aiter_lines():
if line.startswith("data: "):
data_str = line[6:].strip()
if data_str == "[DONE]":
break
if data_str:
try:
raw = json.loads(data_str)
yield self.parse_chunk(raw)
except json.JSONDecodeError:
logger.warning("openai_stream_parse_error", raw=data_str[:100])
async def close(self) -> None:
if self._client:
await self._client.aclose()
self._client = None
class OpenAIFormatAdapterFactory:
"""工厂:创建 OpenAI 格式适配器。"""
@staticmethod
def create(provider_config: ProviderConfig) -> OpenAIFormatAdapter:
return OpenAIFormatAdapter(provider_config=provider_config)
-164
View File
@@ -1,164 +0,0 @@
from __future__ import annotations
import random
from collections.abc import Callable
from dataclasses import dataclass, field
from typing import Any, Protocol
@dataclass
class PoolContext:
request_id: str = ""
excluded_endpoints: set[str] = field(default_factory=set)
class SchedulingStrategy(Protocol):
def select(self, excluded: set[str]) -> dict[str, Any] | None: ...
class WeightedRoundRobinStrategy:
"""策略 1:在所有活跃端点间加权轮询。"""
def __init__(self, endpoints: list[dict[str, Any]]) -> None:
self._endpoints = endpoints
self._index = 0
def select(self, excluded: set[str]) -> dict[str, Any] | None:
available = [
e
for e in self._endpoints
if e.get("name") not in excluded and e.get("status", "active") == "active"
]
if not available:
return None
weights = [e.get("weight", 100) for e in available]
self._index = (self._index + 1) % len(available)
return random.choices(available, weights=weights, k=1)[0]
class PriorityFailoverStrategy:
"""策略 2:选择最高优先级;当排除时顺延。"""
def __init__(self, endpoints: list[dict[str, Any]]) -> None:
self._endpoints = sorted(endpoints, key=lambda e: e.get("priority", 100))
def select(self, excluded: set[str]) -> dict[str, Any] | None:
for ep in self._endpoints:
if ep.get("name") not in excluded and ep.get("status", "active") == "active":
return ep
return None
class LowestLatencyStrategy:
"""策略 3:选择最近延迟最低的端点。"""
def __init__(self, endpoints: list[dict[str, Any]]) -> None:
self._endpoints = endpoints
def select(self, excluded: set[str]) -> dict[str, Any] | None:
available = [
e
for e in self._endpoints
if e.get("name") not in excluded and e.get("status", "active") == "active"
]
if not available:
return None
return min(available, key=lambda e: e.get("latency_ms", 1000))
class LeastConnectionsStrategy:
"""策略 4:选择活跃连接数最少的端点。"""
def __init__(self, endpoints: list[dict[str, Any]]) -> None:
self._endpoints = endpoints
def select(self, excluded: set[str]) -> dict[str, Any] | None:
available = [
e
for e in self._endpoints
if e.get("name") not in excluded and e.get("status", "active") == "active"
]
if not available:
return None
return min(available, key=lambda e: e.get("connections", 0))
class CostAwareStrategy:
"""策略 5:按每 token 成本选择最便宜的端点。"""
def __init__(
self, endpoints: list[dict[str, Any]], get_cost: Callable[[str], float] | None = None
) -> None:
self._endpoints = endpoints
self._get_cost = get_cost or (lambda _: 0.0)
def select(self, excluded: set[str]) -> dict[str, Any] | None:
available = [
e
for e in self._endpoints
if e.get("name") not in excluded and e.get("status", "active") == "active"
]
if not available:
return None
return min(available, key=lambda e: self._get_cost(e.get("name", "")))
_STRATEGIES = {
"weighted_round_robin": WeightedRoundRobinStrategy,
"priority_failover": PriorityFailoverStrategy,
"lowest_latency": LowestLatencyStrategy,
"least_connections": LeastConnectionsStrategy,
"cost_aware": CostAwareStrategy,
}
class EndpointPool:
"""模型池,带可插拔的调度策略。"""
def __init__(
self,
logical_model_name: str,
endpoints_configs: list[dict[str, Any]],
strategy: str = "weighted_round_robin",
) -> None:
self.logical_model_name = logical_model_name
self._endpoints = endpoints_configs
strategy_cls = _STRATEGIES.get(strategy, WeightedRoundRobinStrategy)
self.strategy = (
strategy_cls(endpoints_configs)
if strategy == "priority_failover"
else strategy_cls(endpoints_configs)
)
def select(self, excluded: set[str]) -> dict[str, Any] | None:
"""使用配置的策略选择一个端点。"""
return self.strategy.select(excluded)
def list_endpoints(self) -> list[dict[str, Any]]:
return list(self._endpoints)
class PoolFactory:
"""工厂:创建端点池。"""
@staticmethod
def create_pool(
logical_model_name: str,
endpoints: list[dict[str, Any]],
strategy_name: str = "weighted_round_robin",
) -> EndpointPool:
return EndpointPool(
logical_model_name=logical_model_name,
endpoints_configs=endpoints,
strategy=strategy_name,
)
@staticmethod
def create_strategy(
name: str,
endpoints: list[dict[str, Any]],
) -> SchedulingStrategy:
strategy_cls = _STRATEGIES.get(name)
if strategy_cls is None:
raise ValueError(f"Unknown scheduling strategy: {name}")
return strategy_cls(endpoints)
-115
View File
@@ -1,115 +0,0 @@
from __future__ import annotations
import json
from collections import defaultdict
from typing import Any
from app.ai_provider.types.chat import ChatChunk, ChatResult, TokenUsage
class ToolCallAccumulator:
"""将流式 tool call 分片累积为完整的 tool call。
处理流式场景中常见的多分片 tool call 参数,
按索引拼接部分 JSON 字符串。
"""
def __init__(self) -> None:
self.content: str = ""
self.reasoning: str = ""
self._tool_call_slots: dict[int, dict[str, Any]] = defaultdict(
lambda: {"index": 0, "id": "", "name": "", "args": ""}
)
self.usage: TokenUsage | None = None
def add_chunk(self, chunk: ChatChunk) -> None:
"""处理一个 ChatChunk,累积 content/reasoning/tool-calls/usage。"""
if chunk.delta_content:
self.content += chunk.delta_content
if chunk.delta_reasoning:
self.reasoning += chunk.delta_reasoning
if chunk.usage is not None:
self.usage = chunk.usage
for tc in chunk.tool_call_chunks:
idx = tc.get("index", 0)
if idx not in self._tool_call_slots:
self._tool_call_slots[idx] = {"index": idx, "id": "", "name": "", "args": ""}
slot = self._tool_call_slots[idx]
# The first chunk for a given index carries id/name; subsequent
# arg-only chunks have them as None and should NOT overwrite.
if tc.get("id"):
slot["id"] = tc["id"]
if tc.get("name"):
slot["name"] = tc["name"]
slot["args"] += tc.get("args", "")
def drain(self) -> list[dict[str, Any]]:
"""返回所有已累积的 tool call 并重置内部槽位。"""
result: list[dict[str, Any]] = []
for idx in sorted(self._tool_call_slots.keys()):
slot = self._tool_call_slots[idx]
result.append(
{
"index": slot["index"],
"id": slot["id"] or f"call_{idx}",
"name": slot["name"],
"args": slot["args"],
}
)
self._tool_call_slots.clear()
return result
class StreamHandler:
"""收集流式分片并生成最终的 ChatResult。"""
def __init__(self) -> None:
self._accumulator = ToolCallAccumulator()
self._chunks: list[ChatChunk] = []
def add_chunk(self, chunk: ChatChunk) -> None:
"""收集一个流式分片。"""
self._chunks.append(chunk)
self._accumulator.add_chunk(chunk)
@property
def chunks(self) -> list[ChatChunk]:
"""返回所有已收集分片的副本。"""
return list(self._chunks)
def to_result(self, model_name: str = "") -> ChatResult:
"""从所有累积分片构建最终的 ChatResult。"""
raw_tool_calls = self._accumulator.drain()
parsed_calls: list[dict[str, Any]] = []
for tc in raw_tool_calls:
args_raw = tc.get("args", "{}")
try:
parsed_args = json.loads(args_raw)
except json.JSONDecodeError:
parsed_args = {"_raw": args_raw}
parsed_calls.append(
{
"name": tc["name"],
"args": parsed_args,
"id": tc["id"],
}
)
return ChatResult(
content=self._accumulator.content,
reasoning=self._accumulator.reasoning,
tool_calls=parsed_calls,
usage=self._accumulator.usage,
model_name=model_name,
)
class StreamFactory:
"""流处理器和累积器的工厂。"""
@staticmethod
def create_handler() -> StreamHandler:
return StreamHandler()
@staticmethod
def create_accumulator() -> ToolCallAccumulator:
return ToolCallAccumulator()
-463
View File
@@ -1,463 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
from app.ai_provider.config.loader import (
load_models_config,
load_pricing_config,
load_providers_config,
)
from app.ai_provider.config.models import ModelConfig, ProviderConfig
from app.ai_provider.core.billing import (
BillingFactory,
BillingTracker,
BudgetController,
PricingInfo,
PricingRegistry,
)
from app.ai_provider.core.registry import ModelRegistry, ProviderRegistry
from app.ai_provider.core.resilience import CircuitBreaker
from app.ai_provider.llm.anthropic_format import (
AnthropicFormatAdapter,
AnthropicFormatAdapterFactory,
)
from app.ai_provider.llm.openai_format import OpenAIFormatAdapter, OpenAIFormatAdapterFactory
from app.ai_provider.llm.stream import StreamHandler
from app.ai_provider.routing import (
CapabilityPermissionController,
ModelSelector,
MultiModalRouter,
)
from app.ai_provider.types.chat import ChatResult, TokenUsage
from app.ai_provider.types.errors import AIProviderError
from app.ai_provider.types.messages import (
AIMessage,
BaseMessage,
HumanMessage,
SystemMessage,
)
from app.core.config import settings
@dataclass
class _ResolvedEndpoint:
"""端点到适配器和凭证的解析结果。"""
adapter: OpenAIFormatAdapter | AnthropicFormatAdapter
model: ModelConfig
provider: ProviderConfig
api_key: str
base_url: str | None
breaker: CircuitBreaker | None = None
class ProviderManager:
"""所有 AI 提供商操作的统一入口。
取代旧的 ModelRouter + get_model_router() 模式。
组合了注册表、适配器、池、计费、弹性和路由。
"""
def __init__(
self,
model_registry: ModelRegistry,
provider_registry: ProviderRegistry,
billing_tracker: BillingTracker | None = None,
budget_controller: BudgetController | None = None,
pricing_registry: PricingRegistry | None = None,
) -> None:
self.model_registry = model_registry
self.provider_registry = provider_registry
self._billing = billing_tracker or BillingFactory.create_tracker()
self._budget = budget_controller
self._pricing_registry = pricing_registry or PricingRegistry()
self._adapter_cache: dict[str, OpenAIFormatAdapter | AnthropicFormatAdapter] = {}
self._model_selector = ModelSelector(model_registry)
self._multi_modal_router = MultiModalRouter()
self._permission_controller = CapabilityPermissionController()
# ── 模型解析 ──────────────────────────────────────────────
def resolve_model(self, model_hint: str | None = None) -> str:
if model_hint is None:
return self.model_registry.get_default_name()
return self.model_registry.resolve_name(model_hint)
def get_model_config(self, name: str) -> ModelConfig:
return self.model_registry.get(name)
def get_provider_config(self, name: str) -> ProviderConfig:
return self.provider_registry.get(name)
# ── 适配器解析 ────────────────────────────────────────────
def get_adapter(self, model_name: str) -> OpenAIFormatAdapter | AnthropicFormatAdapter:
if model_name in self._adapter_cache:
return self._adapter_cache[model_name]
model = self.model_registry.get(model_name)
provider = self.provider_registry.get(model.provider)
if model.format.value == "openai":
adapter = OpenAIFormatAdapterFactory.create(provider)
else:
adapter = AnthropicFormatAdapterFactory.create(provider)
self._adapter_cache[model_name] = adapter
return adapter
# ── 消息辅助 ───────────────────────────────────────────────
@staticmethod
def to_message_list(
system_messages: list[str],
history: list[tuple[str, str]],
) -> list[BaseMessage]:
messages: list[BaseMessage] = []
for msg in system_messages:
if msg:
messages.append(SystemMessage(content=msg))
for role, content in history:
if role == "system":
messages.append(SystemMessage(content=content))
elif role == "user":
messages.append(HumanMessage(content=content))
elif role == "assistant":
messages.append(AIMessage(content=content))
return messages
# ── Embedding ──────────────────────────────────────────────────────
def create_embedding(self, model_name: str) -> Any:
"""按模型名创建 Embedding 适配器。
优先读取 model endpoint 级 base_url/api_key 覆盖;无 endpoint 时
回退 provider 级凭证。
"""
from app.ai_provider.embedding.openai_format import OpenAIFormatEmbeddingAdapter
model = self.model_registry.get(model_name)
provider = self.provider_registry.get(model.provider)
# 尝试从 model endpoint 读取 base_url / api_key 覆盖
base_url: str | None = None
api_key: str = ""
if model.endpoints and len(model.endpoints) > 0:
ep = model.endpoints[0] # 取第一个活跃 endpoint
if ep.base_url:
base_url = ep.base_url
elif ep.base_url_env and hasattr(settings, ep.base_url_env.lower()):
base_url = getattr(settings, ep.base_url_env.lower())
if ep.api_key:
api_key = ep.api_key
elif ep.api_key_env and hasattr(settings, ep.api_key_env.lower()):
api_key = getattr(settings, ep.api_key_env.lower(), "")
if not base_url:
_, base_url = self._resolve_credentials(provider)
if not base_url:
base_url = provider.base_url
if not api_key:
api_key, _ = self._resolve_credentials(provider)
return OpenAIFormatEmbeddingAdapter(
base_url=base_url or "",
api_key=api_key or "",
model=model.model,
request_timeout_seconds=model.timeout_seconds,
)
# ── Reranker ───────────────────────────────────────────────────────
def create_reranker(self, model_name: str) -> Any:
"""按模型名创建 Reranker 适配器。"""
from app.ai_provider.reranker.cohere_format import CohereFormatRerankerAdapter
model = self.model_registry.get(model_name)
provider = self.provider_registry.get(model.provider)
api_key, base_url = self._resolve_credentials(provider)
if not base_url:
base_url = provider.base_url
return CohereFormatRerankerAdapter(
base_url=base_url or "",
api_key=api_key,
model=model.model,
request_timeout_seconds=model.timeout_seconds,
)
# ── 对话 ──────────────────────────────────────────────────────────
def _resolve_endpoint(self, model_name: str) -> _ResolvedEndpoint:
"""解析模型名→(适配器, 凭证, 熔断器)。
处理池选择、熔断初始化、凭证回退。
"""
model = self.model_registry.get(model_name)
provider = self.provider_registry.get(model.provider)
adapter = self.get_adapter(model_name)
breaker: CircuitBreaker | None = None
if model.endpoint_pool_enabled and model.endpoints:
from app.ai_provider.llm.pool import PoolFactory
pool = PoolFactory.create_pool(
logical_model_name=model_name,
endpoints=[ep.model_dump(exclude_none=True) for ep in model.endpoints],
strategy_name=model.routing_strategy,
)
selected = pool.select(set())
if selected is not None:
api_key = selected.get("api_key") or (
settings.__getattribute__(provider.api_key_env.lower())
if hasattr(settings, provider.api_key_env.lower())
else ""
)
base_url_env = selected.get("base_url_env") or provider.base_url_env
base_url = selected.get("base_url") or (
settings.__getattribute__(base_url_env.lower())
if hasattr(settings, base_url_env.lower())
else None
)
# 熔断器
from app.ai_provider.core.resilience import ResilienceFactory
breaker = ResilienceFactory.create_circuit_breaker(
name=f"{model_name}@{selected.get('name', 'default')}",
failure_threshold=3,
cooldown_seconds=30,
)
if not breaker.allow_request():
raise AIProviderError(
message=f"Circuit breaker open for {selected.get('name', 'unknown')}",
code="CIRCUIT_BREAKER_OPEN",
status_code=503,
)
return _ResolvedEndpoint(
adapter=adapter,
model=model,
provider=provider,
api_key=api_key,
base_url=base_url,
breaker=breaker,
)
# 无活跃 endpoint,回退 provider 凭证(无熔断)
api_key, base_url = self._resolve_credentials(provider)
return _ResolvedEndpoint(
adapter=adapter,
model=model,
provider=provider,
api_key=api_key,
base_url=base_url,
)
# 无池,直接走 provider 凭证
api_key, base_url = self._resolve_credentials(provider)
return _ResolvedEndpoint(
adapter=adapter,
model=model,
provider=provider,
api_key=api_key,
base_url=base_url,
)
async def chat(
self,
messages: list[BaseMessage],
model_hint: str | None = None,
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
) -> ChatResult:
model_name = self.resolve_model(model_hint)
if self._budget and not self._budget.allow_request():
raise AIProviderError(
message="Budget limit exceeded", code="BUDGET_EXCEEDED", status_code=429
)
ep = self._resolve_endpoint(model_name)
try:
result = await ep.adapter.chat(
model=ep.model,
messages=messages,
api_key=ep.api_key,
base_url=ep.base_url,
params=params,
tools=tools,
tool_choice=tool_choice,
)
if ep.breaker:
ep.breaker.record_success()
except Exception:
if ep.breaker:
ep.breaker.record_failure()
raise
self._record_usage(
model_name, ep.provider.name, result.usage, self._pricing_registry.get(model_name)
)
return result
async def chat_stream(
self,
messages: list[BaseMessage],
model_hint: str | None = None,
params: dict[str, Any] | None = None,
tools: list[dict[str, Any]] | None = None,
tool_choice: str | dict[str, Any] | None = None,
):
model_name = self.resolve_model(model_hint)
if self._budget and not self._budget.allow_request():
raise AIProviderError(
message="Budget limit exceeded", code="BUDGET_EXCEEDED", status_code=429
)
ep = self._resolve_endpoint(model_name)
try:
stream = ep.adapter.chat_stream(
model=ep.model,
messages=messages,
api_key=ep.api_key,
base_url=ep.base_url,
params=params,
tools=tools,
tool_choice=tool_choice,
)
handler = StreamHandler()
async for chunk in stream:
handler.add_chunk(chunk)
yield chunk
if ep.breaker:
ep.breaker.record_success()
except Exception:
if ep.breaker:
ep.breaker.record_failure()
raise
result = handler.to_result(model_name=model_name)
self._record_usage(
model_name, ep.provider.name, result.usage, self._pricing_registry.get(model_name)
)
def _resolve_credentials(self, provider: ProviderConfig) -> tuple[str, str | None]:
"""从 settings 解析 provider 级别的 API key 和 base URL。"""
api_key = (
settings.__getattribute__(provider.api_key_env.lower())
if hasattr(settings, provider.api_key_env.lower())
else ""
)
base_url = (
settings.__getattribute__(provider.base_url_env.lower())
if hasattr(settings, provider.base_url_env.lower())
else None
)
return api_key, base_url
# ── 路由辅助 ───────────────────────────────────────────────
def select_model_for_context(
self,
needs_vision: bool = False,
needs_thinking: bool = False,
) -> str:
capabilities: set[str] = set()
if needs_vision:
capabilities.add("vision")
if needs_thinking:
capabilities.add("thinking")
model = self._model_selector.select({"capabilities": capabilities})
return model.name
def get_adapter_for_tool_binding(
self,
model_name: str,
tools: list[dict[str, Any]],
tool_choice: str | dict[str, Any] | None = None,
):
"""返回 (adapter, request_body_base) 用于工具绑定。
为 Agent 运行时保留独立的 bind_tools 步骤。
"""
adapter = self.get_adapter(model_name)
body = {"model": self.model_registry.get(model_name).model}
adapter.bind_tools(body, tools, tool_choice=tool_choice)
return adapter, body
# ── 用量与计费 ───────────────────────────────────────────────
def _record_usage(
self,
model_name: str,
provider: str,
usage: TokenUsage | None,
pricing: PricingInfo | None = None,
) -> None:
record = self._billing.record(
model_name=model_name,
provider=provider,
usage=usage,
pricing=pricing,
)
if self._budget and record.cost > 0:
self._budget.add_cost_cny(
record.cost if record.currency == "CNY" else record.cost * 7.3
)
def get_usage_records(self, **kwargs) -> list:
return self._billing.get_records(**kwargs)
def get_budget_status(self) -> dict[str, bool]:
if self._budget:
return {"within_daily": self._budget.allow_request()}
return {"within_daily": True}
# ── 生命周期 ─────────────────────────────────────────────────────
async def close(self) -> None:
for adapter in self._adapter_cache.values():
await adapter.close()
async def __aenter__(self) -> ProviderManager:
return self
async def __aexit__(self, *exc_info: object) -> None:
await self.close()
class ProviderManagerFactory:
"""工厂:从配置文件创建 ProviderManager。"""
@staticmethod
def create(
models_path: str | None = None,
providers_path: str | None = None,
aliases: dict[str, str] | None = None,
daily_budget_cny: float = 0.0,
) -> ProviderManager:
from pathlib import Path
models = load_models_config(Path(models_path) if models_path else None)
providers = load_providers_config(Path(providers_path) if providers_path else None)
pricing_data = load_pricing_config()
from app.ai_provider.core.registry import RegistryFactory
registries = RegistryFactory.create(
models, providers, aliases=aliases, pricing_data=pricing_data
)
billing = BillingFactory.create_tracker()
budget = (
BillingFactory.create_budget_controller(daily_budget_cny=daily_budget_cny)
if daily_budget_cny > 0
else None
)
return ProviderManager(
model_registry=registries.model_registry,
provider_registry=registries.provider_registry,
billing_tracker=billing,
budget_controller=budget,
pricing_registry=registries.pricing_registry,
)
-172
View File
@@ -1,172 +0,0 @@
"""配置驱动的 AI 提供商五阶段参数管道。
根据 ProviderConfig.parameter_mappings 在分发前对参数进行
转换、归一化、过滤和注入。第五阶段 HandlerStep 处理跨参数
依赖等复杂场景。
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
from app.ai_provider.config.models import ModelConfig, ProviderConfig
@dataclass
class PipelineContext:
"""管道的输入上下文。"""
params: dict[str, Any] = field(default_factory=dict)
model_config: ModelConfig | None = None
provider_config: ProviderConfig | None = None
messages: list[dict[str, Any]] | None = None
tools: list[dict[str, Any]] | None = None
@dataclass
class PipelineResult:
"""经过管道处理后的结果。"""
params: dict[str, Any] = field(default_factory=dict)
filtered_params: dict[str, Any] = field(default_factory=dict)
messages: list[dict[str, Any]] | None = None
tools: list[dict[str, Any]] | None = None
class PipelineStep:
"""管道步骤的基类。"""
def apply(self, ctx: PipelineContext, result: PipelineResult) -> None:
raise NotImplementedError
class NormalizeStep(PipelineStep):
"""步骤 1:归一化参数范围(夹紧到 min/max)。"""
def apply(self, ctx: PipelineContext, result: PipelineResult) -> None:
mappings = ctx.provider_config.parameter_mappings if ctx.provider_config else {}
for key, value in ctx.params.items():
mapping = mappings.get(key)
if mapping:
if mapping.min is not None:
value = max(mapping.min, value)
if mapping.max is not None:
value = min(mapping.max, value)
result.params[key] = value
else:
result.params[key] = value
class FilterStep(PipelineStep):
"""步骤 2:过滤模型/提供商不支持的参数。"""
def apply(self, ctx: PipelineContext, result: PipelineResult) -> None:
allowed = (
set(ctx.provider_config.parameter_mappings.keys())
if ctx.provider_config
else set(result.params.keys())
)
filtered = {}
for key in list(result.params.keys()):
if key not in allowed:
filtered[key] = result.params.pop(key)
result.filtered_params = filtered
class TransformStep(PipelineStep):
"""步骤 3:转换参数名称和值为提供商所需格式。"""
def apply(self, ctx: PipelineContext, result: PipelineResult) -> None:
mappings = ctx.provider_config.parameter_mappings if ctx.provider_config else {}
remapped: dict[str, Any] = {}
for key, value in result.params.items():
mapping = mappings.get(key)
if mapping:
provider_key = mapping.provider_param
if mapping.transform == "int":
value = int(value)
elif mapping.transform == "str":
value = str(value)
elif mapping.transform == "multiply_1000":
value = int(value * 1000)
remapped[provider_key] = value
else:
remapped[key] = value
result.params = remapped
class InjectStep(PipelineStep):
"""步骤 4:为缺失参数注入提供商默认值。"""
def apply(self, ctx: PipelineContext, result: PipelineResult) -> None:
mappings = ctx.provider_config.parameter_mappings if ctx.provider_config else {}
for mapping in mappings.values():
provider_key = mapping.provider_param
if provider_key not in result.params and mapping.default is not None:
result.params[provider_key] = mapping.default
class HandlerStep(PipelineStep):
"""步骤 5:执行已注册的参数处理器。
处理 YAML 映射规则无法表达的跨参数依赖和复杂变换:
- Anthropic thinking 参数要求 max_tokens > budget_tokens
- 根据模型能力过滤不支持的参数
- 结构性参数重组
"""
def apply(self, ctx: PipelineContext, result: PipelineResult) -> None:
if not ctx.provider_config:
return
from app.ai_provider.pipeline_handlers import ParameterHandlerRegistry
registry = ParameterHandlerRegistry
# 阶段 A:逐参数处理器(来自 ParameterMapping.handler)
for key, mapping in ctx.provider_config.parameter_mappings.items():
if mapping.handler:
handler = registry.get(mapping.handler)
handler(ctx, result, key) # key is the param key
# 阶段 B:全局处理器(来自 ProviderConfig.parameter_handlers)
for handler_name in ctx.provider_config.parameter_handlers:
handler = registry.get(handler_name)
handler(ctx, result, None) # global handler: no param_key
class ParameterPipeline:
"""配置驱动的五阶段参数管道。"""
def __init__(
self,
steps: list[PipelineStep],
provider_config: ProviderConfig | None = None,
) -> None:
self._steps = steps
self._provider_config = provider_config
def run(self, ctx: PipelineContext) -> PipelineResult:
if ctx.provider_config is None and self._provider_config is not None:
ctx.provider_config = self._provider_config
result = PipelineResult(params=dict(ctx.params))
for step in self._steps:
step.apply(ctx, result)
return result
class PipelineFactory:
"""工厂:从提供商配置构建管道。"""
@staticmethod
def create(provider_config: ProviderConfig) -> ParameterPipeline:
return ParameterPipeline(
provider_config=provider_config,
steps=[
NormalizeStep(),
FilterStep(),
TransformStep(),
InjectStep(),
HandlerStep(),
],
)
@@ -1,153 +0,0 @@
"""参数处理器注册中心与内置 handlers。
处理跨参数依赖、结构性转换等复杂场景——这些无法通过
YAML ParameterMapping 的 min/max/transform/default 表达。
使用方式(在 providers.yaml 中):
```yaml
parameter_mappings:
thinking: { provider_param: thinking, handler: anthropic_thinking }
parameter_handlers:
- strip_unsupported_capability_params
```
"""
from __future__ import annotations
from collections.abc import Callable
from app.ai_provider.pipeline import PipelineContext, PipelineResult
# Type alias for a parameter handler function.
# param_key is passed for per-param handlers (from ParameterMapping.handler);
# for global handlers (from ProviderConfig.parameter_handlers), param_key is None.
ParameterHandler = Callable[[PipelineContext, PipelineResult, str | None], None]
class ParameterHandlerRegistry:
"""全局参数处理器注册中心。
使用 register() 装饰器注册 handler,名称字符串即可在 YAML 中引用。
"""
_handlers: dict[str, ParameterHandler] = {}
@classmethod
def register(cls, name: str) -> Callable[[ParameterHandler], ParameterHandler]:
"""装饰器:将函数注册为命名 handler。"""
def decorator(handler: ParameterHandler) -> ParameterHandler:
cls._handlers[name] = handler
return handler
return decorator
@classmethod
def get(cls, name: str) -> ParameterHandler:
"""按名称获取 handler。未找到时抛出 KeyError。"""
if name not in cls._handlers:
msg = f"Unknown parameter handler: '{name}'. "
msg += f"Available: {', '.join(sorted(cls._handlers))}"
raise KeyError(msg)
return cls._handlers[name]
@classmethod
def list(cls) -> list[str]:
"""返回所有已注册的 handler 名称。"""
return list(cls._handlers.keys())
# ── 内建 Handlers ──────────────────────────────────────────────
@ParameterHandlerRegistry.register("anthropic_thinking")
def handle_anthropic_thinking(
_ctx: PipelineContext,
result: PipelineResult,
_param_key: str | None = None,
) -> None:
"""处理 Anthropic thinking 参数的跨参数约束。
Anthropic API 要求:
- max_tokens 必须大于 thinking.budget_tokens
- 开启 thinking 时 temperature 必须为 1(不应传此参数)
- thinking 参数自身需转换为 {type, budget_tokens} 结构
"""
thinking = result.params.get("thinking")
if thinking is None:
return
# 将标量 thinking=True 转换为启用 extended thinking
if thinking is True:
result.params["thinking"] = {"type": "enabled"}
# 提取 budget_tokens
if isinstance(thinking, dict):
budget = thinking.get("budget_tokens", 0)
# 确保 max_tokens > budget_tokens
max_tokens = result.params.get("max_tokens", 0)
if budget > 0 and max_tokens <= budget:
result.params["max_tokens"] = budget + 1
# 开启 thinking 时移除 temperature 参数(Anthropic 静默忽略,
# 但显式移除更安全干净)
result.params.pop("temperature", None)
@ParameterHandlerRegistry.register("validate_max_tokens")
def validate_max_tokens(
ctx: PipelineContext,
result: PipelineResult,
_param_key: str | None = None,
) -> None:
"""校验 max_tokens 不超过模型上限。"""
model_config = ctx.model_config
if model_config is None:
return
max_tokens = result.params.get("max_tokens")
if max_tokens is None:
return
limit = model_config.max_output_tokens
if limit and max_tokens > limit:
result.params["max_tokens"] = limit
@ParameterHandlerRegistry.register("validate_temperature")
def validate_temperature(
_ctx: PipelineContext,
_result: PipelineResult,
_param_key: str | None = None,
) -> None:
"""温度参数跨提供商规范校验。
非 reasoning/thinking 模式下强制温度 1.0 不是必需的规则,
OpenAI/DeepSeek 无此要求,此 handler 留作扩展点。
"""
# 预留:可为特定 provider 添加温度校验逻辑
pass
@ParameterHandlerRegistry.register("strip_unsupported_capability_params")
def strip_unsupported_capability_params(
ctx: PipelineContext,
result: PipelineResult,
_param_key: str | None = None,
) -> None:
"""根据模型能力过滤不支持的参数。
当模型不支持 vision 时,移除 image_url 等视觉参数。
当模型不支持 tool_use 时,移除 tools 相关参数。
"""
model_config = ctx.model_config
if model_config is None:
return
caps = model_config.resolved_capabilities()
if not caps.get("vision", False):
result.params.pop("image_url", None)
result.params.pop("image", None)
if not caps.get("thinking", False):
result.params.pop("thinking", None)
@@ -1,135 +0,0 @@
"""重排序服务 — Reranker 适配器外观。
用法::
from app.ai_provider.reranker import RerankerService
service = RerankerService(manager)
result = await service.rerank("查询", ["文档1", "文档2"])
"""
from __future__ import annotations
from app.ai_provider.reranker.base import (
RerankerProtocol,
RerankerServiceError,
RerankRequest,
RerankResponse,
RerankResult,
)
from app.ai_provider.types.errors import AIProviderError
class RerankerService:
"""外观类,负责解析 Reranker 模型并委托给 ProviderManager。
接受 ProviderManager 或实现了以下接口的适配器:
- ``.model_registry`` → 具有 ``list_by_capability(capability)`` 方法
- ``.create_reranker(name)`` → 返回 RerankerProtocol 实例
"""
def __init__(self, manager) -> None:
self._manager = manager
self._adapter: RerankerProtocol | None = None
# ── Registry 兼容适配 ─────────────────────────────────────────
@property
def _registry(self):
mgr = self._manager
if hasattr(mgr, "model_registry"):
return mgr.model_registry
return mgr
@property
def _factory(self):
mgr = self._manager
if hasattr(mgr, "create_reranker"):
return mgr
return None
# ── Reranker ───────────────────────────────────────────────────
@property
def adapter(self) -> RerankerProtocol:
"""解析并缓存 Reranker 适配器。"""
if self._adapter is None:
self._adapter = self._resolve()
return self._adapter
async def rerank(
self,
query: str,
documents: list[str],
model_hint: str | None = None,
top_n: int | None = None,
) -> RerankResponse:
"""快捷方法:构造请求并调用 Reranker 适配器。"""
model = self._resolve_model_name(model_hint)
request = RerankRequest(
query=query,
documents=documents,
model=model,
top_n=top_n,
)
return await self.adapter.rerank(request)
# ── 解析 ───────────────────────────────────────────────────────
def _resolve(self) -> RerankerProtocol:
"""查找第一个活跃的 Reranker 模型并通过工厂创建。"""
models = self._list_capability_models("reranker")
if not models:
raise AIProviderError(
status_code=503,
code="reranker_no_model",
message="No reranker model configured in the model registry.",
)
factory = self._factory
if factory is None:
raise AIProviderError(
status_code=501,
code="reranker_no_factory",
message="Reranker adapter factory not available (provider manager missing create_reranker).",
)
return factory.create_reranker(models[0].name)
def _resolve_model_name(self, model_hint: str | None) -> str:
models = self._list_capability_models("reranker")
if model_hint:
return model_hint
if models:
return models[0].name
return ""
def _list_capability_models(self, tag: str):
reg = self._registry
if hasattr(reg, "list_capability_models"):
return reg.list_capability_models(tag)
if hasattr(reg, "list_by_capability"):
return reg.list_by_capability(tag)
return []
# ── 生命周期 ────────────────────────────────────────────────────
async def close(self) -> None:
if self._adapter is not None:
close = getattr(self._adapter, "close", None)
if close is not None:
await close()
async def __aenter__(self) -> RerankerService:
return self
async def __aexit__(self, *exc_info: object) -> None:
await self.close()
__all__ = [
"RerankerProtocol",
"RerankRequest",
"RerankResponse",
"RerankResult",
"RerankerService",
"RerankerServiceError",
]
-147
View File
@@ -1,147 +0,0 @@
"""重排序服务类型 — 请求/响应模型和协议定义。
Reranker 适配器通过 HTTP 调用重排序端点,经由 ``RerankerService``
或直接从 ``ProviderManager`` 消费。
用法::
service = RerankerService(manager)
result = await service.rerank("查询", ["文档1", "文档2"])
# 或通过工厂直接调用:
adapter = manager.create_reranker("rerank-model")
result = await adapter.rerank(RerankRequest(query="...", documents=["..."]))
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Protocol, runtime_checkable
from app.ai_provider.types.errors import AIProviderError
if TYPE_CHECKING:
from app.ai_provider.config.models import ModelConfig, ModelEndpointConfig
class RerankerServiceError(AIProviderError):
"""Reranker API 调用失败的基础错误。"""
# ── 请求 / 响应 ────────────────────────────────────────────────────────
@dataclass
class RerankRequest:
"""重排序请求。
属性:
query: 查询文本。
documents: 待重排序的文档列表。
model: 模型标识符。
top_n: 返回的最相关文档数量(nil 表示全部)。
max_chunks_per_doc: 每个文档最多切分为多少块。
return_documents: 返回结果中是否包含文档文本。
"""
query: str
documents: list[str]
model: str = ""
top_n: int | None = None
max_chunks_per_doc: int | None = None
return_documents: bool = False
@dataclass
class RerankResult:
"""单条重排序结果。"""
index: int
relevance_score: float
text: str | None = None
@dataclass
class RerankResponse:
"""重排序 API 的完整响应。"""
results: list[RerankResult]
model: str = ""
usage: dict[str, int] | None = None
# ── Protocol ────────────────────────────────────────────────────────────
@runtime_checkable
class RerankerProtocol(Protocol):
"""Reranker 适配器的协议。"""
async def rerank(self, request: RerankRequest) -> RerankResponse:
"""对文档进行重排序。
异常:
RerankerServiceError: API 调用失败或响应异常时抛出。
"""
...
async def close(self) -> None:
"""释放适配器持有的 HTTP 连接。"""
...
# ── 共享工厂基类 ──────────────────────────────────────────────────────
class RerankerAdapterBase:
"""Reranker 适配器基类,提供共享的工厂方法。
子类应在 ``__init__`` 中接受 ``base_url``、``api_key``、``model``、
``request_timeout_seconds`` 参数。
"""
_adapter_type: str = "reranker"
_default_timeout: int = 60
@classmethod
def from_endpoint(
cls,
endpoint: ModelEndpointConfig,
*,
timeout_seconds: int | None = None,
) -> Any:
"""从已解析的端点配置构建适配器。"""
base_url = endpoint.base_url
api_key = endpoint.api_key
if not base_url:
raise RerankerServiceError(
status_code=500,
code="reranker_endpoint_invalid",
message="Reranker endpoint has no resolved base_url.",
details={"endpoint": endpoint.name},
)
return cls(
base_url=base_url,
api_key=api_key,
model=endpoint.model,
request_timeout_seconds=timeout_seconds
or endpoint.request_timeout_seconds
or cls._default_timeout,
)
@classmethod
def from_model_config(
cls,
config: ModelConfig,
*,
timeout_seconds: int | None = None,
) -> Any:
"""从逻辑模型配置构建适配器(使用第一个活跃端点)。"""
for ep in config.endpoints:
if ep.status == "active":
return cls.from_endpoint(ep, timeout_seconds=timeout_seconds)
raise RerankerServiceError(
status_code=500,
code="reranker_no_endpoint",
message=f"Reranker model '{config.name}' has no active endpoint.",
)
@@ -1,186 +0,0 @@
"""Cohere 兼容格式的 Reranker 适配器 — ``POST /v1/rerank``。
覆盖 Cohere、mixedbread、Jina AI 等采用 Cohere 兼容重排序端点的提供商。
请求::
POST {base_url}/v1/rerank
Authorization: Bearer {api_key}
Content-Type: application/json
{
"model": "rerank-model",
"query": "查询文本",
"documents": ["文档1", "文档2"],
"top_n": 3,
"return_documents": false
}
响应::
{
"results": [
{"index": 0, "relevance_score": 0.99, "document": {"text": "文档..."}},
{"index": 1, "relevance_score": 0.85, "document": {"text": "文档..."}}
],
"meta": {"billed_units": {"search_units": 2}}
}
另见:
- Cohere Rerank API: https://docs.cohere.com/reference/rerank
- Jina Reranker: https://api.jina.ai/redoc#tag/rerank
"""
from __future__ import annotations
from typing import Any
import httpx
from app.ai_provider.reranker.base import (
RerankerAdapterBase,
RerankerProtocol,
RerankerServiceError,
RerankRequest,
RerankResponse,
RerankResult,
)
from app.core.logging import logger
class CohereFormatRerankerAdapter(RerankerAdapterBase, RerankerProtocol):
"""Cohere 兼容重排序适配器,覆盖多数重排序服务提供商。"""
_adapter_type = "reranker"
_default_timeout = 60
def __init__(
self,
*,
base_url: str,
api_key: str,
model: str = "rerank-model",
request_timeout_seconds: int = 60,
) -> None:
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._model = model
self._timeout = request_timeout_seconds
self._client: httpx.AsyncClient | None = None
# ── 客户端 ───────────────────────────────────────────────────────────
@property
def _http(self) -> httpx.AsyncClient:
if self._client is None or self._client.is_closed:
self._client = httpx.AsyncClient(timeout=httpx.Timeout(self._timeout))
return self._client
async def close(self) -> None:
if self._client is not None and not self._client.is_closed:
await self._client.aclose()
# ── RerankerProtocol ────────────────────────────────────────────────
async def rerank(self, request: RerankRequest) -> RerankResponse:
"""通过 ``POST /v1/rerank`` 对文档进行重排序。"""
url = f"{self._base_url}/v1/rerank"
payload: dict[str, Any] = {
"model": request.model or self._model,
"query": request.query,
"documents": request.documents,
"return_documents": request.return_documents,
}
if request.top_n is not None:
payload["top_n"] = request.top_n
if request.max_chunks_per_doc is not None:
payload["max_chunks_per_doc"] = request.max_chunks_per_doc
headers = {
"Authorization": f"Bearer {self._api_key}",
"Content-Type": "application/json",
}
logger.info(
"reranker.request",
model=payload["model"],
query_length=len(request.query),
document_count=len(request.documents),
top_n=request.top_n,
url=url,
)
try:
response = await self._http.post(url, json=payload, headers=headers)
except httpx.TimeoutException as exc:
raise RerankerServiceError(
status_code=504,
code="reranker_timeout",
message="Reranker request timed out.",
) from exc
except httpx.RequestError as exc:
raise RerankerServiceError(
status_code=503,
code="reranker_connection_error",
message=f"Reranker connection error: {exc}",
) from exc
if response.status_code != 200:
body = response.text[:500]
raise RerankerServiceError(
status_code=response.status_code,
code="reranker_api_error",
message=f"Reranker API returned {response.status_code}: {body}",
)
result = self._parse_response(response.json())
logger.info(
"reranker.success",
model=payload["model"],
total_results=len(result.results),
)
return result
# ── 辅助方法 ──────────────────────────────────────────────────────────
@staticmethod
def _parse_response(body: dict) -> RerankResponse:
"""从 Cohere 格式的重排序响应中提取结果。"""
try:
raw_results = body.get("results", [])
results = []
for item in raw_results:
results.append(
RerankResult(
index=item.get("index", 0),
relevance_score=float(item.get("relevance_score", 0.0)),
text=item.get("document", {}).get("text")
if isinstance(item.get("document"), dict)
else None,
)
)
# 按相关性分数降序排列
results.sort(key=lambda r: r.relevance_score, reverse=True)
# 提取用量信息(不同提供商格式各异)
usage: dict[str, int] | None = None
meta = body.get("meta")
if isinstance(meta, dict):
billed = meta.get("billed_units")
if isinstance(billed, dict):
usage = {k: int(v) for k, v in billed.items() if isinstance(v, (int, float))}
return RerankResponse(
results=results,
model=body.get("model") or body.get("id", ""),
usage=usage,
)
except Exception as exc:
raise RerankerServiceError(
status_code=500,
code="reranker_response_parse_error",
message=f"Failed to parse reranker response: {exc}",
) from exc
@@ -1,14 +0,0 @@
from app.ai_provider.routing.multimodal import MultiModalDecision, MultiModalRouter
from app.ai_provider.routing.permissions import (
CapabilityPermissionController,
PermissionRule,
)
from app.ai_provider.routing.selector import ModelSelector
__all__ = [
"MultiModalDecision",
"MultiModalRouter",
"ModelSelector",
"PermissionRule",
"CapabilityPermissionController",
]
@@ -1,81 +0,0 @@
from __future__ import annotations
from enum import StrEnum
from typing import Any
class MultiModalDecision(StrEnum):
"""基于消息内容分析的选路决策。
每个值代表所选模型必须满足的一项能力需求。
PLAIN_TEXT 表示不需要特殊能力。
"""
PLAIN_TEXT = "plain_text"
VISION = "vision"
THINKING = "thinking"
TOOL_USE = "tool_use"
STRUCTURED_OUTPUT = "structured_output"
class MultiModalRouter:
"""分析消息列表以确定所需的模型能力。
用法::
messages = [{"role": "user", "content": "Hello"}]
decisions = MultiModalRouter.decide_from_messages(messages)
# -> {MultiModalDecision.PLAIN_TEXT}
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "What's in this image?"},
{"type": "image_url", "image_url": {"url": "..."}},
],
},
]
decisions = MultiModalRouter.decide_from_messages(messages)
# -> {MultiModalDecision.VISION}
"""
@staticmethod
def decide_from_messages(messages: list[dict[str, Any]]) -> set[MultiModalDecision]:
"""分析消息并返回所需能力的集合。
参数:
messages: 消息字典列表,每个字典至少包含 ``role`` 和 ``content`` 键(兼容 OpenAI 格式)。
返回:
:class:`MultiModalDecision` 值的集合。如果未检测到特殊能力,
返回 ``{PLAIN_TEXT}``。
"""
if not messages:
return {MultiModalDecision.PLAIN_TEXT}
decisions: set[MultiModalDecision] = set()
for msg in messages:
role = msg.get("role", "")
content = msg.get("content", "")
# --- VISION: content is a list of blocks with image type ---
if isinstance(content, list):
for block in content:
if isinstance(block, dict):
block_type = block.get("type", "")
if block_type in ("image_url", "image"):
decisions.add(MultiModalDecision.VISION)
# --- TOOL_USE: assistant sent tool_calls, or message is tool result ---
if msg.get("tool_calls"):
decisions.add(MultiModalDecision.TOOL_USE)
if role == "tool":
decisions.add(MultiModalDecision.TOOL_USE)
# --- THINKING: assistant included reasoning ---
if msg.get("reasoning"):
decisions.add(MultiModalDecision.THINKING)
return decisions if decisions else {MultiModalDecision.PLAIN_TEXT}
@@ -1,100 +0,0 @@
from __future__ import annotations
import fnmatch
from typing import Any
class PermissionRule:
"""模型能力访问控制的单条权限规则。
规则使用 ``fnmatch`` 风格的通配符模式匹配模型名称。
参数:
model_pattern: ``fnmatch`` 模式,用于匹配模型名(如 ``"deepseek-*"``)。
capability: 此规则管控的能力(如 ``"vision"``)。
allowed: 能力是否允许(``True`` 允许,``False`` 拒绝)。
示例::
# 拒绝 "basic-*" 模型上的 vision 能力
PermissionRule(model_pattern="basic-*", capability="vision", allowed=False)
# 允许所有模型的 thinking 能力
PermissionRule(model_pattern="*", capability="thinking", allowed=True)
"""
def __init__(
self,
model_pattern: str,
capability: str,
allowed: bool = True,
) -> None:
self.model_pattern = model_pattern
self.capability = capability
self.allowed = allowed
def match(self, model_name: str) -> bool:
"""检查此规则是否适用于给定的模型名。"""
return fnmatch.fnmatch(model_name, self.model_pattern)
def __repr__(self) -> str:
verb = "ALLOW" if self.allowed else "DENY"
return f"PermissionRule({verb} {self.capability} on {self.model_pattern!r})"
def __eq__(self, other: Any) -> bool:
if not isinstance(other, PermissionRule):
return NotImplemented
return (
self.model_pattern == other.model_pattern
and self.capability == other.capability
and self.allowed == other.allowed
)
def __hash__(self) -> int:
return hash((self.model_pattern, self.capability, self.allowed))
class CapabilityPermissionController:
"""控制哪些模型允许哪些能力。
按顺序评估 :class:`PermissionRule` 列表。
**最后匹配的规则** 胜出(最具体的规则应放在较宽泛的规则之后)。
如果没有规则匹配某个模型/能力对,该能力**默认允许**。
用法::
controller = CapabilityPermissionController([
PermissionRule(model_pattern="*", capability="vision", allowed=False),
PermissionRule(model_pattern="vision-*", capability="vision", allowed=True),
])
controller.check("fast-model", "vision") # False
controller.check("vision-model", "vision") # True
"""
def __init__(self, rules: list[PermissionRule] | None = None) -> None:
self._rules = list(rules) if rules else []
def check(self, model_name: str, capability: str) -> bool:
"""检查 *capability* 是否在 *model_name* 上被允许。
返回 ``True`` 表示允许,``False`` 表示拒绝。
"""
result: bool | None = None
for rule in self._rules:
if rule.capability == capability and rule.match(model_name):
result = rule.allowed
# Default: allow if no rule matched
return result if result is not None else True
def is_allowed(self, model_name: str, capability: str) -> bool:
""":meth:`check` 的别名。"""
return self.check(model_name, capability)
def filter_capabilities(
self,
model_name: str,
capabilities: set[str],
) -> set[str]:
"""返回 *capabilities* 中在 *model_name* 上允许的子集。"""
return {c for c in capabilities if self.check(model_name, c)}
-111
View File
@@ -1,111 +0,0 @@
from __future__ import annotations
from typing import Any
from app.ai_provider.config.models import ModelConfig
from app.ai_provider.core.registry import ModelRegistry
from app.ai_provider.routing.multimodal import MultiModalDecision
from app.ai_provider.types.errors import AIProviderError
_SELECTOR_CAPABILITY_MAP: dict[MultiModalDecision, str] = {
MultiModalDecision.VISION: "vision",
MultiModalDecision.THINKING: "thinking",
MultiModalDecision.TOOL_USE: "tool_use",
MultiModalDecision.STRUCTURED_OUTPUT: "structured_output",
MultiModalDecision.PLAIN_TEXT: "chat",
}
class ModelSelector:
"""为给定上下文选择最佳模型。
用法::
selector = ModelSelector(registry)
model = selector.select({"capabilities": {"vision", "thinking"}})
model = selector.select({"preferred": "fast-model"})
model = selector.select_from_decisions({MultiModalDecision.VISION})
"""
def __init__(self, registry: ModelRegistry) -> None:
self._registry = registry
def select(self, context: dict[str, Any]) -> ModelConfig:
"""根据给定的上下文选择模型。
上下文键:
* ``capabilities`` (``set[str]``, 可选) — 必需的能力。
* ``preferred`` (``str``, 可选) — 偏好的模型名或别名。
* ``avoid`` (``set[str]``, 可选) — 要排除的模型名。
如果给了偏好模型但不能满足所有必需的能力,此方法会回退到
能满足要求的模型。
"""
capabilities: set[str] = set(context.get("capabilities", set()))
preferred: str | None = context.get("preferred")
avoid: set[str] = set(context.get("avoid", set()))
# If preferred is given, resolve it and check capabilities
if preferred is not None:
model = self._registry.resolve(preferred)
if (
not capabilities or self._model_satisfies_all(model, capabilities)
) and model.name not in avoid:
return model
# Find models that satisfy all required capabilities
if capabilities:
return self._select_by_capabilities(capabilities, avoid)
# Fall back to default
return self._registry.resolve()
def select_from_decisions(
self,
decisions: set[MultiModalDecision],
) -> ModelConfig:
"""基于 :class:`MultiModalDecision` 值选择模型。
将决策翻译为能力字符串并委托给 :meth:`select`。
"""
capabilities: set[str] = set()
for d in decisions:
cap = _SELECTOR_CAPABILITY_MAP.get(d)
if cap and cap != "chat":
capabilities.add(cap)
return self.select({"capabilities": capabilities})
# ------------------------------------------------------------------
def _select_by_capabilities(
self,
capabilities: set[str],
avoid: set[str],
) -> ModelConfig:
"""找到支持所有必需能力的模型。"""
# Narrow down by each capability (ModelConfig is unhashable, use lists)
candidates: list[ModelConfig] = list(self._registry.list_all())
for cap in capabilities:
cap_models = list(self._registry.list_by_capability(cap))
cap_names = {m.name for m in cap_models}
candidates = [m for m in candidates if m.name in cap_names]
# Remove avoided models
candidates = [m for m in candidates if m.name not in avoid]
if not candidates:
raise AIProviderError(
message=f"No active model supports all capabilities: {capabilities}",
code="CAPABILITY_NOT_FOUND",
status_code=404,
)
# Prefer higher-weight models (higher = better)
return max(candidates, key=lambda m: m.weight)
@staticmethod
def _model_satisfies_all(model: ModelConfig, capabilities: set[str]) -> bool:
caps = model.resolved_capabilities()
return all(caps.get(cap, False) for cap in capabilities)
-21
View File
@@ -1,21 +0,0 @@
from app.ai_provider.types.chat import ChatChunk, ChatResult, TokenUsage
from app.ai_provider.types.errors import AIProviderError
from app.ai_provider.types.messages import (
AIMessage,
BaseMessage,
HumanMessage,
SystemMessage,
ToolMessage,
)
__all__ = [
"AIMessage",
"BaseMessage",
"HumanMessage",
"SystemMessage",
"ToolMessage",
"ChatChunk",
"ChatResult",
"TokenUsage",
"AIProviderError",
]
-41
View File
@@ -1,41 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
@dataclass
class TokenUsage:
prompt_tokens: int = 0
completion_tokens: int = 0
total_tokens: int = 0
prompt_tokens_cached: int = 0 # 缓存命中的 prompt token 数(按缓存费率计费)
def __add__(self, other: TokenUsage) -> TokenUsage:
return TokenUsage(
prompt_tokens=self.prompt_tokens + other.prompt_tokens,
completion_tokens=self.completion_tokens + other.completion_tokens,
total_tokens=self.total_tokens + other.total_tokens,
prompt_tokens_cached=self.prompt_tokens_cached + other.prompt_tokens_cached,
)
@dataclass
class ChatChunk:
delta_content: str = ""
delta_reasoning: str = ""
tool_call_chunks: list[dict[str, Any]] = field(default_factory=list)
usage: TokenUsage | None = None
finish_reason: str | None = None
content_block_index: int | None = None
content_block_start: dict[str, Any] | None = None
content_block_delta: dict[str, Any] | None = None
@dataclass
class ChatResult:
content: str = ""
reasoning: str = ""
tool_calls: list[dict[str, Any]] = field(default_factory=list)
usage: TokenUsage | None = None
model_name: str = ""
-33
View File
@@ -1,33 +0,0 @@
from __future__ import annotations
class AIProviderError(Exception):
def __init__(
self,
message: str,
code: str = "UNKNOWN",
status_code: int = 500,
provider: str | None = None,
original: Exception | None = None,
) -> None:
self.message = message
self.code = code
self.status_code = status_code
self.provider = provider
self.original = original
super().__init__(message)
@classmethod
def from_provider(
cls,
message: str,
code: str = "PROVIDER_ERROR",
provider: str | None = None,
original: Exception | None = None,
) -> AIProviderError:
return cls(
message=message, code=code, status_code=502, provider=provider, original=original
)
def __str__(self) -> str:
return self.message
-81
View File
@@ -1,81 +0,0 @@
from __future__ import annotations
from typing import Any, Literal
Role = Literal["system", "user", "assistant", "tool"]
class BaseMessage:
role: Role
content: str
def __init__(self, *, content: str, role: Role) -> None:
self.content = content
self.role = role
def to_dict(self) -> dict[str, Any]:
return {"role": self.role, "content": self.content}
@classmethod
def from_dict(cls, d: dict[str, Any]) -> BaseMessage:
role = d["role"]
content = d.get("content", "")
if role == "system":
return SystemMessage(content=content)
if role == "user":
return HumanMessage(content=content)
if role == "assistant":
return AIMessage(
content=content,
tool_calls=d.get("tool_calls"),
reasoning=d.get("reasoning"),
raw_content_blocks=d.get("raw_content_blocks"),
)
if role == "tool":
return ToolMessage(content=content, tool_call_id=d.get("tool_call_id", ""))
raise ValueError(f"Unknown role: {role}")
class SystemMessage(BaseMessage):
def __init__(self, content: str) -> None:
super().__init__(content=content, role="system")
class HumanMessage(BaseMessage):
def __init__(self, content: str) -> None:
super().__init__(content=content, role="user")
class AIMessage(BaseMessage):
def __init__(
self,
content: str,
tool_calls: list[dict[str, Any]] | None = None,
reasoning: str | None = None,
raw_content_blocks: list[dict[str, Any]] | None = None,
) -> None:
super().__init__(content=content, role="assistant")
self.tool_calls = tool_calls or []
self.reasoning = reasoning or ""
self.raw_content_blocks = raw_content_blocks or []
def to_dict(self) -> dict[str, Any]:
d = super().to_dict()
if self.tool_calls:
d["tool_calls"] = self.tool_calls
if self.reasoning:
d["reasoning"] = self.reasoning
if self.raw_content_blocks:
d["raw_content_blocks"] = self.raw_content_blocks
return d
class ToolMessage(BaseMessage):
def __init__(self, content: str, tool_call_id: str) -> None:
super().__init__(content=content, role="tool")
self.tool_call_id = tool_call_id
def to_dict(self) -> dict[str, Any]:
d = super().to_dict()
d["tool_call_id"] = self.tool_call_id
return d
+22 -14
View File
@@ -4,30 +4,35 @@ from fastapi import APIRouter
from app.api.v1.endpoints import ( from app.api.v1.endpoints import (
admin, admin,
audio,
auth, auth,
avatar,
character_import, character_import,
characters, characters,
drive,
events, events,
goals,
health, health,
imports, imports,
embryo_drafts, embryo_drafts,
jingsheng_characters, jingsheng_characters,
kernel_runs, kernel_runs,
lorebooks, lorebooks,
me,
baigong,
hetu,
soul,
tianzai,
kouxin,
tiangong,
xingyi,
yuanqi,
mirror_trees, mirror_trees,
mcp, mcp,
personality,
runs, runs,
safety, safety,
sandbox_files, sandbox_files,
settings, settings,
skills,
tools, tools,
world,
xirang, xirang,
) )
@@ -38,24 +43,27 @@ api_router.include_router(characters.router)
api_router.include_router(character_import.router) api_router.include_router(character_import.router)
api_router.include_router(runs.router) api_router.include_router(runs.router)
api_router.include_router(kernel_runs.router) api_router.include_router(kernel_runs.router)
api_router.include_router(me.router)
api_router.include_router(hetu.world_router)
api_router.include_router(soul.soul_router)
api_router.include_router(tiangong.task_router)
api_router.include_router(kouxin.knock_router)
api_router.include_router(tianzai.device_router)
api_router.include_router(xingyi.form_router)
api_router.include_router(yuanqi.cov_router)
for _baigong_r in baigong.routers:
api_router.include_router(_baigong_r)
api_router.include_router(embryo_drafts.router) api_router.include_router(embryo_drafts.router)
api_router.include_router(jingsheng_characters.router) api_router.include_router(jingsheng_characters.router)
api_router.include_router(mirror_trees.router) api_router.include_router(mirror_trees.router)
api_router.include_router(xirang.router) api_router.include_router(xirang.router)
api_router.include_router(mirror_trees.node_router) api_router.include_router(mirror_trees.node_router)
api_router.include_router(lorebooks.router) api_router.include_router(lorebooks.router)
api_router.include_router(personality.router)
api_router.include_router(tools.router) api_router.include_router(tools.router)
api_router.include_router(skills.router)
api_router.include_router(mcp.router) api_router.include_router(mcp.router)
api_router.include_router(safety.router) api_router.include_router(safety.router)
api_router.include_router(sandbox_files.router) api_router.include_router(sandbox_files.router)
api_router.include_router(settings.router) api_router.include_router(settings.router)
api_router.include_router(world.router)
api_router.include_router(imports.router) api_router.include_router(imports.router)
api_router.include_router(drive.router)
api_router.include_router(events.router) api_router.include_router(events.router)
api_router.include_router(goals.router)
api_router.include_router(admin.router) api_router.include_router(admin.router)
api_router.include_router(audio.router)
api_router.include_router(avatar.router)
+7 -28
View File
@@ -1,6 +1,6 @@
"""API 层依赖注入。 """API 层依赖注入。
从 ``app.state`` 取 lifespan 装配的共享运行时(ToolRuntime / ConfirmationStore / EventStore)。 从 ``app.state`` 取 lifespan 装配的共享运行时(乾坤沙箱客户端 / EventStore)。
CRUD 类路由直接调用 ``services`` 函数(函数内部自管 ``AsyncSessionLocal``),不在此注入 session。 CRUD 类路由直接调用 ``services`` 函数(函数内部自管 ``AsyncSessionLocal``),不在此注入 session。
""" """
@@ -11,12 +11,10 @@ import hmac
from fastapi import Depends, HTTPException, Request from fastapi import Depends, HTTPException, Request
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.application.agent.confirmation import ConfirmationRegistry
from app.application.tool_runtime.runtime import ToolRuntime
from app.core.config import settings from app.core.config import settings
from app.db.session import get_session from app.db.session import get_session
from app.integrations.sandbox.client import SandboxClient
from app.modules.runtime.events import EventStore from app.modules.runtime.events import EventStore
from app.modules.safety.confirmation import ConfirmationStore
from app.schemas.auth import CurrentUser as CurrentUserSchema from app.schemas.auth import CurrentUser as CurrentUserSchema
from app.services.auth import decode_token, get_user_by_id, get_user_permissions from app.services.auth import decode_token, get_user_by_id, get_user_permissions
@@ -28,30 +26,11 @@ def _unavailable(message: str) -> HTTPException:
) )
def get_tool_runtime(request: Request) -> ToolRuntime: def get_sandbox_client(request: Request) -> SandboxClient:
runtime: ToolRuntime | None = getattr(request.app.state, "tool_runtime", None) sandbox: SandboxClient | None = getattr(request.app.state, "baigong_sandbox", None)
if runtime is None: if sandbox is None:
raise _unavailable("工具运行时未就绪(沙箱未配置或未启动)") raise _unavailable("乾坤沙箱未就绪(SANDBOX 未配置或未启动)")
return runtime return sandbox
def get_optional_tool_runtime(request: Request) -> ToolRuntime | None:
"""宽松版:tool_runtime 未就绪时返回 None(不抛 503),供聚合视图降级呈现。"""
return getattr(request.app.state, "tool_runtime", None)
def get_confirmation_store(request: Request) -> ConfirmationStore:
store = getattr(request.app.state, "confirmation_store", None)
if store is None:
raise _unavailable("确认存储未就绪")
return store
def get_confirmation_registry(request: Request) -> ConfirmationRegistry:
registry: ConfirmationRegistry = getattr(request.app.state, "confirmation_registry", None)
if registry is None:
raise _unavailable("确认注册表未就绪")
return registry
def get_event_store(request: Request) -> EventStore: def get_event_store(request: Request) -> EventStore:
+14 -152
View File
@@ -4,7 +4,7 @@
+ 五组治理对象 CRUD(§19.4)。治理写操作由 service 层记审计并入起居注。 + 五组治理对象 CRUD(§19.4)。治理写操作由 service 层记审计并入起居注。
路由直接调用 ``services`` 函数(函数内部自管 ``AsyncSessionLocal``),不持有 session; 路由直接调用 ``services`` 函数(函数内部自管 ``AsyncSessionLocal``),不持有 session;
``ToolRuntime`` / ``ConfirmationStore`` 等运行时对象经 ``deps`` 从 ``app.state`` 注入。 聚合视图自持查询(百工域常驻),无注入运行时。
""" """
import json import json
@@ -13,49 +13,29 @@ from fastapi import APIRouter, Depends, Query
from pydantic import BaseModel from pydantic import BaseModel
from app.api.v1.deps import ( from app.api.v1.deps import (
get_confirmation_store,
get_current_user, get_current_user,
get_optional_tool_runtime,
require_admin, require_admin,
) )
from app.application.tool_runtime.runtime import ToolRuntime
from app.models.runtime import Event from app.models.runtime import Event
from app.modules.safety.confirmation import ConfirmationStore
from app.schemas.admin import (
AdminCreateUserRequest,
AdminUpdateUserRequest,
AdminUserResponse,
AgentRosterResponse,
AuditEntry,
CreateAgentRequest,
CreateGoalRequest,
CreatePerceptionChannelRequest,
CreatePlanRequest,
CreateWorldEntityRequest,
GoalNode,
PendingApproval,
PerceptionChannelResponse,
PivotHealthReport,
PlanSummaryResponse,
TimeEventListResponse,
TimeEventResponse,
TraceDetail,
TraceListResponse,
UpdateAgentRequest,
UpdateGoalRequest,
UpdatePerceptionChannelRequest,
UpdatePlanRequest,
UpdateWorldEntityRequest,
WorldEntityResponse,
)
from app.schemas.admin_character import ( from app.schemas.admin_character import (
AdminCreateTemplateRequest, AdminCreateTemplateRequest,
AdminTemplateSummaryResponse, AdminTemplateSummaryResponse,
AdminUpdateTemplateRequest, AdminUpdateTemplateRequest,
) )
from app.schemas.admin import (
AdminCreateUserRequest,
AdminUpdateUserRequest,
AdminUserResponse,
AuditEntry,
PendingApproval,
PivotHealthReport,
TimeEventListResponse,
TimeEventResponse,
TraceDetail,
TraceListResponse,
)
from app.schemas.auth import CurrentUser from app.schemas.auth import CurrentUser
from app.schemas.common import Page from app.schemas.common import Page
from app.services import governance as governance_service
from app.services.admin import ( from app.services.admin import (
get_audit_timeline, get_audit_timeline,
get_pending_approvals, get_pending_approvals,
@@ -80,13 +60,9 @@ router = APIRouter(prefix="/admin", tags=["admin"], dependencies=[Depends(requir
@router.get("/pivots/health", response_model=PivotHealthReport) @router.get("/pivots/health", response_model=PivotHealthReport)
async def pivots_health( async def pivots_health(
confirmation_store: ConfirmationStore = Depends(get_confirmation_store),
tool_runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
current_user: CurrentUser = Depends(get_current_user), current_user: CurrentUser = Depends(get_current_user),
) -> PivotHealthReport: ) -> PivotHealthReport:
return await get_pivots_health( return await get_pivots_health(
confirmation_store=confirmation_store,
tool_runtime=tool_runtime,
owner_scope=current_user.owner_scope, owner_scope=current_user.owner_scope,
) )
@@ -96,12 +72,8 @@ async def pivots_health(
@router.get("/safety/pending", response_model=Page[PendingApproval]) @router.get("/safety/pending", response_model=Page[PendingApproval])
async def safety_pending( async def safety_pending(
confirmation_store: ConfirmationStore = Depends(get_confirmation_store),
tool_runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> Page[PendingApproval]: ) -> Page[PendingApproval]:
items = await get_pending_approvals( items = await get_pending_approvals()
confirmation_store=confirmation_store, tool_runtime=tool_runtime
)
return Page(items=items, page=1, page_size=len(items), total=len(items)) return Page(items=items, page=1, page_size=len(items), total=len(items))
@@ -120,139 +92,29 @@ async def audit_timeline(
# —— §19.4 太一枢 · 目标 —— # —— §19.4 太一枢 · 目标 ——
@router.get("/goals", response_model=list[GoalNode])
async def list_goals(
character_id: str | None = Query(default=None, description="按人物过滤。"),
) -> list[GoalNode]:
if character_id:
return await governance_service.list_goals_by_character(character_id)
return await governance_service.list_goals()
@router.post("/goals", response_model=GoalNode)
async def create_goal(payload: CreateGoalRequest) -> GoalNode:
return await governance_service.create_goal(payload)
@router.patch("/goals/{goal_id}", response_model=GoalNode)
async def update_goal(
goal_id: str,
payload: UpdateGoalRequest,
) -> GoalNode:
return await governance_service.update_goal(goal_id, payload)
@router.delete("/goals/{goal_id}", status_code=204)
async def delete_goal(goal_id: str) -> None:
await governance_service.delete_goal(goal_id)
# —— §19.4 烛照枢 · 感知通道 —— # —— §19.4 烛照枢 · 感知通道 ——
@router.get("/perception/channels", response_model=list[PerceptionChannelResponse])
async def list_perception_channels() -> list[PerceptionChannelResponse]:
return await governance_service.list_perception_channels()
@router.post("/perception/channels", response_model=PerceptionChannelResponse)
async def create_perception_channel(
payload: CreatePerceptionChannelRequest,
) -> PerceptionChannelResponse:
return await governance_service.create_perception_channel(payload)
@router.patch("/perception/channels/{channel}", response_model=PerceptionChannelResponse)
async def update_perception_channel(
channel: str,
payload: UpdatePerceptionChannelRequest,
) -> PerceptionChannelResponse:
return await governance_service.update_perception_channel(channel, payload)
@router.delete("/perception/channels/{channel}", status_code=204)
async def delete_perception_channel(channel: str) -> None:
await governance_service.delete_perception_channel(channel)
# —— §19.4 河图枢 · 世界实体 ——
@router.get("/world/entities", response_model=list[WorldEntityResponse])
async def list_world_entities() -> list[WorldEntityResponse]:
return await governance_service.list_world_entities()
@router.post("/world/entities", response_model=WorldEntityResponse)
async def create_world_entity(
payload: CreateWorldEntityRequest,
) -> WorldEntityResponse:
return await governance_service.create_world_entity(payload)
@router.patch("/world/entities/{entity_id}", response_model=WorldEntityResponse)
async def update_world_entity(
entity_id: str,
payload: UpdateWorldEntityRequest,
) -> WorldEntityResponse:
return await governance_service.update_world_entity(entity_id, payload)
@router.delete("/world/entities/{entity_id}", status_code=204)
async def delete_world_entity(entity_id: str) -> None:
await governance_service.delete_world_entity(entity_id)
# —— §19.4 洛书枢 · 计划 —— # —— §19.4 洛书枢 · 计划 ——
@router.get("/plans", response_model=list[PlanSummaryResponse])
async def list_plans() -> list[PlanSummaryResponse]:
return await governance_service.list_plans()
@router.post("/plans", response_model=PlanSummaryResponse)
async def create_plan(payload: CreatePlanRequest) -> PlanSummaryResponse:
return await governance_service.create_plan(payload)
@router.patch("/plans/{plan_id}", response_model=PlanSummaryResponse)
async def update_plan(
plan_id: str,
payload: UpdatePlanRequest,
) -> PlanSummaryResponse:
return await governance_service.update_plan(plan_id, payload)
@router.delete("/plans/{plan_id}", status_code=204)
async def delete_plan(plan_id: str) -> None:
await governance_service.delete_plan(plan_id)
# —— §19.4 天工阁枢 · 角色在岗 —— # —— §19.4 天工阁枢 · 角色在岗 ——
@router.get("/agents/roster", response_model=list[AgentRosterResponse])
async def list_agent_roster() -> list[AgentRosterResponse]:
return await governance_service.list_agent_roster()
@router.post("/agents/roster", response_model=AgentRosterResponse)
async def create_agent(payload: CreateAgentRequest) -> AgentRosterResponse:
return await governance_service.create_agent(payload)
@router.patch("/agents/roster/{role}", response_model=AgentRosterResponse)
async def update_agent(
role: str,
payload: UpdateAgentRequest,
) -> AgentRosterResponse:
return await governance_service.update_agent(role, payload)
@router.delete("/agents/roster/{role}", status_code=204)
async def delete_agent(role: str) -> None:
await governance_service.delete_agent(role)
# —— §19.5 链路追踪 —— # —— §19.5 链路追踪 ——
-275
View File
@@ -1,275 +0,0 @@
"""音频相关 API 端点 — TTS(文本转语音)、ASR(语音转文本)等。
音频端点使用神识的模型注册表查找可用的音频模型,并调用对应音频适配器合成 /
转写。ASR 与 TTS 按 ``tags`` 中的 ``asr`` / ``tts`` 能力区分,ASR 内部再按
``provider`` 区分协议:``yuanqi`` 走 multipart ``/v1/audio/transcriptions``,
其他走 MiMo 风格的 ``/chat/completions`` + ``input_audio``。
"""
from __future__ import annotations
from fastapi import APIRouter, File, Form, HTTPException, Response, UploadFile
from pydantic import BaseModel, Field
from app.ai_provider.audio import AudioService, AudioServiceError
from app.ai_provider.manager import ProviderManagerFactory
from app.core.logging import logger
router = APIRouter(tags=["audio"])
_MIME_MAP = {
"wav": "audio/wav",
"mp3": "audio/mpeg",
"opus": "audio/opus",
"flac": "audio/flac",
}
class _AudioModelFactory:
"""Adapter: wraps ProviderManager to provide ModelFactory interface for AudioService.
``create_audio(name)`` 按 ``tags`` 判断 ``asr`` / ``tts`` 能力:
- ``tts`` → ``HunjingTTSOpenAI``(``/chat/completions`` + ``audio``)
- ``asr`` → 按 ``provider`` 分派:
- ``yuanqi`` → ``YuanqiASRWhisper``(multipart ``/v1/audio/transcriptions``)
- 其他 → ``HunjingASROpenAI``(``/chat/completions`` + ``input_audio``)
"""
def __init__(self, manager) -> None:
self._manager = manager
@property
def registry(self):
return self._manager.model_registry
def create_audio(self, name: str):
model = self._manager.get_model_config(name)
is_asr = "asr" in model.tags
if is_asr:
return self._create_asr(model)
return self._create_tts(model)
def _provider_or_none(self, model):
"""安全获取 provider 配置(provider 未配置时返回 None,走纯端点解析)。"""
try:
return self._manager.get_provider_config(model.provider)
except Exception:
return None
# ── ASR ─────────────────────────────────────────────────────────────
def _create_asr(self, model):
"""按 provider 选择 ASR 协议适配器。"""
# yuanqi 本地引擎走 multipart /v1/audio/transcriptions
if model.provider == "yuanqi":
from app.ai_provider.audio.yuanqi_asr import YuanqiASRWhisper
if model.endpoints:
return YuanqiASRWhisper.from_model_config(
model, provider=self._provider_or_none(model)
)
provider = self._manager.get_provider_config(model.provider)
api_key, base_url = self._resolve_credentials(provider)
if not base_url:
raise AudioServiceError(
status_code=500,
code="audio_asr_no_base_url",
message=f"Provider '{model.provider}' has no base_url configured.",
)
return YuanqiASRWhisper(
base_url=base_url,
api_key=api_key or "",
model=model.model,
request_timeout_seconds=model.timeout_seconds or 300,
)
# MiMo 等走 /chat/completions + input_audio
from app.ai_provider.audio.asr import HunjingASROpenAI
if model.endpoints:
return HunjingASROpenAI.from_model_config(
model, provider=self._provider_or_none(model)
)
provider = self._manager.get_provider_config(model.provider)
api_key, base_url = self._resolve_credentials(provider)
if not base_url:
raise AudioServiceError(
status_code=500,
code="audio_asr_no_base_url",
message=f"Provider '{model.provider}' has no base_url configured.",
)
return HunjingASROpenAI(
base_url=base_url,
api_key=api_key or "",
model=model.model,
request_timeout_seconds=model.timeout_seconds or 300,
)
# ── TTS ─────────────────────────────────────────────────────────────
def _create_tts(self, model):
from app.ai_provider.audio.tts import HunjingTTSOpenAI
# 有 endpoints 时交给 from_model_config(按端点的 status/model/超时)
if model.endpoints:
return HunjingTTSOpenAI.from_model_config(
model, provider=self._provider_or_none(model)
)
# 无 endpoint 时回退到 provider 级别凭证
provider = self._manager.get_provider_config(model.provider)
api_key, base_url = self._resolve_credentials(provider)
if not base_url:
raise AudioServiceError(
status_code=500,
code="audio_tts_no_base_url",
message=f"Provider '{model.provider}' has no base_url configured.",
)
return HunjingTTSOpenAI(
base_url=base_url,
api_key=api_key or "",
model=model.model,
request_timeout_seconds=model.timeout_seconds or 120,
)
def _resolve_credentials(self, provider):
"""从 settings 解析 provider 级别的 API key 和 base URL。"""
from app.core.config import settings
api_key = (
settings.__getattribute__(provider.api_key_env.lower())
if hasattr(settings, provider.api_key_env.lower())
else ""
)
base_url = (
settings.__getattribute__(provider.base_url_env.lower())
if hasattr(settings, provider.base_url_env.lower())
else None
)
return api_key, base_url
class TTSRequest(BaseModel):
"""TTS 合成请求。"""
text: str = Field(..., min_length=1, max_length=4096, description="要合成语音的文本。")
voice: str = Field("default", description="语音角色标识(取决于模型,如 default 或声纹 ID)。")
format: str = Field("wav", description="音频格式:wav / mp3 / opus / flac。")
@router.post("/audio/tts")
async def synthesize_speech(body: TTSRequest) -> Response:
"""合成文本为语音,返回音频二进制流。
使用神识注册表中第一个活跃的 TTS 模型(capability: tts)。
未配置 TTS 模型时返回 503。
"""
if not body.text.strip():
raise HTTPException(status_code=400, detail="text 不能为空。")
mime = _MIME_MAP.get(body.format, "audio/wav")
try:
manager = ProviderManagerFactory.create()
service = AudioService(_AudioModelFactory(manager))
async with service:
result = await service.synthesize(
text=body.text,
voice=body.voice,
response_format=body.format,
)
except AudioServiceError as exc:
logger.error(
"audio.tts.api_error",
voice=body.voice,
format=body.format,
error_code=exc.code,
error_message=exc.message,
)
raise HTTPException(
status_code=exc.status_code or 502,
detail=exc.message,
) from exc
logger.info(
"audio.tts.success",
voice=body.voice,
format=body.format,
audio_bytes=len(result.audio_data),
)
return Response(
content=result.audio_data,
media_type=mime,
headers={
"Content-Disposition": f'inline; filename="speech.{body.format}"',
"X-Audio-Format": body.format,
"Cache-Control": "public, max-age=3600",
},
)
# ── ASR(语音转文本)───────────────────────────────────────────────────────
class TranscriptionResponse(BaseModel):
"""OpenAI Whisper 兼容的转写响应。"""
text: str = Field(description="转写后的文本")
@router.post("/audio/transcriptions", response_model=TranscriptionResponse)
async def transcribe_audio(
file: UploadFile = File(..., description="音频文件(wav/mp3/m4a/flac/ogg/opus/webm 等)"),
language: str | None = Form(default=None, description="语言提示(ISO-639-1:zh/en/ja/ko 等),省略则自动检测"),
model: str | None = Form(default=None, description="可选,指定使用的 ASR 模型逻辑名(如 funasr-sensevoice / mimo-v2.5-asr),省略则用注册表中第一个"),
) -> TranscriptionResponse:
"""转写音频为文本,返回 OpenAI Whisper 兼容的 JSON。
不指定 ``model`` 时用神识注册表中第一个活跃的 ASR 模型(capability: asr),
按其 ``provider`` 自动选择协议适配器:``yuanqi`` 走本地 multipart 端点,
其他走 MiMo 风格的 chat completions。未配置 ASR 模型时返回 503。
"""
audio_bytes = await file.read()
if not audio_bytes:
raise HTTPException(status_code=400, detail="音频文件为空。")
fmt = (file.filename or "audio.wav").rsplit(".", 1)[-1].lower() if "." in (file.filename or "") else "wav"
try:
manager = ProviderManagerFactory.create()
service = AudioService(_AudioModelFactory(manager))
async with service:
result = await service.transcribe(
audio_data=audio_bytes,
format=fmt,
language=language,
model_name=model,
)
except AudioServiceError as exc:
logger.error(
"audio.asr.api_error",
filename=file.filename,
format=fmt,
language=language,
error_code=exc.code,
error_message=exc.message,
)
raise HTTPException(
status_code=exc.status_code or 502,
detail=exc.message,
) from exc
logger.info(
"audio.asr.success",
filename=file.filename,
format=fmt,
language=language,
text_length=len(result.text),
text=result.text,
)
return TranscriptionResponse(text=result.text)
-47
View File
@@ -1,47 +0,0 @@
"""角色头像 API。
上传与获取角色头像图片文件。
"""
from fastapi import APIRouter, Depends, File, UploadFile
from app.api.v1.deps import get_current_user
from app.schemas.auth import CurrentUser
from app.services.avatar import (
get_avatar_response,
upload_avatar,
)
router = APIRouter(prefix="/characters", tags=["characters"])
@router.post("/{character_id}/avatar")
async def upload_character_avatar(
character_id: str,
file: UploadFile = File(..., description="头像图片,支持 JPG/PNG/WebP,最大 5MB。"),
current_user: CurrentUser = Depends(get_current_user),
) -> dict:
"""上传角色头像。
支持的格式:JPEG、PNG、WebP。最大 5MB。
新头像会替换旧头像(同角色只保留一张)。
"""
content = await file.read()
avatar_url = await upload_avatar(
character_id=character_id,
content=content,
content_type=file.content_type or "image/jpeg",
owner_scope=current_user.owner_scope,
)
return {"avatar_url": avatar_url}
@router.get("/{character_id}/avatar")
async def get_character_avatar(
character_id: str,
) -> object:
"""获取角色头像图片。
返回原始图片文件。无头像时返回 404。
"""
return await get_avatar_response(character_id)
+5
View File
@@ -0,0 +1,5 @@
"""百工与司命路由(域实现见 app/domains/baigong/api.py)。"""
from app.domains.baigong.api import routers
__all__ = ["routers"]
-73
View File
@@ -1,73 +0,0 @@
"""内驱力 /drive API。"""
from fastapi import APIRouter, Query
from app.schemas.common import Page
from app.schemas.drive import CuriosityResponse, SubgoalResponse
from app.services.drive import (
approve_subgoal,
defer_subgoal,
list_curiosities,
list_subgoals,
reject_subgoal,
)
router = APIRouter(tags=["drive"])
@router.get(
"/characters/{character_id}/curiosities",
response_model=Page[CuriosityResponse],
)
async def list_curiosities_endpoint(
character_id: str,
source: str | None = Query(default=None, description="按来源过滤(契约字典内驱力 source)"),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=20, ge=1, le=100),
) -> Page[CuriosityResponse]:
"""列出某人物的好奇问题(大周天内驱力生成)。"""
return await list_curiosities(
character_id=character_id,
source=source,
page=page,
page_size=page_size,
)
@router.get(
"/characters/{character_id}/subgoals",
response_model=Page[SubgoalResponse],
)
async def list_subgoals_endpoint(
character_id: str,
status: str | None = Query(
default=None, description="按状态过滤:candidate/approved/deferred/rejected"
),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=20, ge=1, le=100),
) -> Page[SubgoalResponse]:
"""列出某人物的探索性次生目标。"""
return await list_subgoals(
character_id=character_id,
status=status,
page=page,
page_size=page_size,
)
@router.post("/subgoals/{subgoal_id}/approve", response_model=SubgoalResponse)
async def approve_subgoal_endpoint(subgoal_id: str) -> SubgoalResponse:
"""通过探索性次生目标(仅标记 approved,本期不自动执行)。"""
return await approve_subgoal(subgoal_id)
@router.post("/subgoals/{subgoal_id}/defer", response_model=SubgoalResponse)
async def defer_subgoal_endpoint(subgoal_id: str) -> SubgoalResponse:
"""延后探索性次生目标(status → deferred)。"""
return await defer_subgoal(subgoal_id)
@router.post("/subgoals/{subgoal_id}/reject", response_model=SubgoalResponse)
async def reject_subgoal_endpoint(subgoal_id: str) -> SubgoalResponse:
"""驳回探索性次生目标(status → rejected)。"""
return await reject_subgoal(subgoal_id)
@@ -111,7 +111,8 @@ async def rehearse_embryo(
f"身份锚点:{json.dumps(baseline, ensure_ascii=False)}" f"身份锚点:{json.dumps(baseline, ensure_ascii=False)}"
) )
run = await kernel.start_rehearsal_run( run = await kernel.start_rehearsal_run(
embryo_id=embryo_id, prompt=payload.prompt, system_contract=system_contract embryo_id=embryo_id, prompt=payload.prompt, system_contract=system_contract,
owner_id=user.owner_scope,
) )
await jingsheng.record_rehearsal( await jingsheng.record_rehearsal(
embryo_id, user.owner_scope, run_id=run.id, prompt=payload.prompt embryo_id, user.owner_scope, run_id=run.id, prompt=payload.prompt
-21
View File
@@ -1,21 +0,0 @@
"""Per-Character 目标管理端点(非 admin,供 /drive 等前端使用)。"""
from fastapi import APIRouter, Query
from app.schemas.admin import CreateGoalRequest, GoalNode
from app.services import governance as governance_service
router = APIRouter(prefix="/goals", tags=["goals"])
@router.get("", response_model=list[GoalNode])
async def list_goals(
character_id: str = Query(..., description="按人物过滤。"),
status: str | None = Query(default=None, description="按状态过滤(§5.12)。"),
) -> list[GoalNode]:
return await governance_service.list_goals_by_character(character_id, status_filter=status)
@router.post("", response_model=GoalNode, status_code=201)
async def create_goal(payload: CreateGoalRequest) -> GoalNode:
return await governance_service.create_goal(payload)
+5
View File
@@ -0,0 +1,5 @@
"""河图世界路由(域实现见 app/domains/hetu/api.py)。"""
from app.domains.hetu.api import world_router
__all__ = ["world_router"]
-119
View File
@@ -10,12 +10,6 @@ from fastapi import APIRouter, Depends, File, Form, Query, UploadFile
from app.api.v1.deps import get_current_user from app.api.v1.deps import get_current_user
from app.schemas.auth import CurrentUser from app.schemas.auth import CurrentUser
from app.schemas.candidates import (
BatchReviewRequest,
BatchReviewResult,
MemoryCandidateResponse,
ProfileCandidateResponse,
)
from app.schemas.common import Page from app.schemas.common import Page
from app.schemas.imports import ( from app.schemas.imports import (
CreateImportRequest, CreateImportRequest,
@@ -23,18 +17,11 @@ from app.schemas.imports import (
ImportJobResponse, ImportJobResponse,
) )
from app.services.imports import ( from app.services.imports import (
batch_review_memory_candidates,
batch_review_profile_candidates,
create_import_from_json, create_import_from_json,
create_import_from_upload, create_import_from_upload,
get_import, get_import,
list_import_items, list_import_items,
list_jobs_page, list_jobs_page,
list_memory_candidates,
list_profile_candidates,
trigger_memory_extraction,
trigger_personality_extraction,
trigger_profile_extraction,
) )
router = APIRouter(tags=["imports"]) router = APIRouter(tags=["imports"])
@@ -116,109 +103,3 @@ async def read_import_items(
page_size=page_size, page_size=page_size,
owner_scope=current_user.owner_scope, owner_scope=current_user.owner_scope,
) )
# —— 候选抽取(5b,异步后台任务)——
@router.post(
"/characters/{character_id}/imports/{import_id}/profile-candidates",
response_model=ImportJobResponse,
)
async def trigger_profile_candidates(
character_id: str,
import_id: str,
current_user: CurrentUser = Depends(get_current_user),
) -> ImportJobResponse:
return await trigger_profile_extraction(
character_id, import_id, owner_scope=current_user.owner_scope
)
@router.get(
"/characters/{character_id}/imports/{import_id}/profile-candidates",
response_model=list[ProfileCandidateResponse],
)
async def list_profile_candidates_api(
character_id: str,
import_id: str,
status: str | None = Query(default=None, description="按审核状态过滤。"),
current_user: CurrentUser = Depends(get_current_user),
) -> list[ProfileCandidateResponse]:
return await list_profile_candidates(
character_id, import_id, status=status, owner_scope=current_user.owner_scope
)
@router.post(
"/characters/{character_id}/imports/{import_id}/memory-candidates",
response_model=ImportJobResponse,
)
async def trigger_memory_candidates(
character_id: str,
import_id: str,
current_user: CurrentUser = Depends(get_current_user),
) -> ImportJobResponse:
return await trigger_memory_extraction(
character_id, import_id, owner_scope=current_user.owner_scope
)
@router.get(
"/characters/{character_id}/imports/{import_id}/memory-candidates",
response_model=list[MemoryCandidateResponse],
)
async def list_memory_candidates_api(
character_id: str,
import_id: str,
status: str | None = Query(default=None, description="按审核状态过滤。"),
current_user: CurrentUser = Depends(get_current_user),
) -> list[MemoryCandidateResponse]:
return await list_memory_candidates(
character_id, import_id, status=status, owner_scope=current_user.owner_scope
)
@router.post(
"/characters/{character_id}/imports/{import_id}/personality-proposals",
response_model=ImportJobResponse,
)
async def trigger_personality_proposals(
character_id: str,
import_id: str,
current_user: CurrentUser = Depends(get_current_user),
) -> ImportJobResponse:
return await trigger_personality_extraction(
character_id, import_id, owner_scope=current_user.owner_scope
)
# —— 候选审核应用(5c)——
@router.post(
"/characters/{character_id}/profile-candidates/batch-review",
response_model=BatchReviewResult,
)
async def batch_review_profile(
character_id: str,
payload: BatchReviewRequest,
current_user: CurrentUser = Depends(get_current_user),
) -> BatchReviewResult:
return await batch_review_profile_candidates(
character_id, payload, owner_scope=current_user.owner_scope
)
@router.post(
"/characters/{character_id}/memory-candidates/batch-review",
response_model=BatchReviewResult,
)
async def batch_review_memory(
character_id: str,
payload: BatchReviewRequest,
current_user: CurrentUser = Depends(get_current_user),
) -> BatchReviewResult:
return await batch_review_memory_candidates(
character_id, payload, owner_scope=current_user.owner_scope
)
+1 -1
View File
@@ -50,6 +50,6 @@ async def create_kernel_run(
db, payload.character_id, owner_scope=user.owner_scope db, payload.character_id, owner_scope=user.owner_scope
) )
run = await kernel.start_foreground_run( run = await kernel.start_foreground_run(
character_id=character.id, prompt=payload.prompt character_id=character.id, prompt=payload.prompt, owner_id=user.owner_scope
) )
return await get_run_response(run.id) return await get_run_response(run.id)
+5
View File
@@ -0,0 +1,5 @@
"""叩心路由(域实现见 app/domains/kouxin/api.py)。"""
from app.domains.kouxin.api import knock_router
__all__ = ["knock_router"]
+17 -50
View File
@@ -7,15 +7,12 @@ stdio 模式通过 /start、/stop、/restart 控制子进程生命周期。
from fastapi import APIRouter, Depends, Query from fastapi import APIRouter, Depends, Query
from fastapi import status as http_status from fastapi import status as http_status
from app.api.v1.deps import get_optional_tool_runtime
from app.application.tool_runtime.runtime import ToolRuntime
from app.integrations.mcp.process_manager import get_process_manager from app.integrations.mcp.process_manager import get_process_manager
from app.schemas.mcp import ( from app.schemas.mcp import (
CreateMcpServerRequest, CreateMcpServerRequest,
McpServerResponse, McpServerResponse,
McpServerStatusResponse, McpServerStatusResponse,
UpdateMcpServerRequest, UpdateMcpServerRequest)
)
from app.services import mcp as mcp_service from app.services import mcp as mcp_service
router = APIRouter(prefix="/tools/mcp-servers", tags=["mcp"]) router = APIRouter(prefix="/tools/mcp-servers", tags=["mcp"])
@@ -24,12 +21,9 @@ router = APIRouter(prefix="/tools/mcp-servers", tags=["mcp"])
@router.post( @router.post(
"", "",
response_model=McpServerResponse, response_model=McpServerResponse,
status_code=http_status.HTTP_201_CREATED, status_code=http_status.HTTP_201_CREATED)
)
async def register_mcp_server( async def register_mcp_server(
payload: CreateMcpServerRequest, payload: CreateMcpServerRequest) -> McpServerResponse:
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> McpServerResponse:
"""注册 MCP server。 """注册 MCP server。
streamable-http:同步拉取工具清单并注册到运行时。 streamable-http:同步拉取工具清单并注册到运行时。
@@ -45,28 +39,23 @@ async def register_mcp_server(
env_json=payload.env_json, env_json=payload.env_json,
cwd=payload.cwd, cwd=payload.cwd,
status=payload.status, status=payload.status,
runtime=runtime,
process_manager=get_process_manager(), process_manager=get_process_manager(),
) )
@router.get("", response_model=list[McpServerResponse]) @router.get("", response_model=list[McpServerResponse])
async def list_mcp_servers( async def list_mcp_servers(
status: str | None = Query(default=None, description="按状态过滤:enabled/disabled"), status: str | None = Query(default=None, description="按状态过滤:enabled/disabled")) -> list[McpServerResponse]:
) -> list[McpServerResponse]:
"""列举 MCP server(含工具清单摘要)。""" """列举 MCP server(含工具清单摘要)。"""
return await mcp_service.list_mcp_servers(status=status) return await mcp_service.list_mcp_servers(status=status)
@router.patch( @router.patch(
"/{server_id}", "/{server_id}",
response_model=McpServerResponse, response_model=McpServerResponse)
)
async def update_mcp_server( async def update_mcp_server(
server_id: str, server_id: str,
payload: UpdateMcpServerRequest, payload: UpdateMcpServerRequest) -> McpServerResponse:
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> McpServerResponse:
"""更新 MCP server 字段(部分更新)。""" """更新 MCP server 字段(部分更新)。"""
return await mcp_service.update_mcp_server( return await mcp_service.update_mcp_server(
server_id, server_id,
@@ -77,79 +66,61 @@ async def update_mcp_server(
args_json=payload.args_json, args_json=payload.args_json,
env_json=payload.env_json, env_json=payload.env_json,
cwd=payload.cwd, cwd=payload.cwd,
runtime=runtime,
process_manager=get_process_manager(), process_manager=get_process_manager(),
) )
@router.delete("/{server_id}", status_code=http_status.HTTP_204_NO_CONTENT) @router.delete("/{server_id}", status_code=http_status.HTTP_204_NO_CONTENT)
async def delete_mcp_server( async def delete_mcp_server(
server_id: str, server_id: str) -> None:
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> None:
"""删除 MCP server(先 stop 子进程再删)。""" """删除 MCP server(先 stop 子进程再删)。"""
await mcp_service.delete_mcp_server( await mcp_service.delete_mcp_server(
server_id, server_id,
runtime=runtime,
process_manager=get_process_manager(), process_manager=get_process_manager(),
) )
@router.post( @router.post(
"/{server_id}/start", "/{server_id}/start",
response_model=McpServerStatusResponse, response_model=McpServerStatusResponse)
)
async def start_mcp_server( async def start_mcp_server(
server_id: str, server_id: str) -> McpServerStatusResponse:
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> McpServerStatusResponse:
"""启动 stdio MCP server 子进程,拉取工具并注册。""" """启动 stdio MCP server 子进程,拉取工具并注册。"""
return await mcp_service.start_mcp_server( return await mcp_service.start_mcp_server(
server_id, server_id,
runtime=runtime,
process_manager=get_process_manager(), process_manager=get_process_manager(),
) )
@router.post( @router.post(
"/{server_id}/stop", "/{server_id}/stop",
response_model=McpServerStatusResponse, response_model=McpServerStatusResponse)
)
async def stop_mcp_server( async def stop_mcp_server(
server_id: str, server_id: str) -> McpServerStatusResponse:
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> McpServerStatusResponse:
"""停止 stdio MCP server 子进程,注销工具。""" """停止 stdio MCP server 子进程,注销工具。"""
return await mcp_service.stop_mcp_server( return await mcp_service.stop_mcp_server(
server_id, server_id,
runtime=runtime,
process_manager=get_process_manager(), process_manager=get_process_manager(),
) )
@router.post( @router.post(
"/{server_id}/restart", "/{server_id}/restart",
response_model=McpServerStatusResponse, response_model=McpServerStatusResponse)
)
async def restart_mcp_server( async def restart_mcp_server(
server_id: str, server_id: str) -> McpServerStatusResponse:
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> McpServerStatusResponse:
"""重启 stdio MCP server。""" """重启 stdio MCP server。"""
return await mcp_service.restart_mcp_server( return await mcp_service.restart_mcp_server(
server_id, server_id,
runtime=runtime,
process_manager=get_process_manager(), process_manager=get_process_manager(),
) )
@router.get( @router.get(
"/{server_id}/status", "/{server_id}/status",
response_model=McpServerStatusResponse, response_model=McpServerStatusResponse)
)
async def get_mcp_server_status( async def get_mcp_server_status(
server_id: str, server_id: str) -> McpServerStatusResponse:
) -> McpServerStatusResponse:
"""查询 MCP server 进程状态。""" """查询 MCP server 进程状态。"""
return await mcp_service.get_mcp_server_status( return await mcp_service.get_mcp_server_status(
server_id, server_id,
@@ -159,15 +130,11 @@ async def get_mcp_server_status(
@router.post( @router.post(
"/{server_id}/refresh", "/{server_id}/refresh",
response_model=McpServerResponse, response_model=McpServerResponse)
)
async def refresh_mcp_server_tools( async def refresh_mcp_server_tools(
server_id: str, server_id: str) -> McpServerResponse:
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> McpServerResponse:
"""重新拉取 MCP server 工具清单并注册到运行时。""" """重新拉取 MCP server 工具清单并注册到运行时。"""
return await mcp_service.refresh_mcp_server_tools( return await mcp_service.refresh_mcp_server_tools(
server_id, server_id,
runtime=runtime,
process_manager=get_process_manager(), process_manager=get_process_manager(),
) )
+71
View File
@@ -0,0 +1,71 @@
"""用户面 API(神识阁数据源):BYOK 密钥管理与用量查询。
灵台是神识唯一调用方:本端点鉴权后以 ``X-Hunjing-User-Id`` 代理转发;
密钥明文只进不出(登记请求体 → 神识加密落库,响应永远脱敏)。
"""
from fastapi import APIRouter, Depends, Request, status
from app.api.v1.deps import get_current_user
from app.core.errors import AppError
from app.kernel import Kernel
from app.schemas.auth import CurrentUser
from app.schemas.me import (
ByokKeyListResponse,
MyUsageResponse,
RegisterByokKeyRequest,
)
from app.shenshi_client.client import ShenshiClient
from app.shenshi_client.types import ShenshiError
router = APIRouter(prefix="/me", tags=["me"])
def _shenshi(request: Request) -> ShenshiClient:
kernel: Kernel | None = getattr(request.app.state, "kernel", None)
if kernel is None:
raise AppError("service_unavailable", "Kernel 未就绪。", status_code=503)
return kernel.shenshi
@router.get("/byok-keys", response_model=ByokKeyListResponse)
async def list_byok_keys(
request: Request, user: CurrentUser = Depends(get_current_user)
) -> ByokKeyListResponse:
keys = await _shenshi(request).list_byok_keys(user.owner_scope)
return ByokKeyListResponse(keys=keys)
@router.post("/byok-keys", status_code=status.HTTP_201_CREATED)
async def register_byok_key(
payload: RegisterByokKeyRequest,
request: Request,
user: CurrentUser = Depends(get_current_user),
) -> dict:
key = await _shenshi(request).register_byok_key(
user.owner_scope,
name=payload.name,
type=payload.type,
base_url=payload.base_url,
api_key=payload.api_key,
model=payload.model,
)
return key
@router.delete("/byok-keys/{key_id}", status_code=status.HTTP_204_NO_CONTENT)
async def revoke_byok_key(
key_id: str, request: Request, user: CurrentUser = Depends(get_current_user)
) -> None:
try:
await _shenshi(request).revoke_byok_key(user.owner_scope, key_id)
except ShenshiError as exc:
raise AppError(exc.code, exc.message, status_code=404) from None
@router.get("/usage", response_model=MyUsageResponse)
async def my_usage(
request: Request, user: CurrentUser = Depends(get_current_user)
) -> MyUsageResponse:
data = await _shenshi(request).get_my_usage(user.owner_scope)
return MyUsageResponse(rows=data.get("rows", []), quota=data.get("quota"))
-122
View File
@@ -1,122 +0,0 @@
"""人格 API。"""
from fastapi import APIRouter, Depends, Query, status
from app.api.v1.deps import get_current_user
from app.schemas.auth import CurrentUser
from app.schemas.common import Page
from app.schemas.personality import (
PersonalityProposalResponse,
PersonalityResponse,
ProposePersonalityUpdateRequest,
)
from app.services.personality import (
approve_personality_proposal,
get_current_personality,
list_personality_proposals,
list_personality_versions,
propose_personality_update,
reject_personality_proposal,
rollback_personality,
)
router = APIRouter(prefix="/personality", tags=["personality"])
@router.get("/current", response_model=PersonalityResponse)
async def current_personality(
current_user: CurrentUser = Depends(get_current_user),
) -> PersonalityResponse:
return await get_current_personality(owner_scope=current_user.owner_scope)
@router.get("/versions", response_model=Page[PersonalityResponse])
async def personality_versions(
character_id: str | None = Query(default=None, description="人物 ID,省略取当前激活人物。"),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=20, ge=1, le=100),
current_user: CurrentUser = Depends(get_current_user),
) -> Page[PersonalityResponse]:
return await list_personality_versions(
character_id=character_id,
page=page,
page_size=page_size,
owner_scope=current_user.owner_scope,
)
@router.get("/proposals", response_model=Page[PersonalityProposalResponse])
async def personality_proposals(
character_id: str | None = Query(default=None, description="人物 ID,省略取当前激活人物。"),
status: str | None = Query(
default=None, description="按状态过滤:pending / approved / rejected / applied"
),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=20, ge=1, le=100),
current_user: CurrentUser = Depends(get_current_user),
) -> Page[PersonalityProposalResponse]:
"""人格修改建议列表(待审审批墙数据源)。"""
return await list_personality_proposals(
character_id=character_id,
status=status,
page=page,
page_size=page_size,
owner_scope=current_user.owner_scope,
)
# —— 人格修改建议与版本回滚(api-spec §11) ——
@router.post(
"/propose-update",
response_model=PersonalityProposalResponse,
status_code=status.HTTP_201_CREATED,
)
async def propose_personality_update_endpoint(
payload: ProposePersonalityUpdateRequest,
current_user: CurrentUser = Depends(get_current_user),
) -> PersonalityProposalResponse:
"""提交人格修改建议;进入 pending 待审核,不直接写入人格版本。"""
return await propose_personality_update(
character_id=payload.character_id,
fields=payload.fields,
reason=payload.reason,
owner_scope=current_user.owner_scope,
)
@router.post(
"/proposals/{proposal_id}/approve",
response_model=PersonalityProposalResponse,
)
async def approve_proposal(
proposal_id: str,
current_user: CurrentUser = Depends(get_current_user),
) -> PersonalityProposalResponse:
"""审核通过人格建议:派生新人格版本并切换人物当前指针。"""
return await approve_personality_proposal(proposal_id, owner_scope=current_user.owner_scope)
@router.post(
"/proposals/{proposal_id}/reject",
response_model=PersonalityProposalResponse,
)
async def reject_proposal(
proposal_id: str,
current_user: CurrentUser = Depends(get_current_user),
) -> PersonalityProposalResponse:
"""驳回人格建议:标记 rejected,不生成新版本。"""
return await reject_personality_proposal(proposal_id, owner_scope=current_user.owner_scope)
@router.post("/versions/{version_id}/rollback", response_model=PersonalityResponse)
async def rollback_personality_version(
version_id: str,
character_id: str | None = Query(default=None, description="人物 ID,省略取当前激活人物。"),
current_user: CurrentUser = Depends(get_current_user),
) -> PersonalityResponse:
"""回滚到指定人格版本;被取代的旧版本标记 rolled_back_at。"""
return await rollback_personality(
character_id=character_id, version_id=version_id, owner_scope=current_user.owner_scope
)
+4 -109
View File
@@ -1,32 +1,23 @@
"""Run 与 SSE API。""" """Run 与 SSE API(Run 的启动契约见 kernel_runs.py)。"""
import asyncio import asyncio
import json import json
from collections.abc import AsyncGenerator from collections.abc import AsyncGenerator
from typing import Any from typing import Any
from fastapi import APIRouter, Header, Query, Request from fastapi import APIRouter, Header, Query
from sse_starlette.sse import EventSourceResponse from sse_starlette.sse import EventSourceResponse
from app.application.agent.confirmation import ConfirmationRegistry
from app.application.agent.run_registry import AgentRunRegistry
from app.core.config import settings from app.core.config import settings
from app.core.errors import AppError from app.core.errors import AppError
from app.core.time import to_iso, utc_now from app.core.time import to_iso, utc_now
from app.db.session import AsyncSessionLocal from app.db.session import AsyncSessionLocal
from app.repositories import runtime as runtime_repo from app.repositories import runtime as runtime_repo
from app.schemas.common import Page from app.schemas.common import Page
from app.schemas.runtime import ( from app.schemas.runtime import EventResponse, RunResponse
AgentControlRequest,
AgentControlResponse,
ConfirmationResolutionResponse,
EventResponse,
RunResponse,
)
from app.services.event_bus import bus from app.services.event_bus import bus
from app.services.runtime import ( from app.services.runtime import (
cancel_run, cancel_run,
confirmation_exists_in_run,
get_run_response, get_run_response,
list_run_event_responses, list_run_event_responses,
list_runs_response, list_runs_response,
@@ -66,107 +57,11 @@ async def list_run_events(
return await list_run_event_responses(run_id, limit=limit, offset=offset) return await list_run_event_responses(run_id, limit=limit, offset=offset)
def _get_agent_run_registry(request: Request) -> AgentRunRegistry:
registry: AgentRunRegistry | None = getattr(request.app.state, "agent_run_registry", None)
if registry is None:
raise AppError("service_unavailable", "Agent 运行控制未启用。", status_code=503)
return registry
@router.post("/{run_id}/cancel", response_model=RunResponse) @router.post("/{run_id}/cancel", response_model=RunResponse)
async def cancel_run_api(run_id: str, request: Request) -> RunResponse: async def cancel_run_api(run_id: str) -> RunResponse:
registry = getattr(request.app.state, "agent_run_registry", None)
if registry is not None:
await registry.abort(run_id)
return await cancel_run(run_id) return await cancel_run(run_id)
@router.post("/{run_id}/steer", response_model=AgentControlResponse)
async def steer_run(
run_id: str,
payload: AgentControlRequest,
request: Request,
) -> AgentControlResponse:
if not _get_agent_run_registry(request).steer(run_id, payload.content):
raise AppError("conflict", "Run 当前不在本进程执行,无法 steer。", status_code=409)
return AgentControlResponse()
@router.post("/{run_id}/follow-up", response_model=AgentControlResponse)
async def follow_up_run(
run_id: str,
payload: AgentControlRequest,
request: Request,
) -> AgentControlResponse:
if not _get_agent_run_registry(request).follow_up(run_id, payload.content):
raise AppError("conflict", "Run 当前不在本进程执行,无法追加 follow-up。", status_code=409)
return AgentControlResponse()
def _get_confirmation_registry(request: Request) -> ConfirmationRegistry:
"""从 app.state 取 ConfirmationRegistry;未挂载时 503。"""
registry: ConfirmationRegistry | None = getattr(
request.app.state, "confirmation_registry", None
)
if registry is None:
raise AppError(
"service_unavailable",
"确认机制未启用(进程内 Future 注册表未挂载)",
status_code=503,
)
return registry
async def _resolve_confirmation(
request: Request,
run_id: str,
confirmation_id: str,
approved: bool,
) -> ConfirmationResolutionResponse:
"""approve/deny 共用逻辑:通过 ConfirmationRegistry 更新确认状态。"""
registry = _get_confirmation_registry(request)
if await registry.resolve(run_id, approved, confirmation_id=confirmation_id):
return ConfirmationResolutionResponse(approved=approved)
# 记录已处理/不存在——查 events 表区分 410 与 404
event_store = getattr(request.app.state, "event_store", None)
if event_store is not None and await confirmation_exists_in_run(
event_store, run_id, confirmation_id
):
raise AppError(
"confirmation_expired",
"确认已失效(服务可能已重启或已处理),请重新发起对话",
status_code=410,
)
raise AppError("confirmation_not_found", "确认项不存在", status_code=404)
@router.post(
"/{run_id}/confirmations/{confirmation_id}/approve",
response_model=ConfirmationResolutionResponse,
)
async def approve_confirmation(
run_id: str,
confirmation_id: str,
request: Request,
) -> ConfirmationResolutionResponse:
"""批准高风险工具执行:触发 Agent 循环恢复(confirmed=True 重试)。"""
return await _resolve_confirmation(request, run_id, confirmation_id, approved=True)
@router.post(
"/{run_id}/confirmations/{confirmation_id}/deny",
response_model=ConfirmationResolutionResponse,
)
async def deny_confirmation(
run_id: str,
confirmation_id: str,
request: Request,
) -> ConfirmationResolutionResponse:
"""拒绝高风险工具执行:Agent 循环回填拒绝 ToolMessage。"""
return await _resolve_confirmation(request, run_id, confirmation_id, approved=False)
@router.get("/{run_id}/stream") @router.get("/{run_id}/stream")
async def stream_run( async def stream_run(
run_id: str, run_id: str,
+7 -139
View File
@@ -1,162 +1,30 @@
"""安全 API(api-spec §16)。 """安全 API(§16):审计时间线查询。
audit-events 按人物/枢过滤查询审计时间线;confirmations 的 approve/deny 负责 工具策略治理与确认流已由百工域承接(/capabilities /mandates /petitions);
高风险工具确认:通过 ConfirmationRegistry 更新工具确认状态唤醒 Agent 循环。 本路由保留 audit-events(行动账时间线)。
""" """
from __future__ import annotations from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status from fastapi import APIRouter, Depends, Query
from app.api.v1.deps import get_confirmation_registry, get_optional_tool_runtime from app.api.v1.deps import get_current_user
from app.application.agent.confirmation import ConfirmationRegistry
from app.application.tool_runtime.runtime import ToolRuntime
from app.schemas.admin import AuditEntry from app.schemas.admin import AuditEntry
from app.schemas.auth import CurrentUser
from app.schemas.common import Page from app.schemas.common import Page
from app.schemas.safety import (
CreateToolPolicyRequest,
SafetyPolicyItem,
ToolPolicyResponse,
UpdateToolPolicyRequest,
)
from app.services import audit as audit_service from app.services import audit as audit_service
from app.services import governance as governance_service
router = APIRouter(prefix="/safety", tags=["safety"]) router = APIRouter(prefix="/safety", tags=["safety"])
# —— §16 审计事件查询 ——
@router.get("/audit-events", response_model=Page[AuditEntry]) @router.get("/audit-events", response_model=Page[AuditEntry])
async def list_audit_events( async def list_audit_events(
character_id: str | None = Query(default=None, description="按人物过滤。"), character_id: str | None = Query(default=None, description="按人物过滤。"),
pivot: str | None = Query(default=None, description="按枢过滤(契约字典 §5.9)。"), pivot: str | None = Query(default=None, description="按枢过滤(契约字典 §5.9)。"),
page: int = Query(default=1, ge=1), page: int = Query(default=1, ge=1),
page_size: int = Query(default=50, ge=1, le=200), page_size: int = Query(default=50, ge=1, le=200),
_user: CurrentUser = Depends(get_current_user),
) -> Page[AuditEntry]: ) -> Page[AuditEntry]:
return await audit_service.query_audit_events( return await audit_service.query_audit_events(
character_id=character_id, pivot=pivot, page=page, page_size=page_size character_id=character_id, pivot=pivot, page=page, page_size=page_size
) )
# —— §16 工具策略治理 ——
async def _refresh_governor(runtime: ToolRuntime | None) -> None:
"""策略写操作后刷新 ToolRuntime 的司命决策器;tool_runtime 未就绪则跳过(重启后从 db 加载)。"""
if runtime is not None:
await runtime.reload_governor()
@router.get("/policies", response_model=list[SafetyPolicyItem])
async def list_safety_policies() -> list[SafetyPolicyItem]:
return await governance_service.list_safety_policies()
@router.post("/policies", response_model=ToolPolicyResponse, status_code=status.HTTP_201_CREATED)
async def create_safety_policy(
payload: CreateToolPolicyRequest,
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> ToolPolicyResponse:
response = await governance_service.create_tool_policy(payload)
await _refresh_governor(runtime)
return response
@router.patch("/policies/{tool}", response_model=ToolPolicyResponse)
async def update_safety_policy(
tool: str,
payload: UpdateToolPolicyRequest,
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> ToolPolicyResponse:
response = await governance_service.update_tool_policy(tool, payload)
await _refresh_governor(runtime)
return response
@router.delete("/policies/{tool}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_safety_policy(
tool: str,
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> Response:
await governance_service.delete_tool_policy(tool)
await _refresh_governor(runtime)
return Response(status_code=status.HTTP_204_NO_CONTENT)
# —— §16 高风险工具确认 ——
def _not_found(confirmation_id: str) -> HTTPException:
return HTTPException(
status_code=404,
detail={
"error": {"code": "not_found", "message": f"确认不存在或已被消费: {confirmation_id}"}
},
)
def _cancelled(confirmation_id: str) -> dict[str, str]:
return {"status": "cancelled", "confirmation_id": confirmation_id}
@router.post("/confirmations/{confirmation_id}/approve")
async def approve(
confirmation_id: str,
registry: ConfirmationRegistry = Depends(get_confirmation_registry),
) -> dict[str, bool | str]:
"""批准工具执行。通过 ConfirmationRegistry 唤醒等待的 Agent 循环。"""
from sqlalchemy import select
from app.db.session import AsyncSessionLocal
from app.models.safety import PendingConfirmationRecord
async with AsyncSessionLocal() as session:
result = await session.execute(
select(PendingConfirmationRecord).where(PendingConfirmationRecord.id == confirmation_id)
)
record = result.scalar_one_or_none()
if record is None or record.run_id is None:
raise _not_found(confirmation_id)
# 更新 resolution 并唤醒进程内 Event
ok = await registry.resolve(
record.run_id,
approved=True,
confirmation_id=confirmation_id,
)
if not ok:
raise _not_found(confirmation_id)
return {"status": "resolved", "approved": True, "run_id": record.run_id}
@router.post("/confirmations/{confirmation_id}/deny")
async def deny(
confirmation_id: str,
registry: ConfirmationRegistry = Depends(get_confirmation_registry),
) -> dict[str, str]:
"""拒绝工具执行。通过 ConfirmationRegistry 通知 Agent 循环。"""
from sqlalchemy import select
from app.db.session import AsyncSessionLocal
from app.models.safety import PendingConfirmationRecord
async with AsyncSessionLocal() as session:
result = await session.execute(
select(PendingConfirmationRecord).where(PendingConfirmationRecord.id == confirmation_id)
)
record = result.scalar_one_or_none()
if record is None or record.run_id is None:
raise _not_found(confirmation_id)
ok = await registry.resolve(
record.run_id,
approved=False,
confirmation_id=confirmation_id,
)
if not ok:
raise _not_found(confirmation_id)
return {"status": "cancelled", "confirmation_id": confirmation_id}
+11 -8
View File
@@ -22,8 +22,9 @@ import mimetypes
from fastapi import APIRouter, Depends, HTTPException, Query from fastapi import APIRouter, Depends, HTTPException, Query
from starlette.responses import Response from starlette.responses import Response
from app.api.v1.deps import get_current_user, get_tool_runtime from app.api.v1.deps import get_current_user, get_sandbox_client
from app.application.tool_runtime.runtime import ToolRuntime from app.domains.baigong import files as baigong_files
from app.integrations.sandbox.client import SandboxClient
from app.integrations.sandbox.exceptions import SandboxError, SandboxNotFound from app.integrations.sandbox.exceptions import SandboxError, SandboxNotFound
from app.schemas.auth import CurrentUser from app.schemas.auth import CurrentUser
@@ -54,7 +55,7 @@ async def read_sandbox_file(
"write_file 写入的文件在 workspace/<character_id>/ 下," "write_file 写入的文件在 workspace/<character_id>/ 下,"
"传递 write_file 的 path 时保持 True 即可。", "传递 write_file 的 path 时保持 True 即可。",
), ),
runtime: ToolRuntime = Depends(get_tool_runtime), sandbox: SandboxClient = Depends(get_sandbox_client),
_current_user: CurrentUser = Depends(get_current_user), _current_user: CurrentUser = Depends(get_current_user),
) -> Response: ) -> Response:
"""读取沙箱内文件,以原始字节返回(MIME 类型由路径扩展名推导)。 """读取沙箱内文件,以原始字节返回(MIME 类型由路径扩展名推导)。
@@ -76,8 +77,9 @@ async def read_sandbox_file(
# 2) 从沙箱读取 # 2) 从沙箱读取
try: try:
result = await runtime.read_sandbox_file( result = await baigong_files.read_sandbox_file(
path=path, sandbox,
path,
run_id=run_id, run_id=run_id,
character_id=character_id, character_id=character_id,
isolated=isolated, isolated=isolated,
@@ -117,7 +119,7 @@ async def list_sandbox_files(
description="人物 ID。提供时自动应用 isolated_workspace 隔离前缀。", description="人物 ID。提供时自动应用 isolated_workspace 隔离前缀。",
), ),
isolated: bool = Query(True, description="是否自动补全隔离前缀"), isolated: bool = Query(True, description="是否自动补全隔离前缀"),
runtime: ToolRuntime = Depends(get_tool_runtime), sandbox: SandboxClient = Depends(get_sandbox_client),
_current_user: CurrentUser = Depends(get_current_user), _current_user: CurrentUser = Depends(get_current_user),
) -> list[dict[str, object]]: ) -> list[dict[str, object]]:
"""列出沙箱工作区目录内容。 """列出沙箱工作区目录内容。
@@ -126,8 +128,9 @@ async def list_sandbox_files(
适用于前端文件浏览器导航。 适用于前端文件浏览器导航。
""" """
try: try:
entries = await runtime.list_sandbox_dir( entries = await baigong_files.list_sandbox_dir(
path=path, sandbox,
path,
run_id=run_id, run_id=run_id,
character_id=character_id, character_id=character_id,
isolated=isolated, isolated=isolated,
+3 -38
View File
@@ -1,60 +1,25 @@
"""设置 API(模型网关状态 + 生成偏好)。 """设置 API(生成偏好)。
对应前端 settingsApi。模型网关为只读总览,生成偏好可读可写; 生成偏好可读可写;模型网关状态由神识承接(用户面见神识阁)。
真实密钥不进入响应。业务逻辑在 services/preferences(与系统配置 core/config/settings 区分)。 业务逻辑在 services/preferences(与系统配置 core/config/settings 区分)。
""" """
from fastapi import APIRouter, Depends from fastapi import APIRouter, Depends
from app.api.v1.deps import get_current_user from app.api.v1.deps import get_current_user
from app.core.errors import AppError
from app.schemas.auth import CurrentUser from app.schemas.auth import CurrentUser
from app.schemas.preferences import ( from app.schemas.preferences import (
GenerationPreferencesResponse, GenerationPreferencesResponse,
ModelEndpointStatus,
ModelGatewayStatus,
UpdateEndpointOverrideRequest,
UpdateGenerationPreferencesRequest, UpdateGenerationPreferencesRequest,
) )
from app.services.ai_provider_admin import update_endpoint_override
from app.services.preferences import ( from app.services.preferences import (
get_generation_preferences, get_generation_preferences,
get_provider_gateway,
update_generation_preferences, update_generation_preferences,
) )
router = APIRouter(prefix="/settings", tags=["settings"]) router = APIRouter(prefix="/settings", tags=["settings"])
@router.get("/models", response_model=ModelGatewayStatus)
async def read_provider_gateway() -> ModelGatewayStatus:
return get_provider_gateway()
@router.patch(
"/models/{model}/endpoints/{endpoint}",
response_model=ModelEndpointStatus,
)
async def update_model_endpoint(
model: str,
endpoint: str,
payload: UpdateEndpointOverrideRequest,
) -> ModelEndpointStatus:
"""更新 endpoint 覆盖(base_url / api_key / status / weight / priority,DB 优先 env)。
写后热重载 registry(reset_model_router)。响应脱敏,不回 base_url / api_key 原值。
"""
await update_endpoint_override(model, endpoint, payload)
gateway = get_provider_gateway()
for m in gateway.models:
if m.name != model:
continue
for ep in m.endpoints or []:
if ep.name == endpoint:
return ep
raise AppError("not_found", "endpoint 不存在。")
@router.get("/preferences", response_model=GenerationPreferencesResponse) @router.get("/preferences", response_model=GenerationPreferencesResponse)
async def read_generation_preferences( async def read_generation_preferences(
current_user: CurrentUser = Depends(get_current_user), current_user: CurrentUser = Depends(get_current_user),
-54
View File
@@ -1,54 +0,0 @@
"""Skill 能力包 API(/tools/skills,skills-design §6)。"""
from fastapi import APIRouter, Query, status
from app.schemas.skills import (
CreateSkillRequest,
SkillResponse,
UpdateSkillRequest,
)
from app.services import skills as skills_service
router = APIRouter(prefix="/tools/skills", tags=["skills"])
@router.post("", response_model=SkillResponse, status_code=status.HTTP_201_CREATED)
async def create_skill(payload: CreateSkillRequest) -> SkillResponse:
"""注册 Skill 能力包(name 唯一)。"""
return await skills_service.create_skill(
name=payload.name,
description=payload.description,
trigger=payload.trigger,
prompt_fragment=payload.prompt_fragment,
tool_bindings=payload.tool_bindings,
risk_level=payload.risk_level,
status=payload.status,
)
@router.get("", response_model=list[SkillResponse])
async def list_skills(
status: str | None = Query(default=None, description="按状态过滤:enabled/disabled"),
) -> list[SkillResponse]:
"""列举 Skill(可按 status 过滤)。"""
return await skills_service.list_skills(status=status)
@router.patch("/{skill_id}", response_model=SkillResponse)
async def update_skill(skill_id: str, payload: UpdateSkillRequest) -> SkillResponse:
"""更新 / 启用 / 禁用 Skill。"""
return await skills_service.update_skill(
skill_id,
description=payload.description,
trigger=payload.trigger,
prompt_fragment=payload.prompt_fragment,
tool_bindings=payload.tool_bindings,
risk_level=payload.risk_level,
status=payload.status,
)
@router.delete("/{skill_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_skill(skill_id: str) -> None:
"""删除 Skill。"""
await skills_service.delete_skill(skill_id)
+72
View File
@@ -0,0 +1,72 @@
"""道体铸魂与分灵 API(/characters/{id}/soul)。"""
from fastapi import APIRouter, Depends
from pydantic import BaseModel, Field
from app.api.v1.deps import get_current_user
from app.domains.jingsheng import soul
from app.schemas.auth import CurrentUser
soul_router = APIRouter(prefix="/characters/{character_id}/soul", tags=["jingsheng"])
class ForgingRequest(BaseModel):
anchor_changes: dict = Field(description="道体锚点变更(键:identity_baseline_json)")
reason: str = Field(min_length=1, max_length=2000)
class SplitRequest(BaseModel):
reason: str = Field(min_length=1, max_length=2000)
initial_anchor: dict = Field(default_factory=dict)
@soul_router.get("/forgings")
async def list_forgings(
character_id: str, user: CurrentUser = Depends(get_current_user)
) -> list[dict]:
return await soul.list_forgings(character_id, user.owner_scope)
@soul_router.post("/forgings", status_code=201)
async def propose_forging(
character_id: str, payload: ForgingRequest, user: CurrentUser = Depends(get_current_user)
) -> dict:
return await soul.propose_forging(
character_id=character_id,
owner_scope=user.owner_scope,
anchor_changes=payload.anchor_changes,
reason=payload.reason,
)
@soul_router.post("/forgings/{forging_id}/confirm")
async def confirm_forging(
character_id: str, forging_id: str, user: CurrentUser = Depends(get_current_user)
) -> dict:
return await soul.confirm_forging(forging_id, user.owner_scope)
@soul_router.post("/forgings/{forging_id}/reject")
async def reject_forging(
character_id: str, forging_id: str, user: CurrentUser = Depends(get_current_user)
) -> dict:
return await soul.reject_forging(forging_id, user.owner_scope)
@soul_router.post("/split", status_code=201)
async def split_soul(
character_id: str, payload: SplitRequest, user: CurrentUser = Depends(get_current_user)
) -> dict:
return await soul.split_soul(
character_id=character_id,
owner_scope=user.owner_scope,
reason=payload.reason,
initial_anchor=payload.initial_anchor,
)
@soul_router.get("/lineage")
async def get_lineage(
character_id: str, user: CurrentUser = Depends(get_current_user)
) -> list[dict]:
return await soul.get_lineage(character_id, user.owner_scope)
+5
View File
@@ -0,0 +1,5 @@
"""天工阁路由(域实现见 app/domains/tiangong/api.py)。"""
from app.domains.tiangong.api import task_router
__all__ = ["task_router"]
+5
View File
@@ -0,0 +1,5 @@
"""器谱路由(域实现见 app/domains/tianzai/api.py)。"""
from app.domains.tianzai.api import device_router
__all__ = ["device_router"]
+26 -241
View File
@@ -1,252 +1,37 @@
"""工具调用 API(api-spec §15)。 """工具 API(§15 兼容面):清单走百工能力目录,调用走 /capabilities/{id}/invoke。
- GET /tools:列出已注册工具策略元数据(从 tool_policies 表,含 executable 派生)。 旧 ToolRuntime/策略治理端点已随 M5 百工重写退役;本路由只保留
- POST /tools:注册自定义工具策略(百工枢器用治理声明,记审计)。 `/tools`(能力目录投影)与 `/tools/calls/{id}`(一炁事件详情)。
- GET /tools/calls/{tool_call_id}:单次工具调用事件详情。
- PATCH /tools/{tool_name}:编辑工具策略元数据(spec §15: risk_level 等)。
- DELETE /tools/{tool_name}:注销自定义工具(builtin 不可删)。
- POST /tools/{tool_name}/invoke:调用工具;高风险返回 requires_confirmation + confirmation_id,
交由 /safety/confirmations/{id}/approve 消费。
- POST /tools/{tool_name}/invoke/stream:流式调用(SSE),实时推送 tool.stdout/tool.stderr
增量与 tool.completed;司命 deny/ask-未确认/沙箱错误分别以 tool.failed / tool.requires_confirmation 结束流。
策略写操作(POST/PATCH/DELETE)后刷新 ToolRuntime 的司命决策器,使变更即时生效;
tool_runtime 未就绪时仅写库(重启后从 db 加载)。
""" """
from __future__ import annotations from fastapi import APIRouter, Depends
import json from app.api.v1.deps import get_current_user
from typing import Any from app.domains.baigong import service as baigong
from app.schemas.auth import CurrentUser
from fastapi import APIRouter, Depends, Response, status from app.schemas.safety import ToolCallDetail
from pydantic import BaseModel from app.services.runtime import get_tool_call_detail
from sse_starlette.sse import EventSourceResponse
from app.api.v1.deps import get_confirmation_store, get_optional_tool_runtime, get_tool_runtime
from app.application.tool_runtime.runtime import ToolRuntime
from app.db.session import AsyncSessionLocal
from app.integrations.sandbox.exceptions import SandboxError
from app.models.safety import PendingConfirmationRecord
from app.modules.safety.confirmation import (
ConfirmationStore,
PendingConfirmation,
new_confirmation_id,
)
from app.modules.safety.exceptions import PermissionDenied, RequiresConfirmation
from app.schemas.safety import (
CreateToolPolicyRequest,
ToolCallDetail,
ToolPolicyResponse,
UpdateToolPolicyRequest,
)
from app.schemas.skills import ToolCapability
from app.services import governance as governance_service
from app.services import skills as skills_service
router = APIRouter(prefix="/tools", tags=["tools"]) router = APIRouter(prefix="/tools", tags=["tools"])
class InvokeRequest(BaseModel): @router.get("")
input: dict[str, Any] async def list_tools(user: CurrentUser = Depends(get_current_user)) -> list[dict]:
run_id: str | None = None """能力目录投影(新链路以 /capabilities 为准;此处给旧消费方过渡读取)。"""
character_id: str | None = None caps = await baigong.list_capabilities(user.owner_scope)
trace_id: str | None = None return [
confirmed: bool = False {
"name": c["name"],
"description": c["description"],
class ToolResultPayload(BaseModel): "risk_level": max(c["risk_dims"].values(), default="L1"),
ok: bool "permission_policy": "ask" if c["consent_mode"] == "ask" else "allow",
exit_code: int | None = None "enabled": c["enabled"],
stdout: str = "" "source": c["source"],
stderr: str = "" }
duration_ms: int = 0 for c in caps
timeout: bool = False ]
truncated: bool = False
trace_id: str = ""
class CompletedResponse(BaseModel):
status: str = "completed"
result: ToolResultPayload
class RequiresConfirmationResponse(BaseModel):
status: str = "requires_confirmation"
confirmation_id: str
risk_level: str
reason: str
async def _sync_confirmation_record(pending: PendingConfirmation) -> None:
"""把 InMemoryConfirmationStore 的待确认记录同步持久化到 DB。"""
async with AsyncSessionLocal() as session:
record = PendingConfirmationRecord(
id=pending.confirmation_id,
tool=pending.tool,
input_json=json.dumps(pending.input) if pending.input else "{}",
risk_level=pending.risk_level,
reason=pending.reason or "",
run_id=pending.run_id,
character_id=pending.character_id,
trace_id=pending.trace_id,
status="pending",
)
session.add(record)
await session.commit()
# —— §15 工具策略治理 ——
@router.get("", response_model=list[ToolPolicyResponse])
async def list_tools() -> list[ToolPolicyResponse]:
return await governance_service.list_tool_policies()
@router.get("/capabilities", response_model=list[ToolCapability])
async def list_capabilities(
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> list[ToolCapability]:
"""统一列举所有能力(builtin / mcp / skill,含来源标记,skills-design §6)。"""
return await skills_service.list_all_capabilities(runtime)
@router.post("", response_model=ToolPolicyResponse, status_code=status.HTTP_201_CREATED)
async def register_tool(
payload: CreateToolPolicyRequest,
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> ToolPolicyResponse:
response = await governance_service.create_tool_policy(payload)
if runtime is not None:
await runtime.reload_governor()
return response
@router.get("/calls/{tool_call_id}", response_model=ToolCallDetail) @router.get("/calls/{tool_call_id}", response_model=ToolCallDetail)
async def get_tool_call(tool_call_id: str) -> ToolCallDetail: async def get_tool_call_detail(tool_call_id: str) -> ToolCallDetail:
return await governance_service.get_tool_call(tool_call_id) return await get_tool_call_detail(tool_call_id)
@router.patch("/{tool_name}", response_model=ToolPolicyResponse)
async def update_tool(
tool_name: str,
payload: UpdateToolPolicyRequest,
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> ToolPolicyResponse:
response = await governance_service.update_tool_policy(tool_name, payload)
if runtime is not None:
await runtime.reload_governor()
return response
@router.delete("/{tool_name}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_tool(
tool_name: str,
runtime: ToolRuntime | None = Depends(get_optional_tool_runtime),
) -> Response:
await governance_service.delete_tool_policy(tool_name)
if runtime is not None:
await runtime.reload_governor()
return Response(status_code=status.HTTP_204_NO_CONTENT)
# —— §15 工具调用 ——
@router.post("/{tool_name}/invoke", response_model=None)
async def invoke_tool(
tool_name: str,
body: InvokeRequest,
runtime: ToolRuntime = Depends(get_tool_runtime),
confirmations: ConfirmationStore = Depends(get_confirmation_store),
) -> CompletedResponse | RequiresConfirmationResponse:
try:
result = await runtime.execute(
tool_name,
body.input,
run_id=body.run_id,
character_id=body.character_id,
trace_id=body.trace_id,
confirmed=body.confirmed,
)
except RequiresConfirmation as exc:
pending = await confirmations.create(
PendingConfirmation(
confirmation_id=new_confirmation_id(),
tool=tool_name,
input=body.input,
risk_level=exc.risk_level or "L2",
reason=exc.message,
run_id=body.run_id,
character_id=body.character_id,
trace_id=body.trace_id,
)
)
# 同步持久化 PendingConfirmationRecord(供 safety/confirmations 新端点查找)
await _sync_confirmation_record(pending)
return RequiresConfirmationResponse(
confirmation_id=pending.confirmation_id,
risk_level=pending.risk_level,
reason=pending.reason,
)
return CompletedResponse(
result=ToolResultPayload(
ok=result.ok,
exit_code=result.exit_code,
stdout=result.stdout,
stderr=result.stderr,
duration_ms=result.duration_ms,
timeout=result.timeout,
truncated=result.truncated,
trace_id=result.trace_id,
)
)
@router.post("/{tool_name}/invoke/stream")
async def invoke_tool_stream(
tool_name: str,
body: InvokeRequest,
runtime: ToolRuntime = Depends(get_tool_runtime),
) -> EventSourceResponse:
"""流式调用工具(SSE)。
事件:``tool.stdout`` / ``tool.stderr``(data 为 ``{"data":"<base64>"}``,前端解码)、
``tool.completed``(data 为结束汇总 JSON)。司命 deny → ``tool.failed``;
ask 且未确认 → ``tool.requires_confirmation``(前端应先走非流式 invoke 完成 confirmation,
再以 confirmed=True 调用本端点);沙箱错误 → ``tool.failed``。
"""
async def event_gen():
try:
async for ev in runtime.execute_stream(
tool_name,
body.input,
run_id=body.run_id,
character_id=body.character_id,
trace_id=body.trace_id,
confirmed=body.confirmed,
):
if ev.kind in ("stdout", "stderr"):
yield {"event": f"tool.{ev.kind}", "data": json.dumps({"data": ev.payload})}
elif ev.kind == "done":
yield {"event": "tool.completed", "data": json.dumps(ev.payload)}
except RequiresConfirmation as exc:
yield {
"event": "tool.requires_confirmation",
"data": json.dumps({"risk_level": exc.risk_level, "reason": exc.message}),
}
except PermissionDenied as exc:
yield {
"event": "tool.failed",
"data": json.dumps({"error_code": "permission_denied", "error_message": str(exc)}),
}
except SandboxError as exc:
yield {
"event": "tool.failed",
"data": json.dumps({"error_code": exc.code, "error_message": str(exc)}),
}
return EventSourceResponse(event_gen())
-95
View File
@@ -1,95 +0,0 @@
"""世界模型 API(河图枢运行时,api-spec §12)。"""
from fastapi import APIRouter, Depends, Query
from app.api.v1.deps import get_current_user
from app.schemas.auth import CurrentUser
from app.schemas.common import Page
from app.schemas.world import (
WorldEntityCreateRequest,
WorldEntityResponse,
WorldGraphResponse,
WorldRelationCreateRequest,
WorldRelationResponse,
)
from app.services.world import (
create_entity,
create_relation,
delete_relation,
get_entity_by_id,
get_graph,
list_entities_page,
list_relations_page,
)
router = APIRouter(prefix="/world", tags=["world"])
@router.get("/entities", response_model=Page[WorldEntityResponse])
async def list_entities(
current_user: CurrentUser = Depends(get_current_user),
character_id: str | None = Query(default=None, description="所属人物;省略取当前激活。"),
type: str | None = Query(default=None, description="按实体类型过滤(契约字典 §11.1)。"),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=20, ge=1, le=100),
) -> Page[WorldEntityResponse]:
return await list_entities_page(
character_id,
entity_type=type,
page=page,
page_size=page_size,
owner_scope=current_user.owner_scope,
)
@router.post("/entities", response_model=WorldEntityResponse)
async def create_entity_api(
payload: WorldEntityCreateRequest,
current_user: CurrentUser = Depends(get_current_user),
) -> WorldEntityResponse:
return await create_entity(payload, owner_scope=current_user.owner_scope)
@router.get("/entities/{entity_id}", response_model=WorldEntityResponse)
async def get_entity(
entity_id: str,
current_user: CurrentUser = Depends(get_current_user),
) -> WorldEntityResponse:
return await get_entity_by_id(entity_id, owner_scope=current_user.owner_scope)
@router.get("/relations", response_model=Page[WorldRelationResponse])
async def list_relations(
current_user: CurrentUser = Depends(get_current_user),
character_id: str | None = Query(default=None, description="所属人物;省略取当前激活。"),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=20, ge=1, le=100),
) -> Page[WorldRelationResponse]:
return await list_relations_page(
character_id, page=page, page_size=page_size, owner_scope=current_user.owner_scope
)
@router.post("/relations", response_model=WorldRelationResponse)
async def create_relation_api(
payload: WorldRelationCreateRequest,
current_user: CurrentUser = Depends(get_current_user),
) -> WorldRelationResponse:
return await create_relation(payload, owner_scope=current_user.owner_scope)
@router.delete("/relations/{relation_id}", status_code=204)
async def delete_relation_api(
relation_id: str,
current_user: CurrentUser = Depends(get_current_user),
) -> None:
await delete_relation(relation_id, owner_scope=current_user.owner_scope)
@router.get("/graph", response_model=WorldGraphResponse)
async def world_graph(
current_user: CurrentUser = Depends(get_current_user),
character_id: str | None = Query(default=None, description="所属人物;省略取当前激活。"),
) -> WorldGraphResponse:
"""返回人物的完整世界图谱(无分页),供前端 D3/Force 图谱渲染。"""
return await get_graph(character_id, owner_scope=current_user.owner_scope)
+5
View File
@@ -0,0 +1,5 @@
"""形仪路由(域实现见 app/domains/xingyi/api.py)。"""
from app.domains.xingyi.api import form_router
__all__ = ["form_router"]
+5
View File
@@ -0,0 +1,5 @@
"""缘契路由(域实现见 app/domains/yuanqi/api.py)。"""
from app.domains.yuanqi.api import cov_router
__all__ = ["cov_router"]
-82
View File
@@ -1,82 +0,0 @@
"""非流式 Agent 应用层:工具调用闭环编排。
入口:
- ``run_agent_turn`` 执行一轮带工具调用的对话循环
- ``AgentTurnResult`` 循环产出
- ``to_ai_provider_tools`` 把 ToolSpec 转成 OpenAI function schema dict
- ``default_tools`` 返回全部内置工具(依赖司命 + 确认流控风险)
"""
from app.application.agent.agent import (
Agent,
AgentConfig,
AgentState,
AgentStatus,
PendingMessageQueue,
QueueMode,
)
from app.application.agent.confirmation import ConfirmationRegistry
from app.application.agent.extensions import (
AfterToolCallDecision,
AgentContext,
AgentEvent,
AgentRunContext,
BeforeToolCallDecision,
CancellationToken,
ExtensionRegistry,
ToolCallContext,
TurnSnapshot,
)
from app.application.agent.prompt_composer import compose_system_messages
from app.application.agent.python_skills import (
PythonSkillActivation,
PythonSkillCatalog,
PythonSkillDefinition,
activate_python_skills,
discover_python_skills,
)
from app.application.agent.run_registry import AgentRunRegistry
from app.application.agent.runtime import AgentTurnResult, run_agent_turn
from app.application.agent.skill_loader import load_skill_manifests
from app.application.agent.tasks import (
next_step_for_tool,
step_event_payload,
task_from_deliberative_plan,
task_plan_payload,
)
from app.application.agent.tools import default_tools, to_ai_provider_tools
__all__ = [
"AfterToolCallDecision",
"Agent",
"AgentConfig",
"AgentContext",
"AgentEvent",
"AgentRunContext",
"AgentRunRegistry",
"AgentState",
"AgentStatus",
"AgentTurnResult",
"BeforeToolCallDecision",
"CancellationToken",
"ConfirmationRegistry",
"ExtensionRegistry",
"PendingMessageQueue",
"PythonSkillActivation",
"PythonSkillCatalog",
"PythonSkillDefinition",
"QueueMode",
"ToolCallContext",
"TurnSnapshot",
"activate_python_skills",
"compose_system_messages",
"default_tools",
"discover_python_skills",
"load_skill_manifests",
"next_step_for_tool",
"run_agent_turn",
"step_event_payload",
"task_from_deliberative_plan",
"task_plan_payload",
"to_ai_provider_tools",
]
-303
View File
@@ -1,303 +0,0 @@
"""Pi agent-core 风格的有状态 Agent 门面。"""
from __future__ import annotations
import asyncio
from collections.abc import Awaitable, Callable
from contextlib import suppress
from dataclasses import dataclass, field, replace
from enum import StrEnum
from typing import Any
from app.ai_provider.types.chat import ChatChunk, TokenUsage
from app.ai_provider.types.messages import BaseMessage, HumanMessage
from app.application.agent.confirmation import ConfirmationRegistry
from app.application.agent.extensions import (
AgentEvent,
AgentRunContext,
CancellationToken,
ExtensionRegistry,
)
from app.application.agent.runtime import AgentTurnResult, run_agent_turn
from app.modules.tool_runtime.definition import ToolRuntimeLike
class AgentStatus(StrEnum):
idle = "idle"
running = "running"
completed = "completed"
failed = "failed"
cancelled = "cancelled"
class QueueMode(StrEnum):
all = "all"
one_at_a_time = "one-at-a-time"
@dataclass
class AgentState:
"""有状态 Agent 的进程内快照。"""
messages: list[BaseMessage] = field(default_factory=list)
status: AgentStatus = AgentStatus.idle
streaming_message: str = ""
pending_tool_calls: set[str] = field(default_factory=set)
error_message: str | None = None
usage: TokenUsage = field(default_factory=TokenUsage)
class PendingMessageQueue:
"""steering/follow-up 共用的小型 FIFO。"""
def __init__(self, mode: QueueMode = QueueMode.one_at_a_time) -> None:
self.mode = mode
self._messages: list[BaseMessage] = []
def put(self, message: BaseMessage) -> None:
self._messages.append(message)
def drain(self) -> list[BaseMessage]:
if not self._messages:
return []
if self.mode is QueueMode.all:
messages, self._messages = self._messages, []
return messages
return [self._messages.pop(0)]
def clear(self) -> None:
self._messages.clear()
def __bool__(self) -> bool:
return bool(self._messages)
@dataclass(frozen=True)
class AgentConfig:
"""Stateful Agent 的稳定运行配置。"""
tools: tuple[dict[str, Any], ...]
tool_runtime: ToolRuntimeLike | None
model_hint: str | None = None
thinking_enabled: bool | None = None
reasoning_effort: str | None = None
max_iterations: int = 8
confirmation_registry: ConfirmationRegistry | None = None
confirmation_timeout: float = 300.0
on_turn: Callable[[], Awaitable[None]] | None = None
on_chunk: Callable[[ChatChunk], Awaitable[None]] | None = None
on_tool_event: Callable[[str, dict[str, Any]], Awaitable[None]] | None = None
on_confirm_timeout: Callable[[str], Awaitable[None]] | None = None
class Agent:
"""管理状态、消息队列、取消与扩展 Hook;低层循环仍由 run_agent_turn 负责。"""
def __init__(
self,
config: AgentConfig,
*,
messages: list[BaseMessage] | None = None,
run_context: AgentRunContext | None = None,
extensions: ExtensionRegistry | None = None,
runner: Callable[..., Awaitable[AgentTurnResult]] = run_agent_turn,
) -> None:
self.config = config
self.state = AgentState(messages=list(messages or []))
self.run_context = run_context or AgentRunContext()
self.extensions = extensions or ExtensionRegistry()
self._runner = runner
self.steering = PendingMessageQueue()
self.follow_ups = PendingMessageQueue()
self._active_task: asyncio.Task[AgentTurnResult] | None = None
self._accepting_messages = False
self._idle = asyncio.Event()
self._idle.set()
def steer(self, message: str | BaseMessage) -> bool:
if not self._accepting_messages:
return False
self.steering.put(self._as_message(message))
return True
def follow_up(self, message: str | BaseMessage) -> bool:
if not self._accepting_messages:
return False
self.follow_ups.put(self._as_message(message))
return True
async def prompt(self, message: str | BaseMessage) -> AgentTurnResult:
if self.state.status is AgentStatus.running:
raise RuntimeError("Agent 正在运行,请使用 steer 或 follow_up")
self.state.messages.append(self._as_message(message))
return await self._run()
async def continue_run(self) -> AgentTurnResult:
if self.state.status is AgentStatus.running:
raise RuntimeError("Agent 正在运行")
return await self._run()
async def abort(self) -> bool:
task = self._active_task
if task is None or task.done():
return False
self.run_context.cancellation.cancel()
task.cancel()
return True
async def wait_for_idle(self) -> None:
await self._idle.wait()
async def reset(self) -> None:
await self.abort()
await self.wait_for_idle()
self.state = AgentState()
self.steering.clear()
self.follow_ups.clear()
async def _run(self) -> AgentTurnResult:
if self.run_context.cancellation.cancelled:
self.run_context = replace(self.run_context, cancellation=CancellationToken())
self.state.status = AgentStatus.running
self.state.streaming_message = ""
self.state.pending_tool_calls.clear()
self.state.error_message = None
self._accepting_messages = True
self._idle.clear()
try:
await self.extensions.publish(AgentEvent("agent.started"), self.run_context)
task = asyncio.create_task(self._run_loop())
self._active_task = task
result = await task
self._accepting_messages = False
self.state.status = AgentStatus.completed
self.state.streaming_message = result.content
self.state.usage = result.usage
await self.extensions.publish(
AgentEvent("agent.ended", {"iterations": result.iterations}), self.run_context
)
return result
except asyncio.CancelledError:
self._accepting_messages = False
self.state.status = AgentStatus.cancelled
await self.extensions.publish(AgentEvent("agent.cancelled"), self.run_context)
raise
except Exception as exc:
self._accepting_messages = False
self.state.status = AgentStatus.failed
self.state.error_message = str(exc)
with suppress(Exception):
await self.extensions.publish(
AgentEvent("agent.failed", {"error": str(exc)}), self.run_context
)
raise
finally:
self._accepting_messages = False
with suppress(Exception):
await self.extensions.publish(
AgentEvent("agent.settled", {"status": self.state.status.value}),
self.run_context,
)
self._active_task = None
self._idle.set()
async def _run_loop(self) -> AgentTurnResult:
remaining = self.config.max_iterations
total_iterations = 0
total_usage = TokenUsage()
total_tool_history: list[dict[str, Any]] = []
final = AgentTurnResult(content="", reasoning="")
while remaining > 0:
working_messages = list(self.state.messages)
final = await self._runner(
messages=working_messages,
tools=list(self.config.tools),
tool_runtime=self.config.tool_runtime,
model_hint=self.config.model_hint,
thinking_enabled=self.config.thinking_enabled,
reasoning_effort=self.config.reasoning_effort,
max_iterations=remaining,
run_id=self.run_context.run_id,
character_id=self.run_context.character_id,
trace_id=self.run_context.trace_id,
on_turn=self.config.on_turn,
on_chunk=self._on_chunk,
on_tool_event=self._on_tool_event,
confirmation_registry=self.config.confirmation_registry,
confirmation_timeout=self.config.confirmation_timeout,
on_confirm_timeout=self.config.on_confirm_timeout,
extensions=self.extensions,
run_context=self.run_context,
get_steering_messages=self.steering.drain,
)
self.state.messages = working_messages
total_iterations += final.iterations
total_usage += final.usage
total_tool_history.extend(final.tool_history)
remaining -= final.iterations
if final.truncated:
return AgentTurnResult(
content=final.content,
reasoning=final.reasoning,
tool_history=total_tool_history,
iterations=total_iterations,
truncated=True,
usage=total_usage,
)
if remaining <= 0:
return AgentTurnResult(
content=final.content,
reasoning=final.reasoning,
tool_history=total_tool_history,
iterations=total_iterations,
truncated=bool(self.follow_ups),
usage=total_usage,
)
follow_ups = self.follow_ups.drain()
if not follow_ups:
return AgentTurnResult(
content=final.content,
reasoning=final.reasoning,
tool_history=total_tool_history,
iterations=total_iterations,
truncated=final.truncated,
usage=total_usage,
)
self.state.messages.extend(follow_ups)
return final
async def _on_chunk(self, chunk: ChatChunk) -> None:
if chunk.delta_content:
self.state.streaming_message += chunk.delta_content
if self.config.on_chunk is not None:
await self.config.on_chunk(chunk)
async def _on_tool_event(self, event_type: str, payload: dict[str, Any]) -> None:
call_id = str(payload.get("tool_call_id", ""))
if event_type == "tool.started" and call_id:
self.state.pending_tool_calls.add(call_id)
elif event_type in ("tool.completed", "tool.failed") and call_id:
self.state.pending_tool_calls.discard(call_id)
# 先转发 SSE(用户可见流优先),再把同一事件发布进扩展事件流,
# 让 AgentEventListener 也能观测工具级生命周期。
# 扩展 listener 异常被吞掉,避免观测副菜阻断工具事件主流程。
if self.config.on_tool_event is not None:
await self.config.on_tool_event(event_type, payload)
with suppress(Exception):
await self.extensions.publish(
AgentEvent(event_type, payload), self.run_context
)
@staticmethod
def _as_message(message: str | BaseMessage) -> BaseMessage:
return HumanMessage(content=message) if isinstance(message, str) else message
__all__ = [
"Agent",
"AgentConfig",
"AgentState",
"AgentStatus",
"PendingMessageQueue",
"QueueMode",
]
@@ -1,148 +0,0 @@
"""混合确认注册表 — 同进程 Event 唤醒 + 重启后 DB polling 兜底。
Agent 循环在工具需要确认时:
1. 写 PendingConfirmationRecord 到 DB(持久化挂起状态)
2. 调 ``ConfirmationRegistry.wait(run_id)`` — 同进程用 Event 0 延迟唤醒,
重启后 polling DB 兜底
approve/deny 端点统一调 ``registry.resolve(run_id, approved)``:
- 更新 PendingConfirmationRecord.status + resolved_at
- 同一进程直接 set Event
"""
from __future__ import annotations
import asyncio
import time
from datetime import UTC, datetime
from sqlalchemy import or_, select, update
from app.models.safety import PendingConfirmationRecord
class ConfirmationRegistry:
"""混合确认注册表。
- 同进程:``asyncio.Event`` 直接唤醒,0 延迟
- 重启后:Event 不存在 → polling ``PendingConfirmationRecord.status`` 兜底
- approve/deny 端点统一调用 ``resolve(run_id, approved)``
"""
def __init__(self) -> None:
self._events: dict[str, asyncio.Event] = {}
async def wait(
self,
run_id: str,
*,
poll_interval: float = 0.5,
timeout: float = 60,
confirmation_id: str | None = None,
) -> bool:
"""等待确认。
同一进程的事件直接唤醒;跨进程或重启后 polling 兜底。
``confirmation_id`` 可选,传入时会同时注册为该 Event 的别名,
使得外部以 ``confirmation_id`` 调 ``resolve`` 时也能立即唤醒同进程的 Event
(无需等 DB polling 兜底)。
返回:
True=批准, False=拒绝 ; 超时抛出 TimeoutError
"""
event = asyncio.Event()
self._events[run_id] = event
if confirmation_id:
self._events[confirmation_id] = event
try:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
# 1) polling DB 获取真实状态(Event 只用于唤醒,不携带批准/拒绝值)
from app.db.session import AsyncSessionLocal
async with AsyncSessionLocal() as session:
if confirmation_id is None:
stmt = select(PendingConfirmationRecord).where(
or_(
PendingConfirmationRecord.id == run_id,
PendingConfirmationRecord.run_id == run_id,
)
)
else:
stmt = select(PendingConfirmationRecord).where(
PendingConfirmationRecord.id == confirmation_id,
PendingConfirmationRecord.run_id == run_id,
)
result = await session.execute(stmt)
record = result.scalar_one_or_none()
if record is not None and record.status != "pending":
return record.status == "approved"
# 2) 等一个间隔,同时监听 Event(Event set 只表示「有变化」,实际值靠 DB)
try:
await asyncio.wait_for(
asyncio.shield(event.wait()),
timeout=poll_interval,
)
# Event 被唤醒 → 回到循环顶部重新查 DB
continue
except TimeoutError:
continue
# 超时
raise TimeoutError(f"确认超时(等待 {timeout:.0f}s 未裁决)")
finally:
self._events.pop(run_id, None)
if confirmation_id:
self._events.pop(confirmation_id, None)
async def resolve(
self,
run_id: str,
approved: bool,
*,
confirmation_id: str | None = None,
) -> bool:
"""精确裁决一个待确认工具调用,并唤醒对应等待者。"""
from app.db.session import AsyncSessionLocal
target_id = confirmation_id or run_id
async with AsyncSessionLocal() as session:
status = "approved" if approved else "denied"
stmt = update(PendingConfirmationRecord).where(
PendingConfirmationRecord.status == "pending"
)
if confirmation_id is None:
stmt = stmt.where(
or_(
PendingConfirmationRecord.id == run_id,
PendingConfirmationRecord.run_id == run_id,
)
)
else:
stmt = stmt.where(
PendingConfirmationRecord.id == confirmation_id,
PendingConfirmationRecord.run_id == run_id,
)
result = await session.execute(
stmt.values(status=status, resolved_at=datetime.now(UTC))
)
await session.commit()
if result.rowcount == 0:
return False
event = self._events.get(target_id) or self._events.get(run_id)
if event is not None:
event.set()
return True
def has(self, run_id: str) -> bool:
"""是否仍在等待(同进程检查)。"""
event = self._events.get(run_id)
return event is not None and not event.is_set()
__all__ = ["ConfirmationRegistry"]
-229
View File
@@ -1,229 +0,0 @@
"""Agent 扩展契约与注册表。
扩展只接触本次运行的快照与 ToolResult,不持有数据库会话、密钥或安全治理对象。
"""
from __future__ import annotations
import asyncio
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field, replace
from typing import Any
from app.ai_provider.types.chat import TokenUsage
from app.ai_provider.types.messages import BaseMessage
from app.modules.tool_runtime.definition import ToolResult
class CancellationToken:
"""协作式取消信号;阻塞 await 仍由活动 asyncio.Task.cancel() 负责中断。"""
def __init__(self) -> None:
self._cancelled = False
@property
def cancelled(self) -> bool:
return self._cancelled
def cancel(self) -> None:
self._cancelled = True
def raise_if_cancelled(self) -> None:
if self._cancelled:
raise asyncio.CancelledError
@dataclass(frozen=True)
class AgentRunContext:
"""扩展可见的最小运行上下文。"""
run_id: str | None = None
session_id: str | None = None
character_id: str | None = None
trace_id: str | None = None
metadata: dict[str, Any] = field(default_factory=dict)
cancellation: CancellationToken = field(default_factory=CancellationToken)
@dataclass(frozen=True)
class AgentContext:
"""每轮发送给模型前可转换的上下文快照。"""
messages: tuple[BaseMessage, ...]
tools: tuple[dict[str, Any], ...]
model_hint: str | None = None
@dataclass(frozen=True)
class AgentEvent:
"""Agent Core 内部生命周期事件。"""
type: str
payload: dict[str, Any] = field(default_factory=dict)
@dataclass(frozen=True)
class ToolCallContext:
"""工具调用 Hook 的稳定输入。"""
tool_call_id: str
name: str
input: dict[str, Any]
messages: tuple[BaseMessage, ...]
@dataclass(frozen=True)
class BeforeToolCallDecision:
"""执行前决策;扩展只能收紧或改参,不能绕过 ToolRuntime。"""
allowed: bool = True
reason: str | None = None
input: dict[str, Any] | None = None
@dataclass(frozen=True)
class AfterToolCallDecision:
"""执行后结果覆盖与提前终止提示。"""
result: ToolResult | None = None
terminate: bool = False
@dataclass(frozen=True)
class TurnSnapshot:
"""一轮模型调用及其工具执行结果摘要。"""
iteration: int
content: str
reasoning: str
tool_calls: tuple[dict[str, Any], ...]
tool_results: tuple[dict[str, Any], ...]
usage: TokenUsage = field(default_factory=TokenUsage)
ContextTransformer = Callable[[AgentContext, AgentRunContext], Awaitable[AgentContext]]
BeforeToolCallHook = Callable[
[ToolCallContext, AgentRunContext], Awaitable[BeforeToolCallDecision | None]
]
AfterToolCallHook = Callable[
[ToolCallContext, ToolResult, AgentRunContext], Awaitable[AfterToolCallDecision | None]
]
TurnPolicy = Callable[[TurnSnapshot, AgentRunContext], Awaitable[bool]]
AgentEventListener = Callable[[AgentEvent, AgentRunContext], Awaitable[None]]
class ExtensionRegistry:
"""受约束的 Agent 扩展注册表;按注册顺序执行 Hook。"""
def __init__(self) -> None:
self._context_transformers: list[ContextTransformer] = []
self._before_tool_hooks: list[BeforeToolCallHook] = []
self._after_tool_hooks: list[AfterToolCallHook] = []
self._turn_policies: list[TurnPolicy] = []
self._event_listeners: list[AgentEventListener] = []
def add_context_transformer(self, transformer: ContextTransformer) -> None:
self._context_transformers.append(transformer)
def add_before_tool_hook(self, hook: BeforeToolCallHook) -> None:
self._before_tool_hooks.append(hook)
def add_after_tool_hook(self, hook: AfterToolCallHook) -> None:
self._after_tool_hooks.append(hook)
def add_turn_policy(self, policy: TurnPolicy) -> None:
self._turn_policies.append(policy)
def add_event_listener(self, listener: AgentEventListener) -> Callable[[], None]:
self._event_listeners.append(listener)
def unsubscribe() -> None:
if listener in self._event_listeners:
self._event_listeners.remove(listener)
return unsubscribe
def extend(self, registry: ExtensionRegistry) -> None:
"""按原注册顺序合并另一组扩展。"""
self._context_transformers.extend(registry._context_transformers)
self._before_tool_hooks.extend(registry._before_tool_hooks)
self._after_tool_hooks.extend(registry._after_tool_hooks)
self._turn_policies.extend(registry._turn_policies)
self._event_listeners.extend(registry._event_listeners)
async def transform_context(
self, context: AgentContext, run: AgentRunContext
) -> AgentContext:
for transformer in self._context_transformers:
run.cancellation.raise_if_cancelled()
context = await transformer(context, run)
return context
async def before_tool_call(
self, call: ToolCallContext, run: AgentRunContext
) -> BeforeToolCallDecision:
current = call
for hook in self._before_tool_hooks:
run.cancellation.raise_if_cancelled()
decision = await hook(current, run)
if decision is None:
continue
if not decision.allowed:
return decision
if decision.input is not None:
current = replace(current, input=decision.input)
return BeforeToolCallDecision(input=current.input)
async def after_tool_call(
self,
call: ToolCallContext,
result: ToolResult,
run: AgentRunContext,
) -> AfterToolCallDecision:
current = result
terminate = False
for hook in self._after_tool_hooks:
run.cancellation.raise_if_cancelled()
decision = await hook(call, current, run)
if decision is None:
continue
if decision.result is not None:
current = decision.result
terminate = terminate or decision.terminate
return AfterToolCallDecision(result=current, terminate=terminate)
async def should_stop(self, turn: TurnSnapshot, run: AgentRunContext) -> bool:
for policy in self._turn_policies:
run.cancellation.raise_if_cancelled()
if await policy(turn, run):
return True
return False
async def publish(self, event: AgentEvent, run: AgentRunContext) -> None:
for listener in tuple(self._event_listeners):
await listener(event, run)
@property
def enabled(self) -> bool:
return any(
(
self._context_transformers,
self._before_tool_hooks,
self._after_tool_hooks,
self._turn_policies,
self._event_listeners,
)
)
__all__ = [
"AfterToolCallDecision",
"AgentContext",
"AgentEvent",
"AgentRunContext",
"BeforeToolCallDecision",
"CancellationToken",
"ExtensionRegistry",
"ToolCallContext",
"TurnSnapshot",
]
@@ -1,21 +0,0 @@
"""通用 Agent 提示组合。"""
from __future__ import annotations
from app.modules.agent import AgentDefinition
from app.modules.skills import SkillManifest
def compose_system_messages(
base_messages: list[str],
agent: AgentDefinition | None,
skills: list[SkillManifest],
) -> list[str]:
"""按稳定基础提示、Agent 身份、技能说明的顺序组合提示。"""
messages = list(base_messages)
if agent is not None and agent.system_prompt:
messages.append(f"【Agent:{agent.name}】\n{agent.system_prompt}")
for skill in skills:
if skill.prompt:
messages.append(f"【{skill.name}·能力注入】\n{skill.prompt}")
return messages
@@ -1,132 +0,0 @@
"""受信任 Python Skill 的发现、匹配与 Run 级激活。"""
from __future__ import annotations
import importlib
import inspect
import pkgutil
from collections.abc import Awaitable, Callable, Iterable
from dataclasses import dataclass
from types import ModuleType
from typing import Any
from app.application.agent.extensions import ExtensionRegistry
SkillCleanup = Callable[[], Awaitable[None] | None]
SkillSetup = Callable[[ExtensionRegistry], SkillCleanup | None]
SkillMatcher = Callable[[str, dict[str, Any]], bool]
@dataclass(frozen=True)
class PythonSkillDefinition:
"""部署期 Python Skill 契约。"""
name: str
description: str
triggers: tuple[str, ...]
instructions: str
setup: SkillSetup
version: str = "1"
enabled: bool = True
match: SkillMatcher | None = None
@dataclass
class PythonSkillActivation:
"""单个 Run 的 Skill 激活结果。"""
skills: tuple[PythonSkillDefinition, ...]
cleanups: list[SkillCleanup]
async def close(self) -> None:
"""按激活逆序释放 Skill 资源。"""
while self.cleanups:
result = self.cleanups.pop()()
if inspect.isawaitable(result):
await result
class PythonSkillCatalog:
"""只保存 Skill 描述与 setup,不持有 Run 状态。"""
def __init__(self, skills: Iterable[PythonSkillDefinition] = ()) -> None:
self._skills: dict[str, PythonSkillDefinition] = {}
for skill in skills:
if skill.name in self._skills:
raise ValueError(f"Python Skill 名称重复:{skill.name}")
self._skills[skill.name] = skill
@property
def skills(self) -> tuple[PythonSkillDefinition, ...]:
return tuple(self._skills.values())
def match(
self,
context_text: str,
*,
skill_refs: tuple[str, ...] = (),
metadata: dict[str, Any] | None = None,
) -> tuple[PythonSkillDefinition, ...]:
"""返回显式引用、关键词或自定义匹配器命中的 enabled Skills。"""
refs = set(skill_refs)
text = context_text.casefold()
match_metadata = metadata or {}
return tuple(
skill
for skill in self._skills.values()
if skill.enabled
and (
skill.name in refs
or any(trigger.casefold() in text for trigger in skill.triggers if trigger)
or (skill.match is not None and skill.match(context_text, match_metadata))
)
)
async def activate_python_skills(
registry: ExtensionRegistry,
skills: Iterable[PythonSkillDefinition],
) -> PythonSkillActivation:
"""把命中的 Skills 注册到当前 Run,并在部分失败时清理已激活项。"""
activation = PythonSkillActivation(tuple(skills), [])
try:
for skill in activation.skills:
cleanup = skill.setup(registry)
if cleanup is not None:
activation.cleanups.append(cleanup)
except Exception:
await activation.close()
raise
return activation
def discover_python_skills(packages: Iterable[str] = ()) -> PythonSkillCatalog:
"""从受信任 packages 发现定义,但不执行 Skill setup。"""
modules: list[ModuleType] = []
for package_name in dict.fromkeys(packages):
package = importlib.import_module(package_name)
package_path = getattr(package, "__path__", None)
if package_path is None:
raise TypeError(f"Python Skill 包必须是 package:{package_name}")
modules.extend(
importlib.import_module(module.name)
for module in pkgutil.iter_modules(package_path, f"{package.__name__}.")
if not module.name.rsplit(".", 1)[-1].startswith("_")
)
definitions: list[PythonSkillDefinition] = []
for module in sorted(modules, key=lambda item: item.__name__):
skill = getattr(module, "SKILL", None)
if not isinstance(skill, PythonSkillDefinition):
raise TypeError(f"Python Skill 缺少 SKILL 定义:{module.__name__}")
definitions.append(skill)
return PythonSkillCatalog(definitions)
__all__ = [
"PythonSkillActivation",
"PythonSkillCatalog",
"PythonSkillDefinition",
"activate_python_skills",
"discover_python_skills",
]
@@ -1,49 +0,0 @@
"""活动 Agent Run 注册表:为 steer、follow-up、abort 提供进程内控制面。"""
from __future__ import annotations
import asyncio
from app.application.agent.agent import Agent
class AgentRunRegistry:
"""跟踪当前进程中的活动 Agent;持久状态仍以 Run/EventStore 为准。"""
def __init__(self) -> None:
self._agents: dict[str, Agent] = {}
self._lock = asyncio.Lock()
async def register(self, run_id: str, agent: Agent) -> None:
async with self._lock:
self._agents[run_id] = agent
async def unregister(self, run_id: str, agent: Agent) -> None:
async with self._lock:
if self._agents.get(run_id) is agent:
self._agents.pop(run_id, None)
def get(self, run_id: str) -> Agent | None:
return self._agents.get(run_id)
def steer(self, run_id: str, content: str) -> bool:
agent = self.get(run_id)
if agent is None:
return False
return agent.steer(content)
def follow_up(self, run_id: str, content: str) -> bool:
agent = self.get(run_id)
if agent is None:
return False
return agent.follow_up(content)
async def abort(self, run_id: str) -> bool:
agent = self.get(run_id)
if agent is None or not await agent.abort():
return False
await agent.wait_for_idle()
return True
__all__ = ["AgentRunRegistry"]
-840
View File
@@ -1,840 +0,0 @@
"""流式 Agent 循环:stream → 累积 tool_calls → execute → 回填 ToolMessage → 再 stream。
阶段2b 步骤A:把 Agent 工具调用闭环从非流式 ``router.chat`` 改成流式 ``router.stream``,
并在主循环中暴露两个可选 async 回调(``on_chunk`` / ``on_tool_event``)供上层(步骤B
send_message)接入 SSE 增量冲刷与工具事件。本模块不感知 SSE / 前端,回调签名仅约定:
- ``on_chunk(chunk: ChatChunk)``:每个流式 chunk 透传(含 delta_content / delta_reasoning
/ tool_call_chunks / usage 等),上层可据此写 ``message.delta``。
- ``on_tool_event(event_type: str, payload: dict)``:工具生命周期事件:
- ``"tool.started"``:执行前
- ``"tool.completed"``:执行后,含 status(success / error / denied)
- ``"tool.requires_confirmation"``(阶段3 新增):高风险工具抛 RequiresConfirmation 时
发出,带 confirmation_id / risk_level / args,供前端展示确认卡片、approve/deny 端点
据此恢复循环。
阶段3 步骤1:工具确认挂起/恢复机制。
- 主循环遇到 ``RequiresConfirmation`` 时,通过 ``confirmation_registry.register(cid)`` 取
一个 ``Future[bool]`` 并 ``await``;外部 approve/deny 端点(步骤2)调
``registry.resolve(cid, approved)`` 触发恢复。
- 批准:以 ``confirmed=True`` 重试 execute;拒绝:回填"用户拒绝"ToolMessage,循环继续。
- 超时:``confirmation_timeout``(默认 300s)未裁决按拒绝处理;该超时受外层
``run_timeout``(send_message 的 ``asyncio.timeout``)约束——确认必须在 run 超时内完成。
- ``confirmation_registry`` 为 None 时降级:回填"需确认但未启用"消息,循环不卡(向下兼容)。
设计要点:
- 流式 ``router.stream(...)`` 返回 ``AsyncIterator[ChatChunk]``;流结束(async for 正常
退出)即本轮模型输出完毕,``ToolCallAccumulator.drain()`` 给出完整 tool_calls——不依赖
``finish_reason`` 判断。
- 工具执行经 ``ToolRuntime.execute``——司命评估与事件落库由其内部完成,本层只负责调用与
错误兜底。
- 工具失败**不中断循环**:异常被捕获、错误信息以 ToolMessage content 回填,让模型自行决策。
RequiresConfirmation 单独处理(不走错误兜底分支)。
- usage 累计每轮 prompt/completion/total_tokens。
- ``on_chunk`` / ``on_tool_event`` 默认 None;不传时本循环与纯后端逻辑等价。
"""
from __future__ import annotations
import json
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from typing import Any
from app.ai_provider import ProviderManagerFactory, TokenUsage
from app.ai_provider.llm import ToolCallAccumulator
from app.ai_provider.types.chat import ChatChunk
from app.ai_provider.types.messages import AIMessage, BaseMessage, ToolMessage
from app.application.agent.confirmation import ConfirmationRegistry
from app.application.agent.extensions import (
AgentContext,
AgentEvent,
AgentRunContext,
CancellationToken,
ExtensionRegistry,
ToolCallContext,
TurnSnapshot,
)
from app.core.logging import logger
from app.models.safety import PendingConfirmationRecord
from app.modules.safety.confirmation import new_confirmation_id
from app.modules.safety.exceptions import RequiresConfirmation
from app.modules.tool_runtime.definition import ToolResult, ToolRuntimeLike
# 默认最大循环次数:覆盖常见多步工具调用,超限则截断返回,避免无限循环。
DEFAULT_MAX_ITERATIONS = 8
# 默认确认超时:高风险工具挂起等待用户裁决的最长时间(秒);超时按拒绝处理。
DEFAULT_CONFIRMATION_TIMEOUT = 300.0
# 回调类型别名:给上层(步骤B send_message)类型检查用
OnChunkCallback = Callable[[ChatChunk], Awaitable[None]]
OnToolEventCallback = Callable[[str, dict[str, Any]], Awaitable[None]]
@dataclass
class AgentTurnResult:
"""单轮 Agent 循环的产出。"""
content: str # 最终回复正文(最后一次模型输出)
reasoning: str # 最终回复的思考内容(如有)
tool_history: list[dict[str, Any]] = field(
default_factory=list
) # [{name, args, result, status, tool_call_id?}]
iterations: int = 0 # 实际循环次数
truncated: bool = False # 是否因达上限截断
usage: TokenUsage = field(default_factory=TokenUsage) # 累计 token 用量
def _sum_usage(acc: TokenUsage, delta: TokenUsage) -> TokenUsage:
return acc + delta
def _parse_tool_args(args_str: str) -> dict[str, Any]:
"""把累积到的 args JSON 字符串解析成 dict;解析失败时回退 ``{"_raw": ...}``。"""
if not args_str:
return {}
try:
parsed = json.loads(args_str)
except json.JSONDecodeError:
return {"_raw": args_str}
if isinstance(parsed, dict):
return parsed
# 非对象 JSON(数组/字符串/数字)——包成 _raw 让模型自己理解
return {"_raw": args_str}
def _ai_message_from_stream(
content: str,
reasoning: str,
tool_calls: list[dict[str, Any]],
raw_content_blocks: list[dict[str, Any]] | None = None,
) -> AIMessage:
"""把流式累积的 content/reasoning/tool_calls 还原成 AIMessage 以便追加到 messages 历史。
``tool_calls`` 元素结构为 ``{"index", "id"?, "name"?, "args": str}``(由
``ToolCallAccumulator.drain`` 产出,``args`` 为完整 JSON 字符串)。AIMessage 期望
``{"name": ..., "args": dict, "id": ...}``,故需反序列化 args 并补全缺失 id。
"""
lc_tool_calls: list[dict[str, Any]] = []
for entry in tool_calls:
name = entry.get("name")
if not name:
# 没有 name 的 tool_call 无法执行,跳过(保留在累积器视图但不进历史)
continue
tc_id = entry.get("id") or f"call_{name}"
lc_tool_calls.append(
{
"name": name,
"args": _parse_tool_args(entry.get("args", "")),
"id": tc_id,
}
)
return AIMessage(
content=content,
tool_calls=lc_tool_calls,
reasoning=reasoning,
raw_content_blocks=raw_content_blocks,
)
def _truncate(text: str, limit: int = 4000) -> str:
"""截断超长文本(沙箱 stdout/stderr 可能很大),避免喂回模型时爆 token。
措辞明确这是「回显截断」而非数据损坏,并告知取后续内容的方法:否则模型读到大文件
被 read_file 截断的结果,会反复换方法重读,陷入循环直到 ``max_iterations`` 截断。
阈值 4000 兼顾 token 成本与单次可读量;大文件配合 read_file 的 offset/limit 分段读取。
"""
if len(text) <= limit:
return text
return (
f"{text[:limit]}\n…[输出过长,仅回显前 {limit} 字符;原始内容完整。"
f"如需后续内容:read_file 用 offset/limit 分段,或 shell 用 sed -n / tail / head 限定输出]"
)
async def _publish_message_lifecycle(
registry: ExtensionRegistry,
run: AgentRunContext,
message: BaseMessage,
) -> None:
"""发布一条完整消息的 start/end 生命周期。"""
if not registry.enabled:
return
payload = {"role": message.role, "message": message.to_dict()}
await registry.publish(AgentEvent("message.started", payload), run)
await registry.publish(AgentEvent("message.ended", payload), run)
def _tool_result_text(result: ToolResult) -> str:
"""把 ToolResult 转成回填给模型的紧凑文本。
显式标注 ``status=成功/失败``:否则成功路径(如 write_file 落盘后 stdout/stderr 全空)
只剩孤零零的 ``exit_code=0``,模型缺乏「已成功」的明确语义,倾向再调 read_file 验证,
叠加回显截断极易陷入反复重试。
"""
status = "成功" if result.ok else "失败"
parts = [f"status={status} exit_code={result.exit_code}"]
if result.stdout:
parts.append(f"stdout:\n{_truncate(result.stdout)}")
if result.stderr:
parts.append(f"stderr:\n{_truncate(result.stderr)}")
if result.timeout:
parts.append("timeout=true")
if result.truncated:
parts.append("truncated=true(仅截断回显给模型的输出,不影响已落盘等实际副作用)")
return "\n".join(parts)
# 控制流哨兵异常:只在本模块内传递,不泄漏到 run_agent_turn 调用方。
class _ToolDenied(Exception):
"""用户拒绝 / 确认超时。``message`` 为回填文本,``reason`` 为内部归因(denied/timeout)。"""
def __init__(self, message: str, *, reason: str = "denied") -> None:
super().__init__(message)
self.message = message
self.reason = reason
class _ToolSkipped(Exception):
"""confirmation_registry 未启用时的降级:回填"需确认但未启用"消息。"""
def __init__(self, message: str) -> None:
super().__init__(message)
self.message = message
async def _execute_tool_with_confirmation(
*,
tool_runtime: ToolRuntimeLike | None,
name: str,
tool_input: dict[str, Any],
args_str: str,
tool_call_id: str,
run_id: str | None,
character_id: str | None,
trace_id: str | None,
base_confirmed: bool,
confirmation_registry: ConfirmationRegistry | None,
confirmation_timeout: float,
on_tool_event: OnToolEventCallback | None,
cancellation: CancellationToken | None = None,
on_timeout: Callable[[str], Awaitable[None]] | None = None,
) -> ToolResult:
"""执行单次工具调用,处理 RequiresConfirmation 挂起/恢复。
- 正常返回 ToolResult;任意终态(拒绝/超时/降级)以本模块哨兵异常向调用方信号。
- 批准路径以 ``confirmed=True`` 重试 execute(绕过司命 ask),失败仍正常返回 ToolResult
(由调用方按 ok=False 回填 error 文本)。
- ``on_timeout(confirmation_id)``:确认超时时回调,供调用方标记超时。
"""
# base_confirmed=True 表示调用方已保证确认(如 API 已先确认),直接执行
use_confirmed = base_confirmed
if tool_runtime is None:
raise _ToolSkipped("工具运行时未启用,已跳过工具调用。")
while True:
try:
return await tool_runtime.execute(
name,
tool_input,
run_id=run_id,
character_id=character_id,
trace_id=trace_id,
confirmed=use_confirmed,
context={"cancellation": cancellation} if cancellation is not None else None,
)
except RequiresConfirmation as exc:
if use_confirmed:
raise _ToolSkipped(f"工具 {name} 在已确认状态下仍要求确认,已跳过。") from exc
# ---- registry 未启用:降级 ----
if confirmation_registry is None:
raise _ToolSkipped(
f"工具 {name} 需要用户确认(风险等级 {exc.risk_level or 'L2'}),"
"但当前未启用确认流程,已跳过。"
) from exc
# ---- registry 已启用:登记 + 持久化待确认 + 发事件 + await ----
confirmation_id = new_confirmation_id()
# 先持久化待确认记录(多 worker 共享确认状态),再发事件。
# 顺序保证:resolve/deny 回调(on_tool_event 触发的 create_task)看到记录时,
# DB 里已有该行,不会空转超时。
if run_id:
from app.db.session import AsyncSessionLocal
async with AsyncSessionLocal() as session:
record = PendingConfirmationRecord(
id=confirmation_id,
tool=name,
input_json=json.dumps(
{
"name": name,
"args": tool_input,
"tool_call_id": tool_call_id,
"args_str": args_str,
}
),
risk_level=exc.risk_level or "L2",
reason=exc.message,
run_id=run_id,
character_id=character_id,
status="pending",
)
session.add(record)
await session.commit()
if on_tool_event is not None:
await on_tool_event(
"tool.requires_confirmation",
{
"confirmation_id": confirmation_id,
"run_id": run_id,
"tool_name": name,
"tool_call_id": tool_call_id,
"risk_level": exc.risk_level,
"reason": exc.message,
"args": args_str,
},
)
# 使用混合确认注册表(同进程 Event + 重启后 polling)。无 run_id 的
# 独立调用仍可等待到超时,避免把“未裁决”误报成“确认机制未启用”。
confirmation_run_id = run_id or confirmation_id
try:
approved = await confirmation_registry.wait(
confirmation_run_id,
confirmation_id=confirmation_id,
timeout=confirmation_timeout,
)
except TimeoutError:
if on_timeout is not None:
await on_timeout(confirmation_id)
raise _ToolDenied(
f"工具 {name} 的确认超时(等待 {confirmation_timeout:.0f}s 未裁决),已按拒绝处理。",
reason="timeout",
) from None
if not approved:
raise _ToolDenied(f"用户拒绝了工具 {name} 的执行。") from exc
# 批准:以 confirmed=True 重试 execute(循环回到顶部 try)
logger.info(
"agent_tool_confirmed",
tool=name,
confirmation_id=confirmation_id,
run_id=run_id,
)
use_confirmed = True
if on_tool_event is not None:
await on_tool_event(
"tool.started",
{
"tool_name": name,
"tool_call_id": tool_call_id,
"args": args_str,
"confirmed": True,
},
)
async def run_agent_turn(
*,
messages: list[BaseMessage],
tools: list[dict[str, Any]],
tool_runtime: ToolRuntimeLike | None,
model_hint: str | None = None,
thinking_enabled: bool | None = None,
reasoning_effort: str | None = None,
max_iterations: int = DEFAULT_MAX_ITERATIONS,
# 可选上下文:透传给 ToolRuntime.execute 做事件归属与工作目录隔离
run_id: str | None = None,
character_id: str | None = None,
trace_id: str | None = None,
# 可选:高风险工具是否已由上层确认(透传给 ToolRuntime.execute)
confirmed: bool = False,
# 可选:每轮开始时回调(供外层重置 per-turn 状态如 reasoning_finalized)
on_turn: Callable[[], Awaitable[None]] | None = None,
# 可选回调:把流式 chunk 透传给上层做 SSE 增量冲刷(默认 None 表示不回调)
on_chunk: OnChunkCallback | None = None,
# 可选回调:工具生命周期事件("tool.started" / "tool.completed" /
# "tool.requires_confirmation")
on_tool_event: OnToolEventCallback | None = None,
# 可选:确认 Future 注册表(阶段3 步骤1)。缺省 None 时遇 RequiresConfirmation 降级。
confirmation_registry: ConfirmationRegistry | None = None,
# 可选:单次确认挂起超时(秒),默认 300s;超时按拒绝处理。
confirmation_timeout: float = DEFAULT_CONFIRMATION_TIMEOUT,
# 可选:确认超时回调(confirmation_id → None),供调用方标记 DB 超时状态。
on_confirm_timeout: Callable[[str], Awaitable[None]] | None = None,
# 受约束扩展 Hook 与本次运行上下文。
extensions: ExtensionRegistry | None = None,
run_context: AgentRunContext | None = None,
# Stateful Agent 的 steering 队列;每轮工具批次完成后消费。
get_steering_messages: Callable[[], list[BaseMessage]] | None = None,
) -> AgentTurnResult:
"""执行一轮带工具调用的 Agent 循环(流式版)。
循环逻辑见模块 docstring;返回值始终是 ``AgentTurnResult``,工具失败不抛异常。
``max_iterations`` 为硬上限:达到上限且模型仍返回 tool_calls 时返回 truncated=True。
``on_chunk`` / ``on_tool_event`` 缺省 None 时与纯后端逻辑等价。
``on_confirm_timeout(confirmation_id)``:确认超时时回调,用于标记 DB 记录。
确认挂起/恢复(阶段3 步骤1):``confirmation_registry`` 缺省 None 时,遇
``RequiresConfirmation`` 走降级路径(回填"需确认但未启用"消息,循环继续);否则按
register → 发 ``tool.requires_confirmation`` 事件 → ``await Future``(带超时)→
批准重试 / 拒绝回填 的流程处理。``confirmation_timeout`` 须小于上层 run_timeout,否则
run 先超时。
"""
if max_iterations < 1:
raise ValueError("max_iterations 必须 >= 1")
manager = ProviderManagerFactory.create()
extension_registry = extensions or ExtensionRegistry()
extension_context = run_context or AgentRunContext(
run_id=run_id,
character_id=character_id,
trace_id=trace_id,
)
usage = TokenUsage()
tool_history: list[dict[str, Any]] = []
iterations = 0
# 最后一轮累积值(用于 truncated 分支返回 content/reasoning)
last_content = ""
last_reasoning = ""
while iterations < max_iterations:
iterations += 1
if on_turn is not None:
await on_turn()
if extension_registry.enabled:
await extension_registry.publish(
AgentEvent("agent.turn.started", {"iteration": iterations}),
extension_context,
)
# ---- 本轮流式消费 ----
acc = ToolCallAccumulator()
content_buf = ""
reasoning_buf = ""
finish_reason: str | None = None
raw_blocks: dict[int, dict[str, Any]] = {}
params: dict[str, object] = {}
if thinking_enabled is not None:
params["thinking_enabled"] = thinking_enabled
if reasoning_effort is not None:
params["reasoning_effort"] = reasoning_effort
extension_context.cancellation.raise_if_cancelled()
request_context = AgentContext(tuple(messages), tuple(tools), model_hint)
if extension_registry.enabled:
request_context = await extension_registry.transform_context(
request_context,
extension_context,
)
if extension_registry.enabled:
await extension_registry.publish(
AgentEvent("message.started", {"role": "assistant", "iteration": iterations}),
extension_context,
)
stream = manager.chat_stream(
list(request_context.messages),
model_hint=request_context.model_hint,
params=params,
tools=list(request_context.tools) or None,
)
async for chunk in stream:
if on_chunk is not None:
await on_chunk(chunk)
if extension_registry.enabled and (chunk.delta_content or chunk.delta_reasoning):
await extension_registry.publish(
AgentEvent(
"message.updated",
{
"role": "assistant",
"iteration": iterations,
"delta": chunk.delta_content,
"reasoning_delta": chunk.delta_reasoning,
},
),
extension_context,
)
if chunk.delta_content:
content_buf += chunk.delta_content
if chunk.delta_reasoning:
reasoning_buf += chunk.delta_reasoning
if chunk.finish_reason is not None:
finish_reason = chunk.finish_reason
if chunk.content_block_start is not None and chunk.content_block_index is not None:
raw_blocks[chunk.content_block_index] = dict(chunk.content_block_start)
if chunk.content_block_delta is not None and chunk.content_block_index is not None:
block = raw_blocks.get(chunk.content_block_index)
if block is not None:
delta = chunk.content_block_delta
delta_type = delta.get("type")
if delta_type == "text_delta":
block["text"] = f"{block.get('text', '')}{delta.get('text', '')}"
elif delta_type == "thinking_delta":
block["thinking"] = (
f"{block.get('thinking', '')}{delta.get('thinking', '')}"
)
elif delta_type == "signature_delta":
block["signature"] = (
f"{block.get('signature', '')}{delta.get('signature', '')}"
)
elif delta_type == "input_json_delta":
block["_partial_json"] = (
f"{block.get('_partial_json', '')}{delta.get('partial_json', '')}"
)
if chunk.usage is not None:
usage = _sum_usage(usage, chunk.usage)
acc.add_chunk(chunk)
# 流结束即本轮模型输出完毕
tool_calls = acc.drain()
last_content = content_buf
last_reasoning = reasoning_buf
raw_content_blocks: list[dict[str, Any]] = []
for index in sorted(raw_blocks):
block = raw_blocks[index]
partial_json = block.pop("_partial_json", None)
if partial_json is not None:
block["input"] = _parse_tool_args(partial_json)
raw_content_blocks.append(block)
# 把模型输出(含 thinking signature / tool_calls)原样追加到历史。
ai_message = _ai_message_from_stream(
content_buf,
reasoning_buf,
tool_calls,
raw_content_blocks or None,
)
messages.append(ai_message)
if extension_registry.enabled:
await extension_registry.publish(
AgentEvent(
"message.ended",
{
"role": "assistant",
"iteration": iterations,
"message": ai_message.to_dict(),
"finish_reason": finish_reason,
},
),
extension_context,
)
# 无工具调用:先消费 steering,再决定是否结束。
if not any(tc.get("name") for tc in tool_calls):
steering = get_steering_messages() if get_steering_messages is not None else []
turn = TurnSnapshot(
iteration=iterations,
content=content_buf,
reasoning=reasoning_buf,
tool_calls=(),
tool_results=tuple(tool_history),
usage=usage,
)
if extension_registry.enabled:
await extension_registry.publish(
AgentEvent("agent.turn.ended", {"iteration": iterations}),
extension_context,
)
# TurnPolicy 在此分支不参与终止决策:模型无工具调用时已由 steering /
# finish_reason / 默认 return 收尾,强行接入会覆盖 pause_turn / refusal
# 既有处理;其真实终止作用在下方工具分支(should_stop or terminate_after_tools)。
await extension_registry.should_stop(turn, extension_context)
if steering:
messages.extend(steering)
continue
if finish_reason == "pause_turn":
continue
if finish_reason == "refusal":
raise RuntimeError("模型拒绝了当前请求。")
return AgentTurnResult(
content=content_buf,
reasoning=reasoning_buf,
tool_history=tool_history,
iterations=iterations,
truncated=finish_reason == "length",
usage=usage,
)
# 逐个执行 tool_call 并把结果以 ToolMessage 回填。工具可能有副作用或需要确认,
# 在 ToolSpec 未声明并行安全前保持确定性串行执行。
turn_history_start = len(tool_history)
terminate_after_tools = False
for call_index, entry in enumerate(tool_calls):
name = entry.get("name")
if not name:
continue
args_str = entry.get("args", "")
tool_input = _parse_tool_args(args_str)
tool_call_id = entry.get("id") or f"call_{iterations}_{call_index}_{name}"
hook_call = ToolCallContext(
tool_call_id=tool_call_id,
name=name,
input=tool_input,
messages=tuple(messages),
)
if extension_registry.enabled:
decision = await extension_registry.before_tool_call(
hook_call,
extension_context,
)
if not decision.allowed:
denied_text = decision.reason or f"扩展阻止了工具 {name} 的执行。"
tool_history.append(
{
"name": name,
"args": tool_input,
"result": denied_text,
"status": "denied",
"tool_call_id": tool_call_id,
}
)
tool_message = ToolMessage(content=denied_text, tool_call_id=tool_call_id)
messages.append(tool_message)
await _publish_message_lifecycle(
extension_registry, extension_context, tool_message
)
if on_tool_event is not None:
await on_tool_event(
"tool.completed",
{
"tool_name": name,
"tool_call_id": tool_call_id,
"status": "denied",
"result": denied_text[:500],
},
)
continue
if decision.input is not None:
tool_input = decision.input
args_str = json.dumps(tool_input, ensure_ascii=False)
hook_call = ToolCallContext(
tool_call_id=tool_call_id,
name=name,
input=tool_input,
messages=tuple(messages),
)
# 执行前事件
if on_tool_event is not None:
await on_tool_event(
"tool.started",
{
"tool_name": name,
"tool_call_id": tool_call_id,
"args": args_str,
},
)
try:
tool_result = await _execute_tool_with_confirmation(
tool_runtime=tool_runtime,
name=name,
tool_input=tool_input,
args_str=args_str,
tool_call_id=tool_call_id,
run_id=run_id,
character_id=character_id,
trace_id=trace_id,
base_confirmed=confirmed,
confirmation_registry=confirmation_registry,
confirmation_timeout=confirmation_timeout,
on_tool_event=on_tool_event,
cancellation=extension_context.cancellation,
on_timeout=on_confirm_timeout,
)
except _ToolDenied as denied:
# 用户拒绝或确认超时:回填拒绝消息,循环继续让模型自决策
logger.info(
"agent_tool_denied",
tool=name,
reason=denied.reason,
run_id=run_id,
)
denied_text = denied.message
tool_history.append(
{
"name": name,
"args": tool_input,
"result": denied_text,
"status": "denied",
"tool_call_id": tool_call_id,
}
)
tool_message = ToolMessage(content=denied_text, tool_call_id=tool_call_id)
messages.append(tool_message)
await _publish_message_lifecycle(
extension_registry, extension_context, tool_message
)
if on_tool_event is not None:
await on_tool_event(
"tool.completed",
{
"tool_name": name,
"tool_call_id": tool_call_id,
"status": "denied",
"result": denied_text[:500],
},
)
continue
except _ToolSkipped as skipped:
# registry 未启用:回填"需确认但未启用",循环继续
skipped_text = skipped.message
tool_history.append(
{
"name": name,
"args": tool_input,
"result": skipped_text,
"status": "skipped",
"tool_call_id": tool_call_id,
}
)
tool_message = ToolMessage(content=skipped_text, tool_call_id=tool_call_id)
messages.append(tool_message)
await _publish_message_lifecycle(
extension_registry, extension_context, tool_message
)
if on_tool_event is not None:
await on_tool_event(
"tool.completed",
{
"tool_name": name,
"tool_call_id": tool_call_id,
"status": "skipped",
"result": skipped_text[:500],
},
)
continue
except Exception as exc: # noqa: BLE001 - 任意工具错误都回填、不中断循环
# 司命 deny / 沙箱错误 / 未知工具等:错误信息回填,循环继续
logger.warning(
"agent_tool_failed",
tool=name,
error=str(exc),
run_id=run_id,
)
error_text = f"工具执行失败:{type(exc).__name__}: {_truncate(str(exc))}"
tool_history.append(
{
"name": name,
"args": tool_input,
"result": error_text,
"status": "error",
"tool_call_id": tool_call_id,
}
)
tool_message = ToolMessage(content=error_text, tool_call_id=tool_call_id)
messages.append(tool_message)
await _publish_message_lifecycle(
extension_registry, extension_context, tool_message
)
if on_tool_event is not None:
await on_tool_event(
"tool.completed",
{
"tool_name": name,
"tool_call_id": tool_call_id,
"status": "error",
"result": error_text[:500],
},
)
continue
# 成功路径:扩展可变换结果或提示终止;安全裁定已在 ToolRuntime 内完成。
if extension_registry.enabled:
after = await extension_registry.after_tool_call(
hook_call,
tool_result,
extension_context,
)
tool_result = after.result or tool_result
terminate_after_tools = terminate_after_tools or after.terminate
outcome = _tool_result_text(tool_result)
status = "success" if tool_result.ok else "error"
if not tool_result.ok:
# 沙箱执行返回非 ok(exit_code != 0 等)——不抛异常,回填让模型自决策
logger.info(
"agent_tool_returned_error",
tool=name,
exit_code=tool_result.exit_code,
stderr=tool_result.stderr[:500] if tool_result.stderr else "",
run_id=run_id,
)
tool_history.append(
{
"name": name,
"args": tool_input,
"result": outcome,
"status": status,
"tool_call_id": tool_call_id,
}
)
tool_message = ToolMessage(content=outcome, tool_call_id=tool_call_id)
messages.append(tool_message)
await _publish_message_lifecycle(extension_registry, extension_context, tool_message)
if on_tool_event is not None:
await on_tool_event(
"tool.completed",
{
"tool_name": name,
"tool_call_id": tool_call_id,
"status": status,
"result": outcome[:500],
},
)
steering = get_steering_messages() if get_steering_messages is not None else []
if steering:
messages.extend(steering)
turn = TurnSnapshot(
iteration=iterations,
content=content_buf,
reasoning=reasoning_buf,
tool_calls=tuple(tool_calls),
tool_results=tuple(tool_history[turn_history_start:]),
usage=usage,
)
should_stop = terminate_after_tools
if extension_registry.enabled:
await extension_registry.publish(
AgentEvent("agent.turn.ended", {"iteration": iterations}),
extension_context,
)
should_stop = should_stop or await extension_registry.should_stop(
turn,
extension_context,
)
if should_stop and not steering:
return AgentTurnResult(
content=content_buf,
reasoning=reasoning_buf,
tool_history=tool_history,
iterations=iterations,
truncated=False,
usage=usage,
)
# 达上限仍带 tool_calls:截断返回,content 用最后一次模型输出或说明已截断
content = last_content or f"Agent 循环达到最大次数 {max_iterations},已截断。"
return AgentTurnResult(
content=content,
reasoning=last_reasoning,
tool_history=tool_history,
iterations=iterations,
truncated=True,
usage=usage,
)
__all__ = [
"DEFAULT_CONFIRMATION_TIMEOUT",
"DEFAULT_MAX_ITERATIONS",
"AgentTurnResult",
"OnChunkCallback",
"OnToolEventCallback",
"run_agent_turn",
]
@@ -1,22 +0,0 @@
"""从既有 Skill 存储加载运行时 manifest。"""
from __future__ import annotations
from app.modules.skills import SkillManifest, parse_skill_manifest
from app.repositories.json_utils import loads_list
from app.services.skills import get_enabled_skills_by_names, match_active_skills
async def load_skill_manifests(
context_text: str,
*,
skill_refs: tuple[str, ...] = (),
) -> list[SkillManifest]:
"""加载关键词命中或显式引用的 enabled Skill。"""
skills = await match_active_skills(context_text)
if skill_refs:
by_name = {skill.name: skill for skill in skills}
for skill in await get_enabled_skills_by_names(skill_refs):
by_name.setdefault(skill.name, skill)
skills = list(by_name.values())
return [parse_skill_manifest(skill, loads_list(skill.tool_bindings_json)) for skill in skills]
-95
View File
@@ -1,95 +0,0 @@
"""将洛书规划结果映射为通用 Agent 任务,并写入 Run 事件。"""
from __future__ import annotations
from app.modules.agent import AgentTask, Plan, PlanStep
from app.modules.planner import DeliberativePlan
def task_from_deliberative_plan(
plan: DeliberativePlan,
*,
run_id: str,
agent_name: str,
objective: str,
) -> AgentTask:
"""把一次规划结果转换为仅在当前 Run 存活的 AgentTask。"""
return AgentTask(
id=f"task_{run_id}",
objective=objective,
agent_name=agent_name,
run_id=run_id,
status="running",
plan=Plan(
steps=tuple(
PlanStep(
id=str(step.id),
title=step.title,
description=step.description,
tool=step.tool,
risk_level=step.risk_level,
)
for step in plan.steps
),
assumptions=tuple(plan.assumptions),
risks=tuple(plan.risks),
status="running",
),
)
def next_step_for_tool(task: AgentTask, tool_name: str, completed_step_ids: set[str]) -> PlanStep | None:
"""返回尚未处理且明确绑定到工具的首个计划步骤。"""
if task.plan is None:
return None
return next(
(
step
for step in task.plan.steps
if step.id not in completed_step_ids and step.tool == tool_name
),
None,
)
def step_event_payload(
task: AgentTask,
step: PlanStep,
*,
status: str,
result_summary: str | None = None,
) -> dict[str, object]:
"""生成计划步骤生命周期事件 payload。"""
return {
"task_id": task.id,
"step_id": step.id,
"title": step.title,
"tool": step.tool,
"risk_level": step.risk_level,
"status": status,
**({"result_summary": result_summary} if result_summary else {}),
}
def task_plan_payload(task: AgentTask) -> dict[str, object]:
"""生成安全且可直接展示的计划事件 payload。"""
assert task.plan is not None
return {
"task_id": task.id,
"agent_name": task.agent_name,
"objective": task.objective,
"status": task.plan.status,
"assumptions": list(task.plan.assumptions),
"risks": list(task.plan.risks),
"steps": [
{
"id": step.id,
"title": step.title,
"description": step.description,
"tool": step.tool,
"risk_level": step.risk_level,
"status": step.status,
}
for step in task.plan.steps
],
}
-137
View File
@@ -1,137 +0,0 @@
"""ToolSpec → OpenAI 函数 schema 的转换与默认工具集选取。
直接产出 OpenAI function schema dict(``{"type": "function", "function": {...}}``),
与 ChatResult.tool_calls 里的 name/args 回填闭环天然对齐。
ToolSpec 的 name/description 直接映射;参数 schema 因 ToolSpec 未内嵌 JSON Schema,
这里按工具 name 查表给出最小参数 schema(builtins 内工具参数固定,可选参数也显式列出)。
未知工具默认空参数 schema(``{"type": "object", "properties": {}}``),模型按需传参。
"""
from __future__ import annotations
from typing import Any
from app.modules.tool_runtime.definition import ToolSpec
# 内置工具的参数 schema(name → JSON Schema)。新增工具时扩展此表即可。
# 保持最小集合:只声明 known 工具的参数;调用方传入未登记工具时回退空对象 schema。
_PARAM_SCHEMAS: dict[str, dict[str, Any]] = {
"datetime": {"type": "object", "properties": {}, "required": []},
"read_file": {
"type": "object",
"properties": {
"path": {"type": "string", "description": "workspace 内的文件相对路径。"},
"offset": {
"type": "integer",
"description": "起始行号(1-based),默认 1。大文件分段读取后续内容时使用。",
},
"limit": {
"type": "integer",
"description": "读取的行数上限;不传则读到末尾。大文件分段读取时使用。",
},
},
"required": ["path"],
},
"list_dir": {
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "workspace 内的目录相对路径,缺省为 '.'。",
},
},
"required": [],
},
"write_file": {
"type": "object",
"properties": {
"path": {"type": "string", "description": "workspace 内文件相对路径。"},
"content": {"type": "string", "description": "要写入的文本内容。"},
},
"required": ["path"],
},
"shell": {
"type": "object",
"properties": {
"command": {
"type": "string",
"description": "要在沙箱中执行的 shell 命令。工作目录为沙箱隔离的 workspace 目录(并非 /home/user 或 /tmp 等 Linux 路径,这些路径不存在,不要 cd 到它们)。read_file/write_file 操作的也是这个 workspace 目录,shell 里 cd 到别的目录将看不到这些文件。",
},
"timeout_ms": {"type": "integer", "description": "可选超时(毫秒),默认 30000。"},
"stdin": {"type": "string", "description": "可选标准输入。"},
},
"required": ["command"],
},
"fetch_url": {
"type": "object",
"properties": {
"url": {"type": "string", "description": "要抓取的网页 URL。"},
"max_chars": {
"type": "integer",
"description": "正文最大字符数,默认 8000。",
},
},
"required": ["url"],
},
}
_EMPTY_SCHEMA: dict[str, Any] = {"type": "object", "properties": {}, "required": []}
def to_ai_provider_tools(tool_specs: list[ToolSpec]) -> list[dict[str, Any]]:
"""把 ToolSpec 列表转成 OpenAI function schema dict 列表。
先查 ``_PARAM_SCHEMAS``(内置工具),未命中则用 ``spec.input_schema``(MCP 工具),
仍为空则回退无参占位 ``_EMPTY_SCHEMA``。
"""
tools: list[dict[str, Any]] = []
for spec in tool_specs:
params = _PARAM_SCHEMAS.get(spec.name)
if not params and spec.name.startswith("skill:"):
params = _PARAM_SCHEMAS.get(spec.name.rsplit("/", 1)[-1])
if not params:
params = spec.input_schema or _EMPTY_SCHEMA
tools.append(
{
"type": "function",
"function": {
"name": spec.name,
"description": spec.description,
"parameters": params,
},
}
)
return tools
# 核心工具恒绑(沙箱类,始终可用)。
_CORE_TOOL_NAMES = ["datetime", "read_file", "list_dir", "write_file", "shell"]
def default_tools(
tool_specs_by_name: dict[str, ToolSpec] | None = None,
*,
allowed_tools: tuple[str, ...] = (),
) -> list[dict[str, Any]]:
"""返回工具对应的 OpenAI function schema 列表。
- 未传 ``tool_specs_by_name`` 时,从 ``builtin_specs()`` 加载仅返回核心工具
(datetime/read_file/list_dir/write_file/shell),共 5 个。
- 传入自定义工具池时(如 ``tool_runtime._resolve_specs()``),返回池中所有工具,
含 MCP 注册的工具,不限于核心工具。
"""
if tool_specs_by_name is None:
from app.application.tool_runtime.builtins import builtin_specs
tool_specs_by_name = builtin_specs()
chosen = [
tool_specs_by_name[name] for name in _CORE_TOOL_NAMES if name in tool_specs_by_name
]
else:
chosen = list(tool_specs_by_name.values())
if allowed_tools:
allowed = set(allowed_tools)
chosen = [spec for spec in chosen if spec.name in allowed]
return to_ai_provider_tools(chosen)
-198
View File
@@ -1,198 +0,0 @@
"""资料导入候选抽取用例(异步后台任务,跨 services + ai_provider)。
触发端点在校验并置 job 为 ``extracting_*`` 后即返回;实际模型调用在后台任务中
进行(独立 session),完成后落候选并置 job 为 ``reviewing`` / ``parsed``。
仅适用单进程;多 worker 部署需替换为跨进程任务队列。
"""
import asyncio
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.ai_provider import ProviderManager, ProviderManagerFactory
from app.core.logging import logger
from app.db.session import AsyncSessionLocal
from app.models.candidates import MemoryCandidate, ProfileCandidate
from app.models.imports import CharacterImportJob, ImportItem
from app.models.personality import PersonalityProposal
from app.modules.imports.extract import (
parse_memory_candidates,
parse_personality_proposals,
parse_profile_candidates,
)
from app.modules.imports.prompts import (
build_memory_extraction_messages,
build_personality_extraction_messages,
build_profile_extraction_messages,
)
from app.repositories import candidates as candidates_repo
from app.repositories import imports as imports_repo
from app.repositories.json_utils import dumps
# 后台抽取任务强引用:asyncio.create_task 仅保留弱引用,需强引用防 GC。
_running_extraction_tasks: set[asyncio.Task[None]] = set()
_provider_manager: ProviderManager | None = None
def _get_provider_manager() -> ProviderManager:
global _provider_manager
if _provider_manager is None:
_provider_manager = ProviderManagerFactory.create()
return _provider_manager
_PROFILE_MODEL_HINT = "personality-analysis"
_MEMORY_MODEL_HINT = "memory-extraction"
_PERSONALITY_MODEL_HINT = "personality-analysis"
async def drain_pending_extractions() -> None:
"""等待所有进行中的后台抽取任务完成(测试断言 / 优雅停机用)。"""
if _running_extraction_tasks:
await asyncio.gather(*_running_extraction_tasks, return_exceptions=True)
def start_profile_extraction(job_id: str) -> None:
task = asyncio.create_task(_run_profile_extraction(job_id))
_running_extraction_tasks.add(task)
task.add_done_callback(_running_extraction_tasks.discard)
def start_memory_extraction(job_id: str) -> None:
task = asyncio.create_task(_run_memory_extraction(job_id))
_running_extraction_tasks.add(task)
task.add_done_callback(_running_extraction_tasks.discard)
def start_personality_extraction(job_id: str) -> None:
task = asyncio.create_task(_run_personality_extraction(job_id))
_running_extraction_tasks.add(task)
task.add_done_callback(_running_extraction_tasks.discard)
async def _load_job_and_items(
session: AsyncSession, job_id: str
) -> tuple[CharacterImportJob | None, list[ImportItem]]:
job = await imports_repo.get_job(session, job_id)
if job is None:
return None, []
result = await session.execute(
select(ImportItem)
.where(ImportItem.import_job_id == job_id)
.order_by(ImportItem.sequence.asc())
)
return job, list(result.scalars().all())
async def _run_profile_extraction(job_id: str) -> None:
async with AsyncSessionLocal() as session:
try:
job, items = await _load_job_and_items(session, job_id)
if job is None:
return
system, user = build_profile_extraction_messages(_pairs(items))
raw = await _call_model(system, user, _PROFILE_MODEL_HINT, job.character_id)
candidates = [
ProfileCandidate(
character_id=job.character_id,
import_job_id=job.id,
field=draft.field_name,
value=draft.value,
confidence=draft.confidence,
reason=draft.reason,
status="pending",
evidence_ids_json=dumps(_evidence_ids(items, draft.evidence)),
)
for draft in parse_profile_candidates(raw)
]
await candidates_repo.add_profile_candidates(session, candidates)
job.status = "reviewing" if candidates else "parsed"
await session.commit()
except Exception as exc:
await _fail_job(session, job_id, exc)
async def _run_memory_extraction(job_id: str) -> None:
async with AsyncSessionLocal() as session:
try:
job, items = await _load_job_and_items(session, job_id)
if job is None:
return
system, user = build_memory_extraction_messages(_pairs(items))
raw = await _call_model(system, user, _MEMORY_MODEL_HINT, job.character_id)
candidates = [
MemoryCandidate(
character_id=job.character_id,
import_job_id=job.id,
type=draft.type,
title=draft.title,
content=draft.content,
confidence=0.6,
importance=draft.importance,
sensitivity=draft.sensitivity,
reason=draft.reason,
status="pending",
evidence_ids_json=dumps(_evidence_ids(items, draft.evidence)),
)
for draft in parse_memory_candidates(raw)
]
await candidates_repo.add_memory_candidates(session, candidates)
job.status = "reviewing" if candidates else "parsed"
await session.commit()
except Exception as exc:
await _fail_job(session, job_id, exc)
async def _run_personality_extraction(job_id: str) -> None:
async with AsyncSessionLocal() as session:
try:
job, items = await _load_job_and_items(session, job_id)
if job is None:
return
system, user = build_personality_extraction_messages(_pairs(items), "")
raw = await _call_model(system, user, _PERSONALITY_MODEL_HINT, job.character_id)
proposals = [
PersonalityProposal(
character_id=job.character_id,
fields_json=dumps({draft.field_name: draft.change}),
reason=draft.reason or "资料导入抽取的人格完善建议",
status="pending",
)
for draft in parse_personality_proposals(raw)
]
session.add_all(proposals)
await session.flush()
job.status = "reviewing" if proposals else "parsed"
await session.commit()
except Exception as exc:
await _fail_job(session, job_id, exc)
def _pairs(items: list[ImportItem]) -> list[tuple[str, str]]:
return [(item.role, item.content) for item in items]
def _evidence_ids(items: list[ImportItem], indices: list[int]) -> list[str]:
return [items[i].id for i in indices if 0 <= i < len(items)]
async def _call_model(system: str, user: str, model_hint: str, _character_id: str) -> str:
manager = _get_provider_manager()
messages = manager.to_message_list([system], [("user", user)])
response = await manager.chat(messages, model_hint=model_hint)
return response.content
async def _fail_job(session: AsyncSession, job_id: str, exc: Exception) -> None:
logger.error(f"导入候选抽取失败 job={job_id}: {exc}")
try:
job = await imports_repo.get_job(session, job_id)
if job is not None:
job.status = "failed"
job.error_code = "extraction_failed"
job.error_message = str(exc)[:500]
await session.commit()
except Exception: # noqa: BLE001 — 失败处理本身不能再抛
pass
@@ -1,23 +0,0 @@
"""洛书枢 deliberative 规划器(编排层)。
领域数据结构与纯函数归位到 ``app/modules/planner/``,
本模块为薄壳,从 modules 导入并重导出以保持向后兼容。
"""
from app.modules.planner import (
DeliberativePlan,
PlanStep,
build_planning_prompt,
format_plan_for_prompt,
parse_plan,
should_plan,
)
__all__ = [
"DeliberativePlan",
"PlanStep",
"build_planning_prompt",
"format_plan_for_prompt",
"parse_plan",
"should_plan",
]
-83
View File
@@ -1,83 +0,0 @@
"""声明式 Skill 激活:匹配、提示注入、Run 级工具别名和事件。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from app.application.agent.prompt_composer import compose_system_messages
from app.application.agent.skill_loader import load_skill_manifests
from app.modules.runtime.events import Event
if TYPE_CHECKING:
from app.application.tool_runtime.view import RunScopedToolView
from app.modules.runtime.events import EventStore
class SkillExecutor:
"""把数据库 Skill 激活到当前 Run 的工具视图。"""
def __init__(
self,
tool_view: RunScopedToolView | None = None,
event_store: EventStore | None = None,
) -> None:
self._tool_view = tool_view
self._event_store = event_store
async def match_and_activate(
self,
context_text: str,
system_messages: list[str],
*,
run_id: str | None = None,
character_id: str | None = None,
skill_refs: tuple[str, ...] = (),
) -> list[str]:
"""匹配技能并把提示与别名写入当前 Run。"""
if not context_text:
return system_messages
manifests = await load_skill_manifests(context_text, skill_refs=skill_refs)
system_messages.extend(compose_system_messages([], None, manifests))
activated_names: list[str] = []
for manifest in manifests:
if self._tool_view is not None:
for tool_name in manifest.allowed_tools:
self._tool_view.bind_alias(
f"skill:{manifest.name}/{tool_name}",
tool_name,
description_prefix=f"[{manifest.name}] ",
)
activated_names.append(manifest.name)
if activated_names and self._event_store:
await self._emit_activation_event(
activated_names,
run_id=run_id,
character_id=character_id,
)
return system_messages
async def _emit_activation_event(
self,
skill_names: list[str],
*,
run_id: str | None = None,
character_id: str | None = None,
) -> None:
event_store = self._event_store
if event_store is None:
return
await event_store.append(
Event(
type="skill.activated",
source="skill_executor",
visibility="public",
run_id=run_id,
character_id=character_id,
payload={"skills": skill_names},
)
)
__all__ = ["SkillExecutor"]
@@ -1,14 +0,0 @@
"""百工工具运行时编排层。
司命评估 → 直接执行/沙箱执行 → 一炁事件回写 的编排核心。从 ``modules/tool_runtime`` 迁入
``application``,因为编排涉及沙箱(integrations)、司命(modules)、事件存储(services)
等多层协调,属于跨模块用例编排(见后端分层架构设计 §5.3)。
工具规格数据结构(``ToolSpec`` / ``ToolResult``)仍在 ``modules/tool_runtime/definition``。
"""
from app.application.tool_runtime.builtins import builtin_specs
from app.application.tool_runtime.runtime import ToolRuntime
from app.application.tool_runtime.view import RunScopedToolView
__all__ = ["RunScopedToolView", "ToolRuntime", "builtin_specs"]
@@ -1,174 +0,0 @@
"""内置工具集合。覆盖 L0–L3 风险等级,供编排闭环、司命分级演示与前端调用。
文件类工具(read_file / list_dir / write_file)使用沙箱 base 下的持久子目录
``workspace``,并通过 ``isolated_workspace=True`` 由 runtime 改写为按角色隔离的
``workspace/<character_id>``,使同一角色跨会话/跨 run 文件互通、不同角色互不可见
(workdir 相对 base,受沙箱路径逃逸保护)。多用户上线后隔离主体将切换为真实用户标识。
执行路径:
- 沙箱路径(datetime/read_file/list_dir/write_file/shell):``build_request`` 产出 ExecRequest,
由 SandboxClient 执行。参数默认用环境变量传参(避免命令注入);write_file 的 content 因可能
很大(单个 env 受 MAX_ARG_STRLEN≈128KB 限制)改经 stdin 写入——既无大小上限,又不经 shell 解释。
shell 保留直接拼接——本就是任意命令,且经沙箱隔离。
- 直接执行路径(fetch_url):``executor`` 直接发起 HTTP 调用(见 ``web_tools.py``),
不走沙箱;http_client 与密钥由 runtime 注入。
"""
from __future__ import annotations
from app.application.tool_runtime.web_tools import _fetch_url_executor
from app.core.config import settings
from app.integrations.sandbox.models import ExecRequest
from app.modules.safety.governor import PermissionPolicy
from app.modules.tool_runtime.definition import ToolInput, ToolSpec
# 沙箱 base 下的持久工作子目录(文件类工具共享,由 runtime 按角色隔离为 workspace/<character_id>)
_WORKSPACE = "workspace"
def _need_str(input: ToolInput, key: str) -> str:
value = input.get(key)
if not isinstance(value, str) or not value.strip():
raise ValueError(f"工具需要非空字符串 '{key}' 输入")
return value
def _datetime_request(_input: ToolInput) -> ExecRequest:
return ExecRequest(argv=["sh", "-c", "date '+%Y-%m-%d %H:%M:%S %Z'"])
def _as_positive_int(value: object, *, default: int) -> int:
"""把 offset/limit 容错解析为 >=1 的整数;非数字或缺省返回 default。"""
try:
n = int(value) # str/int/float 均可;bool(True)=1 由 >=1 兜底
except (TypeError, ValueError):
return default
return n if n >= 1 else default
def _read_file_request(input: ToolInput) -> ExecRequest:
path = _need_str(input, "path")
# 行分页(offset 起始行 1-based / limit 行数):大文件可分段读取,避免整体 cat 后被回显
# 截断、模型拿不到完整内容而反复试错。offset/limit 经 int() 规整为纯数字后拼入 sed 地址
# (数字无注入风险),行范围用单引号包裹(末行 $ 字面、不经 shell 展开),PATH_ARG 仍走 env。
# 缺省 offset/limit 时读全部(行为不变)。
# 使用绝对路径(/bin/sed, /bin/cat):避免 env_clear 后 PATH 失效。
start = _as_positive_int(input.get("offset"), default=1)
row_limit = _as_positive_int(input.get("limit"), default=0)
if start > 1 or row_limit > 0:
range_expr = f"{start},{start + row_limit - 1}" if row_limit > 0 else f"{start},$"
return ExecRequest(
argv=["sh", "-c", f"sed -n '{range_expr}p' \"$PATH_ARG\""],
env={"PATH_ARG": path},
workdir=_WORKSPACE,
)
return ExecRequest(
argv=["sh", "-c", 'cat -- "$PATH_ARG"'],
env={"PATH_ARG": path},
workdir=_WORKSPACE,
)
def _list_dir_request(input: ToolInput) -> ExecRequest:
path = input.get("path", ".")
return ExecRequest(
argv=["sh", "-c", 'ls -la -- "$PATH_ARG"'],
env={"PATH_ARG": str(path)},
workdir=_WORKSPACE,
)
def _write_file_request(input: ToolInput) -> ExecRequest:
path = _need_str(input, "path")
content = input.get("content", "")
# content 经 stdin 写入(cat > file),不经环境变量:单个 env 受 MAX_ARG_STRLEN≈128KB
# 限制,大文档经 env 会让 execve 返回 E2BIG、沙箱 ok=False,模型反复重试直到截断;
# stdin 既无大小上限、又完全不经过 shell 解释(比 env/argv 更安全,内容里的 `$`、
# 反引号、`%` 等都原样落盘)。PATH_ARG 仍用 env(短、需拼进重定向目标)。
return ExecRequest(
argv=["sh", "-c", 'cat > "$PATH_ARG"'],
env={"PATH_ARG": path},
stdin=str(content),
workdir=_WORKSPACE,
)
def _shell_request(input: ToolInput) -> ExecRequest:
command = _need_str(input, "command")
raw_timeout = input.get("timeout_ms")
timeout_ms = (
int(raw_timeout)
if isinstance(raw_timeout, int | float)
else settings.sandbox_default_timeout_ms
)
stdin = input.get("stdin")
return ExecRequest(
argv=["sh", "-c", command],
stdin=stdin if isinstance(stdin, str) else None,
timeout_ms=timeout_ms,
)
DATETIME = ToolSpec(
name="datetime",
description="获取当前时间(只读,L0)。",
risk_level="L0",
permission_policy=PermissionPolicy.allow,
build_request=_datetime_request,
side_effect=False,
)
READ_FILE = ToolSpec(
name="read_file",
description="读取 workspace 内文件(只读,L1)。",
risk_level="L1",
permission_policy=PermissionPolicy.allow,
build_request=_read_file_request,
side_effect=False,
isolated_workspace=True,
)
LIST_DIR = ToolSpec(
name="list_dir",
description="列出 workspace 内目录(只读,L1)。",
risk_level="L1",
permission_policy=PermissionPolicy.allow,
build_request=_list_dir_request,
side_effect=False,
isolated_workspace=True,
)
WRITE_FILE = ToolSpec(
name="write_file",
description="写入 workspace 内文件(有副作用,L2 需确认)。",
risk_level="L2",
permission_policy=PermissionPolicy.ask,
build_request=_write_file_request,
side_effect=True,
isolated_workspace=True,
)
SHELL = ToolSpec(
name="shell",
description="在沙箱中执行任意 shell 命令(高风险,L3 需确认)。",
risk_level="L3",
permission_policy=PermissionPolicy.always_ask,
build_request=_shell_request,
side_effect=True,
# 与文件类工具统一默认工作目录(workspace/<角色>):否则 shell 跑在沙箱根目录,cat/ls
# 看不到 read_file/write_file 产生的文件,模型会反复试不同路径直到循环截断。shell 本是
# 任意命令,cd 仍可改目录(沙箱路径逃逸保护另兜底),这里只统一默认起点。
isolated_workspace=True,
)
FETCH_URL = ToolSpec(
name="fetch_url",
description="抓取指定 URL 的网页正文(转 markdown)。",
risk_level="L2",
permission_policy=PermissionPolicy.ask,
side_effect=True,
audit_required=True,
executor=_fetch_url_executor,
)
def builtin_specs() -> dict[str, ToolSpec]:
"""返回内置工具规格字典。"""
return {
spec.name: spec for spec in [DATETIME, READ_FILE, LIST_DIR, WRITE_FILE, SHELL, FETCH_URL]
}
@@ -1,82 +0,0 @@
"""MCP 工具转 ToolSpec(executor 绑定 http_client 调 MCP server tools/call)。
MCP 工具命名 ``mcp_{server_name}_{tool_name}``(下划线分隔,冒号不含 OpenAI 命名规范),``risk_level`` 缺省 L2(外部工具),
``permission_policy`` 由 risk 经 ``default_policy`` 推导。executor 闭包捕获
http_client + server endpoint + auth_token。
协议:标准 MCP Streamable HTTP(JSON-RPC 2.0 over POST /message)
"""
from app.integrations.mcp.client import call_tool
from app.modules.safety.governor import default_policy
from app.modules.tool_runtime.definition import ToolSpec
def make_mcp_tool_specs(
server,
tools: list[dict],
http_client,
meta: dict | None = None,
transport=None,
) -> list[ToolSpec]:
"""把 MCP server 工具清单(list[dict])转为 ToolSpec 列表。
Args:
meta: 可选业务上下文
transport: 可选传输层实例(stdio 用,传给 executor 替代 HTTP call_tool)
"""
specs: list[ToolSpec] = []
for tool in tools:
name = tool.get("name")
if not name:
continue
risk_level = tool.get("risk_level", "L2")
input_schema = tool.get("inputSchema") or tool.get("input_schema")
specs.append(
ToolSpec(
name=f"mcp_{server.name}_{name}",
description=tool.get("description", ""),
risk_level=risk_level,
permission_policy=default_policy(risk_level),
executor=_make_mcp_executor(
http_client,
server.endpoint_url,
name,
server.auth_token,
meta=meta,
transport=transport,
),
input_schema=input_schema,
)
)
return specs
def _make_mcp_executor(
http_client,
endpoint_url,
tool_name,
auth_token,
meta: dict | None = None,
transport=None,
):
async def executor(input, context): # noqa: ARG001
call_meta = dict(meta or {})
if context:
user_role = context.get("user_role") or "free"
call_meta.setdefault("user_role", user_role)
call_meta.setdefault("caller", "hunjing")
if context.get("trace_id"):
call_meta["trace_id"] = context["trace_id"]
if transport is not None:
return await transport.call_tool(tool_name, input)
return await call_tool(
http_client,
endpoint_url,
tool_name,
input,
auth_token,
meta=call_meta or None,
)
return executor
@@ -1,529 +0,0 @@
"""百工 Tool Runtime:司命评估 → 直接执行/沙箱执行 → 一炁事件回写 的编排核心。
调用链:
ToolRuntime.execute / execute_stream
→ SafetyGovernor.evaluate(deny→记 tool.failed 并抛 PermissionDenied;
ask→抛 RequiresConfirmation 交上层确认后重试)
→ 执行路径二选一:
- ToolSpec.executor 非 None:构造执行上下文(含 http_client 等)
直接调用 executor,跳过沙箱(HTTP 类工具如 fetch_url)。
- executor 为 None:ToolSpec.build_request 构造沙箱请求(isolated_workspace 工具按
run 隔离 workdir)→ SandboxClient.exec / exec_stream(携带司命授权上下文;
事件经 EventStoreSink 落 EventStore)。
→ ToolResult / 流式 ToolStreamEvent
"""
from __future__ import annotations
import asyncio
import re
import uuid
from collections.abc import AsyncIterator
from contextlib import suppress
from dataclasses import dataclass
from typing import Any
import httpx
from app.db.session import AsyncSessionLocal
from app.integrations.sandbox.audit_sink import AuditEventSink
from app.integrations.sandbox.auth import AuthorizationContext
from app.integrations.sandbox.bridge import EventStoreSink
from app.integrations.sandbox.client import SandboxClient
from app.integrations.sandbox.events import (
CompositeEventSink,
EventSink,
completed_event,
failed_event,
started_event,
)
from app.integrations.sandbox.exceptions import SandboxError
from app.integrations.sandbox.models import (
DirEntry,
ExecResponse,
FileReadRequest,
FileReadResponse,
)
from app.modules.runtime.events import Event, EventStore
from app.modules.safety.exceptions import PermissionDenied, RequiresConfirmation
from app.modules.safety.governor import SafetyDecision, SafetyGovernor
from app.modules.tool_runtime.definition import ToolInput, ToolResult, ToolSpec
@dataclass(frozen=True)
class ToolStreamEvent:
"""工具流式事件:stdout/stderr 增量(base64 str,透传给前端解码)或 done(结束汇总 dict)。"""
kind: str # "stdout" | "stderr" | "done"
payload: str | dict[str, Any]
class ToolRuntime:
"""百工工具运行时。"""
_ISOLATE_SAFE = re.compile(r"[^A-Za-z0-9_.-]")
def __init__(
self,
sandbox: SandboxClient,
governor: SafetyGovernor,
event_store: EventStore,
specs: dict[str, ToolSpec] | None = None,
*,
http_client: httpx.AsyncClient | None = None,
) -> None:
self._sandbox = sandbox
self._governor = governor
self._store = event_store
self._http_client = http_client
# 不在启动期快照 builtin_specs()——那样删除/修改内置工具后需重启后端才生效。
# builtin 工具每次经 _resolve_specs() 实时取(builtin_specs 是纯函数,返回新 dict)。
# 调用方显式传入的 specs 视为"覆盖层",与 register 的工具一起叠加在 builtin 之上。
self._custom_specs: dict[str, ToolSpec] = dict(specs) if specs else {}
self._registered_specs: dict[str, ToolSpec] = {}
@property
def http_client(self) -> httpx.AsyncClient | None:
"""共享 HTTP 客户端(供 MCP executor 等复用,生命周期同 runtime)。"""
return self._http_client
def _resolve_specs(self) -> dict[str, ToolSpec]:
"""合并 builtin(实时)+ custom(调用方传入)+ registered(动态注册)。
后者覆盖前者同名,保证覆盖语义;builtin 每次现取,builtin_specs() 的任何变更
(如新增/删除/修改内置工具)立即对当前 ToolRuntime 实例生效,无需重启。
"""
from app.application.tool_runtime.builtins import builtin_specs
merged: dict[str, ToolSpec] = {}
merged.update(builtin_specs())
merged.update(self._custom_specs)
merged.update(self._registered_specs)
return merged
def register(self, spec: ToolSpec) -> None:
"""注册或覆盖工具规格(叠加在 builtin 与调用方传入的 specs 之上)。"""
self._registered_specs[spec.name] = spec
def unregister(self, name: str) -> None:
"""移除动态注册的工具(builtin / custom 不受影响)。"""
self._registered_specs.pop(name, None)
async def reload_governor(self) -> None:
"""从 tool_policies 表重新加载司命策略(写操作后调用,使变更即时生效)。
读取所有 enabled 策略构造 SafetyGovernor 覆盖当前实例。governor 的 policy
字典作为覆盖层(见 SafetyGovernor.evaluate):命中的工具用表里的 risk_level /
permission_policy,未命中的用 ToolSpec 自带 risk_level 推导默认策略。
"""
from app.db.session import AsyncSessionLocal
from app.repositories import governance as governance_repo
async with AsyncSessionLocal() as session:
policies = await governance_repo.list_tool_policies(session)
data = {
"tools": {
p.name: {
"risk_level": p.risk_level,
"permission_policy": p.permission_policy,
}
for p in policies
if p.enabled
}
}
self._governor = SafetyGovernor.from_policy_dict(data)
def list_specs(self) -> list[ToolSpec]:
"""返回当前可用工具规格(用于 API 暴露工具元数据)。
builtin 实时合并:删除/修改 builtin_specs() 中的工具定义后无需重启即可生效。
"""
return list(self._resolve_specs().values())
async def execute(
self,
tool: str,
input: ToolInput,
*,
run_id: str | None = None,
character_id: str | None = None,
trace_id: str | None = None,
confirmed: bool = False,
context: dict[str, Any] | None = None,
) -> ToolResult:
cancellation = (context or {}).get("cancellation")
if cancellation is not None:
cancellation.raise_if_cancelled()
spec = self._resolve_specs().get(tool)
if spec is None:
await self._record_failed(
tool, run_id, character_id, trace_id, "permission_denied", f"未知工具: {tool}"
)
raise PermissionDenied(f"未知工具: {tool}")
decision = self._governor.evaluate(
tool=tool, risk_level=spec.risk_level, confirmed=confirmed
)
if decision.decision is SafetyDecision.deny:
await self._record_failed(
tool, run_id, character_id, trace_id, "permission_denied", decision.reason
)
raise PermissionDenied(decision.reason)
if decision.decision is SafetyDecision.ask:
# 不执行;上层确认后以 confirmed=True 重试。携带 risk_level 供前端展示。
raise RequiresConfirmation(decision.reason, risk_level=decision.risk_level)
# ---- 直接执行路径(executor 工具,如 fetch_url)----
if spec.executor is not None:
exec_context = self._build_exec_context(context, trace_id=trace_id)
sink = self._make_sink(run_id, character_id)
await sink.emit(
started_event(
tool=spec.name,
argv=[],
risk_level=spec.risk_level,
trace_id=trace_id,
run_id=run_id,
)
)
try:
result = await spec.executor(input, exec_context)
except Exception as exc: # noqa: BLE001 - executor 任意错误统一兜底
await sink.emit(
failed_event(
tool=spec.name,
trace_id=trace_id,
run_id=run_id,
error_code="executor_failed",
error_message=str(exc),
)
)
raise SandboxError(f"工具 {spec.name} 直接执行失败: {exc}") from exc
await sink.emit(
completed_event(
tool=spec.name,
trace_id=trace_id,
run_id=run_id,
exit_code=result.exit_code if result.exit_code is not None else 0,
duration_ms=result.duration_ms,
timeout=result.timeout,
truncated=result.truncated,
stdout_len=len(result.stdout),
stderr_len=len(result.stderr),
)
)
# 补齐 trace_id(executor 可能未填)
if not result.trace_id:
result = ToolResult(
ok=result.ok,
exit_code=result.exit_code,
stdout=result.stdout,
stderr=result.stderr,
duration_ms=result.duration_ms,
timeout=result.timeout,
truncated=result.truncated,
trace_id=trace_id or "",
)
return result
# ---- 沙箱执行路径 ----
request = spec.build_request(input)
if trace_id and not request.trace_id:
request.trace_id = trace_id
# 文件类工具按角色隔离工作目录(同一角色跨 run 文件互通,不同角色互不可见)
if spec.isolated_workspace:
request.workdir = self._isolate_workdir(
request.workdir, subject_id=character_id, run_id=run_id, trace_id=trace_id
)
auth_ctx = AuthorizationContext(
trace_id=request.trace_id or trace_id or "",
risk_level=spec.risk_level,
tool=spec.name,
character_id=character_id,
run_id=run_id,
)
# 本次调用绑定带 run/character 上下文的 sink,使沙箱事件落入一炁事件存储与审计表
sink = self._make_sink(run_id, character_id)
try:
resp: ExecResponse = await self._sandbox.exec(request, auth_ctx, event_sink=sink)
except asyncio.CancelledError:
# 取消 HTTP await 不代表远端子进程已停止;尽力调用沙箱 cancel,随后保持取消语义。
if request.trace_id:
with suppress(Exception):
await asyncio.shield(self._sandbox.cancel(request.trace_id, auth_ctx))
raise
except SandboxError:
# 沙箱错误事件已由 sink 记录(tool.failed),此处不重复写入
raise
return ToolResult(
ok=resp.ok,
exit_code=resp.exit_code,
stdout=resp.stdout,
stderr=resp.stderr,
duration_ms=resp.duration_ms,
timeout=resp.timeout,
truncated=resp.truncated,
trace_id=resp.trace_id,
)
async def execute_stream(
self,
tool: str,
input: ToolInput,
*,
run_id: str | None = None,
character_id: str | None = None,
trace_id: str | None = None,
confirmed: bool = False,
) -> AsyncIterator[ToolStreamEvent]:
"""流式执行:司命评估后转发沙箱 stdout/stderr 增量,末尾一个 done。
started/completed/failed 落一炁事件存储;stdout/stderr 增量只透传不落库。
司命 deny 记 tool.failed 并抛 PermissionDenied;ask 抛 RequiresConfirmation
(流式端点要求调用方先经非流式 invoke 完成 confirmation,再以 confirmed=True 调用)。
"""
spec = self._resolve_specs().get(tool)
if spec is None:
await self._record_failed(
tool, run_id, character_id, trace_id, "permission_denied", f"未知工具: {tool}"
)
raise PermissionDenied(f"未知工具: {tool}")
decision = self._governor.evaluate(
tool=tool, risk_level=spec.risk_level, confirmed=confirmed
)
if decision.decision is SafetyDecision.deny:
await self._record_failed(
tool, run_id, character_id, trace_id, "permission_denied", decision.reason
)
raise PermissionDenied(decision.reason)
if decision.decision is SafetyDecision.ask:
raise RequiresConfirmation(decision.reason, risk_level=decision.risk_level)
request = spec.build_request(input)
if trace_id and not request.trace_id:
request.trace_id = trace_id
if spec.isolated_workspace:
request.workdir = self._isolate_workdir(
request.workdir, subject_id=character_id, run_id=run_id, trace_id=trace_id
)
effective_trace = request.trace_id or trace_id or ""
auth_ctx = AuthorizationContext(
trace_id=effective_trace,
risk_level=spec.risk_level,
tool=spec.name,
character_id=character_id,
run_id=run_id,
)
sink = self._make_sink(run_id, character_id)
await sink.emit(
started_event(
tool=spec.name,
argv=request.argv,
risk_level=spec.risk_level,
trace_id=effective_trace or None,
run_id=run_id,
)
)
final: dict[str, Any] = {}
try:
async for ev in self._sandbox.exec_stream(request, auth_ctx):
if ev.kind in ("stdout", "stderr"):
yield ToolStreamEvent(ev.kind, ev.payload)
elif ev.kind == "done" and isinstance(ev.payload, dict):
final = ev.payload
except SandboxError:
await sink.emit(
failed_event(
tool=spec.name,
trace_id=effective_trace or None,
run_id=run_id,
error_code="sandbox_error",
error_message="沙箱流式执行失败",
)
)
raise
await sink.emit(
completed_event(
tool=spec.name,
trace_id=effective_trace or None,
run_id=run_id,
exit_code=final.get("exit_code"),
duration_ms=int(final.get("duration_ms", 0)),
timeout=bool(final.get("timeout", False)),
truncated=bool(final.get("truncated", False)),
stdout_len=0, # 流式不收集完整文本
stderr_len=0,
)
)
yield ToolStreamEvent("done", final)
def _make_sink(self, run_id: str | None, character_id: str | None) -> EventSink:
"""构造本次执行的事件 sink 组合:一炁流程事件 + 治理审计,互不感知。
EventStoreSink 落 events 表(流程,run 内 sequence),AuditEventSink 落 audit_events
表(治理审计,含 decision / risk_level / argv 脱敏摘要)。两者经 CompositeEventSink 扇出。
"""
return CompositeEventSink(
[
EventStoreSink(self._store, run_id=run_id, character_id=character_id),
AuditEventSink(AsyncSessionLocal, run_id=run_id, character_id=character_id),
]
)
def _isolate_workdir(
self,
base: str | None,
*,
subject_id: str | None,
run_id: str | None,
trace_id: str | None,
) -> str:
"""把工具 workspace 根目录隔离到「主体」级子目录,防跨主体信息泄漏。
隔离主体 subject_id:当前为 character_id(一个角色一个持久 workspace,跨会话/
跨 run 文件互通,符合"一个角色拥有其全部文件"的语义);多用户上线后应改为真实
用户标识(owner_scope / user_id),届时仅需调用方改传参,本函数逻辑不变。
run_id / trace_id 仅在 subject_id 缺失时回退(如脱离聊天流程的直接调用),
保证既有隔离又不至于因缺主体而落到共享根目录。
subject_id 经 sanitize(仅保留 ``[A-Za-z0-9_.-]``)从源头杜绝路径注入;
沙箱端另有 canonicalize + ``starts_with(base)`` 兜底。
"""
root = base or "workspace"
key = subject_id or run_id or trace_id or f"adhoc_{uuid.uuid4().hex[:8]}"
safe = self._ISOLATE_SAFE.sub("_", key).strip("._-") or "adhoc"
return f"{root}/{safe}"
def _build_exec_context(
self, context: dict[str, Any] | None, *, trace_id: str | None
) -> dict[str, Any]:
"""构造 executor 执行上下文:合并 runtime 持有的 http_client、settings 中的密钥等。
调用方传入的 context(per-request 覆盖)优先于 runtime 默认值。
"""
exec_context: dict[str, Any] = {"trace_id": trace_id}
if self._http_client is not None:
exec_context["http_client"] = self._http_client
if context:
exec_context.update(context)
return exec_context
async def read_sandbox_file(
self,
path: str,
*,
run_id: str | None = None,
character_id: str | None = None,
trace_id: str | None = None,
isolated: bool = True,
) -> FileReadResponse:
"""读取沙箱内文件(仅供文件预览端点调用)。
Args:
path: 文件路径。
isolated=True 时视为工作区相对路径(write_file 传入的 path),
自动补全 ``workspace/<character_id>/`` 隔离前缀;
isolated=False 时作为沙箱 base 下的完整相对路径。
run_id: 关联的 run ID(可选,用于授权上下文)。
character_id: 人物 ID(可选)。提供时自动构造隔离工作区路径。
trace_id: 追踪 ID(可选,自动生成)。
isolated: 是否按 isolated_workspace 规则补全路径前缀。
Returns:
FileReadResponse,含 base64url-no-pad 编码的文件内容。
"""
effective_path = path
if isolated:
workdir = self._isolate_workdir(
"workspace", subject_id=character_id, run_id=run_id, trace_id=trace_id
)
effective_path = f"{workdir}/{path.lstrip('/')}"
effective_trace = trace_id or str(uuid.uuid4())
auth_ctx = AuthorizationContext(
trace_id=effective_trace,
risk_level="L0",
tool="file_read",
character_id=character_id,
run_id=run_id,
)
return await self._sandbox.file_read(
FileReadRequest(path=effective_path),
auth_ctx,
)
async def list_sandbox_dir(
self,
path: str = ".",
*,
run_id: str | None = None,
character_id: str | None = None,
trace_id: str | None = None,
isolated: bool = True,
) -> list[DirEntry]:
"""列出沙箱工作区目录(仅供文件浏览端点调用)。
Args:
path: 目录路径,默认工作区根。
isolated: True 时自动应用 isolated_workspace 隔离前缀。
返回排序后的目录条目列表。
"""
from app.integrations.sandbox.models import FileListRequest
effective_path = path
if isolated:
workdir = self._isolate_workdir(
"workspace", subject_id=character_id, run_id=run_id, trace_id=trace_id
)
effective_path = f"{workdir}/{path.lstrip('/')}"
effective_trace = trace_id or str(uuid.uuid4())
auth_ctx = AuthorizationContext(
trace_id=effective_trace,
risk_level="L0",
tool="file_list",
character_id=character_id,
run_id=run_id,
)
result = await self._sandbox.file_list(
FileListRequest(path=effective_path),
auth_ctx,
)
return result.entries
async def _record_failed(
self,
tool: str,
run_id: str | None,
character_id: str | None,
trace_id: str | None,
error_code: str,
error_message: str,
) -> None:
await self._store.append(
Event(
type="tool.failed",
source="tool",
visibility="public",
trace_id=trace_id,
run_id=run_id,
character_id=character_id,
payload={
"tool": tool,
"phase": "safety",
"error_code": error_code,
"error_message": error_message,
},
)
)
@@ -1,57 +0,0 @@
"""单个 Agent Run 可见的工具视图。"""
from __future__ import annotations
from dataclasses import replace
from typing import Any
from app.application.tool_runtime.runtime import ToolRuntime
from app.modules.tool_runtime.definition import ToolInput, ToolResult, ToolSpec
class RunScopedToolView:
"""保存 Run 级工具别名;实际执行始终委托共享 ToolRuntime。"""
def __init__(self, runtime: ToolRuntime) -> None:
self._runtime = runtime
self._aliases: dict[str, tuple[str, ToolSpec]] = {}
def bind_alias(self, alias: str, target: str, *, description_prefix: str = "") -> bool:
"""将已有工具作为 Run 级别名暴露;目标不存在时返回 False。"""
spec = next((item for item in self._runtime.list_specs() if item.name == target), None)
if spec is None:
return False
description = f"{description_prefix}{spec.description}" if description_prefix else spec.description
self._aliases[alias] = (target, replace(spec, name=alias, description=description))
return True
def list_specs(self) -> list[ToolSpec]:
return [*self._runtime.list_specs(), *(spec for _, spec in self._aliases.values())]
def specs_by_name(self) -> dict[str, ToolSpec]:
return {spec.name: spec for spec in self.list_specs()}
async def execute(
self,
tool: str,
input: ToolInput,
*,
run_id: str | None = None,
character_id: str | None = None,
trace_id: str | None = None,
confirmed: bool = False,
context: dict[str, Any] | None = None,
) -> ToolResult:
target = self._aliases.get(tool, (tool,))[0]
return await self._runtime.execute(
target,
input,
run_id=run_id,
character_id=character_id,
trace_id=trace_id,
confirmed=confirmed,
context=context,
)
__all__ = ["RunScopedToolView"]
@@ -1,127 +0,0 @@
"""HTTP 类工具的直接执行器(非沙箱路径)。
``fetch_url``:抓取指定 URL 的 HTML 并用 html2text 转成 markdown。
executor 上下文由 ``ToolRuntime._build_exec_context`` 注入,至少包含:
- ``http_client``:长生命周期的 ``httpx.AsyncClient``(禁止裸 ``async with httpx.AsyncClient()``)。
- ``trace_id``:本次执行的 trace_id(可空)。
设计约束(见 backend/CLAUDE.md):
- 禁止在调用处裸 ``async with httpx.AsyncClient()``,必须复用注入的 client。
- HTTP 错误统一返回 ``ToolResult(ok=False, stderr=...)``,不抛异常(让模型自决策)。
"""
from __future__ import annotations
from typing import Any
import html2text
import httpx
from app.core.config import settings
from app.modules.tool_runtime.definition import ToolInput, ToolResult
# 默认参数
_DEFAULT_MAX_CHARS = 8000
# 合理的浏览器 UA,避免被部分站点拒绝
_USER_AGENT = "Mozilla/5.0 (compatible; HunjingBot/1.0; +https://github.com/hunjing/hunjing)"
# 模块级 html2text 配置(避免每次调用都构造配置对象)
_H2T = html2text.HTML2Text()
_H2T.ignore_links = False
_H2T.ignore_images = True
_H2T.body_width = 0 # 不折行,交给上层截断
def _get_http_client(context: dict[str, Any] | None) -> httpx.AsyncClient | None:
"""从 context 取注入的 httpx.AsyncClient;缺失返回 None。"""
if context is None:
return None
client = context.get("http_client")
return client if isinstance(client, httpx.AsyncClient) else None
def _ok_result(stdout: str, trace_id: str = "") -> ToolResult:
return ToolResult(
ok=True,
exit_code=0,
stdout=stdout,
stderr="",
duration_ms=0,
timeout=False,
truncated=False,
trace_id=trace_id,
)
def _err_result(stderr: str, trace_id: str = "") -> ToolResult:
return ToolResult(
ok=False,
exit_code=1,
stdout="",
stderr=stderr,
duration_ms=0,
timeout=False,
truncated=False,
trace_id=trace_id,
)
async def _fetch_url_executor(input: ToolInput, context: dict[str, Any] | None) -> ToolResult:
"""抓取指定 URL 的网页正文并转 markdown。
input:
url (str, required): 要抓取的 URL。
max_chars (int, optional): 正文最大字符数,默认 8000。
context:
http_client (httpx.AsyncClient): 注入的 HTTP 客户端。
trace_id (str): 本次执行 trace_id。
"""
trace_id = (context or {}).get("trace_id", "") or ""
url = input.get("url")
if not isinstance(url, str) or not url.strip():
return _err_result("fetch_url 需要 'url' 字符串输入。", trace_id)
raw_max = input.get("max_chars", _DEFAULT_MAX_CHARS)
try:
max_chars = int(raw_max) if raw_max is not None else _DEFAULT_MAX_CHARS
except (TypeError, ValueError):
max_chars = _DEFAULT_MAX_CHARS
if max_chars < 1:
max_chars = _DEFAULT_MAX_CHARS
client = _get_http_client(context)
if client is None:
return _err_result("fetch_url 缺少注入的 http_client。", trace_id)
try:
resp = await client.get(
url,
headers={"User-Agent": _USER_AGENT},
timeout=settings.tool_web_http_timeout_seconds,
follow_redirects=True,
)
except httpx.HTTPError as exc:
return _err_result(f"fetch_url 请求失败:{exc}", trace_id)
if resp.status_code >= 400:
return _err_result(f"fetch_url 返回 HTTP {resp.status_code}", trace_id)
content_type = resp.headers.get("content-type", "").lower()
html_text = resp.text
# 无论 content-type 如何都尝试转 markdown(部分站点 content-type 不规范)
try:
markdown = _H2T.handle(html_text)
except Exception: # noqa: BLE001 - html2text 解析失败时回退原文
markdown = html_text
# 规范化多余空白并截断
cleaned = "\n".join(line.rstrip() for line in markdown.splitlines())
cleaned = cleaned.strip()
if len(cleaned) > max_chars:
cleaned = f"{cleaned[:max_chars]}\n\n[...已截断,共 {len(cleaned)} 字符]"
if not cleaned:
return _err_result(f"fetch_url 未抓取到正文内容(content-type={content_type})。", trace_id)
return _ok_result(cleaned, trace_id)
-3
View File
@@ -5,9 +5,6 @@
- ``.env`` 永远放在项目根目录(backend 父级),``Settings`` 初始化时以**绝对路径** - ``.env`` 永远放在项目根目录(backend 父级),``Settings`` 初始化时以**绝对路径**
通过 ``_env_file`` 传入,不使用相对路径,不依赖 ``python-dotenv``。 通过 ``_env_file`` 传入,不使用相对路径,不依赖 ``python-dotenv``。
- ``ai_provider/`` 只**消费**本目录的配置,不持有自己的
``settings.py`` / ``config.py``;领域数据结构(``ModelConfig`` 等)保留在
``ai_provider/config/models.py``。
""" """
from app.core.config.sandbox_config import SandboxSettings from app.core.config.sandbox_config import SandboxSettings
-1
View File
@@ -30,7 +30,6 @@ class AppConfig(BaseSettings):
) )
# ==================== 文件存储 ==================== # ==================== 文件存储 ====================
avatar_dir: str = "../db/avatars" # 角色头像文件存储目录(相对 backend)
# ==================== 默认管理员用户 ==================== # ==================== 默认管理员用户 ====================
default_admin_username: str = "admin" # 种子默认管理员用户名 default_admin_username: str = "admin" # 种子默认管理员用户名
-46
View File
@@ -1,46 +0,0 @@
"""模型网关(神识)env 配置:各 provider 的 API Key / Base URL 与 endpoint 池策略。
注意区分**配置字段**与**领域数据结构**:
- 本文件只承载「从 env 读的开关 / 参数 / 密钥」。
- 领域数据结构(``ModelConfig`` / ``ModelEndpointConfig``,从 ``config/models.yaml`` 加载)
保留在 ``ai_provider/config/models.py``,不外迁到 ``core/config``。
"""
from __future__ import annotations
from pydantic_settings import BaseSettings
class ModelGatewayConfig(BaseSettings):
"""模型网关 env 配置。"""
# ==================== 模型配置文件 ====================
model_config_path: str = "../config/models.yaml" # 模型配置文件路径(相对 backend)
# ==================== provider API Key / Base URL ====================
openai_api_key: str = "" # OpenAI API Key
openai_base_url: str = "https://api.openai.com/v1" # OpenAI API 地址
deepseek_api_key: str = "" # DeepSeek API Key
deepseek_base_url: str = "https://api.deepseek.com" # DeepSeek API 地址(OpenAI 兼容)
xiaomi_mimo_api_key: str = "" # 小米 MiMo API Key
xiaomi_mimo_base_url: str = "https://api.xiaomimimo.com/v1" # 小米 MiMo API 地址
anthropic_api_key: str = "" # Anthropic API Key
anthropic_base_url: str = "https://api.anthropic.com" # Anthropic API 地址
volcengine_ark_api_key: str = "" # 火山方舟 / 豆包 API Key
volcengine_ark_base_url: str = "https://ark.cn-beijing.volces.com/api/v3" # 火山方舟 API 地址
custom_llm_api_key: str = "" # 自定义 LLM API Key
custom_llm_base_url: str = "" # 自定义 LLM API 地址
yuanqi_api_key: str = "" # 元炁本地推理引擎 API Key
yuanqi_base_url: str = "http://localhost:11434" # 元炁本地推理引擎 API 地址
# ==================== endpoint 池配置 ====================
# 物理 endpoint 池路由、failover 与重试策略(见 ai_provider/pool.py 与 resilience.py)
model_endpoint_pool_enabled: bool = True # 是否启用 endpoint 池路由(False 时走逻辑模型直连)
model_endpoint_pool_dry_run_selection: bool = False # dry-run:只预选并记录,不实际路由
model_endpoint_pool_expose_debug_headers: bool = False # 是否在响应头暴露 endpoint 路由调试信息
model_endpoint_pool_agent_failover_enabled: bool = (
False # Agent 调用是否允许跨 endpoint failover
)
model_endpoint_pool_per_endpoint_max_retries: int = 1 # 单 endpoint 失败时的本地重试次数
model_endpoint_pool_retry_base_delay_seconds: float = 0.25 # endpoint 本地重试基础退避(秒)
model_endpoint_pool_retry_max_delay_seconds: float = 2.0 # endpoint 本地重试最大退避(秒)
+11
View File
@@ -81,6 +81,17 @@ class RuntimeConfig(BaseSettings):
chroma_collection: str = "hunjing_memories" # ChromaDB collection 名 chroma_collection: str = "hunjing_memories" # ChromaDB collection 名
chroma_persist_directory: str = "../db/chroma" # ChromaDB 持久化目录(相对 backend) chroma_persist_directory: str = "../db/chroma" # ChromaDB 持久化目录(相对 backend)
embedding_model_hint: str = "bge-m3" # embedding 模型逻辑名 embedding_model_hint: str = "bge-m3" # embedding 模型逻辑名
# 叩心约束(PRD §6.6:总开关/静默窗口/频率/预算;候选默认不自动发送)
knocking_enabled: bool = True
knocking_quiet_start: str = "23:00" # HH:MM(本地时区)
knocking_quiet_end: str = "07:00"
knocking_daily_limit: int = 3 # 24 小时派发上限
knocking_timezone: str = "Asia/Shanghai" # 静默窗口判定时区
knocking_budget_total: int = 0 # 累计预算(0=不限)
# 元炁本地推理引擎(ASR 语音转写;未配置=多模态关闭,文字链路不受影响)
yuanqi_base_url: str = ""
yuanqi_asr_model: str = "iic/SenseVoiceSmall"
# ==================== 自动记忆候选 [PLANNED] ==================== # ==================== 自动记忆候选 [PLANNED] ====================
enable_memory_candidates: bool = False # 是否启用自动记忆候选抽取 enable_memory_candidates: bool = False # 是否启用自动记忆候选抽取
+1 -25
View File
@@ -1,7 +1,7 @@
"""主 Settings:多继承所有子配置,统一以绝对路径 ``_env_file`` 加载 ``.env``。 """主 Settings:多继承所有子配置,统一以绝对路径 ``_env_file`` 加载 ``.env``。
子配置见同目录下的 ``app_config`` / ``auth_config`` / ``database_config`` / ``log_config`` / 子配置见同目录下的 ``app_config`` / ``auth_config`` / ``database_config`` / ``log_config`` /
``runtime_config`` / ``model_config`` / ``sandbox_config``。本文件聚合字段,并承载 ``runtime_config`` / ``sandbox_config``。本文件聚合字段,并承载
跨子配置的路径解析与计算属性。 跨子配置的路径解析与计算属性。
配置优先级(自高到低):命令行参数 > 系统环境变量 > ``.env`` > 类属性默认值。 配置优先级(自高到低):命令行参数 > 系统环境变量 > ``.env`` > 类属性默认值。
@@ -18,7 +18,6 @@ from app.core.config.app_config import AppConfig
from app.core.config.auth_config import AuthConfig from app.core.config.auth_config import AuthConfig
from app.core.config.database_config import DatabaseConfig from app.core.config.database_config import DatabaseConfig
from app.core.config.log_config import LogConfig from app.core.config.log_config import LogConfig
from app.core.config.model_config import ModelGatewayConfig
from app.core.config.runtime_config import RuntimeConfig from app.core.config.runtime_config import RuntimeConfig
from app.core.config.sandbox_config import SandboxSettings from app.core.config.sandbox_config import SandboxSettings
from app.core.config.time_config import TimeConfig from app.core.config.time_config import TimeConfig
@@ -35,7 +34,6 @@ class Settings(
DatabaseConfig, DatabaseConfig,
LogConfig, LogConfig,
RuntimeConfig, RuntimeConfig,
ModelGatewayConfig,
SandboxSettings, SandboxSettings,
TimeConfig, TimeConfig,
BaseSettings, BaseSettings,
@@ -117,28 +115,6 @@ class Settings(
return example return example
return configured return configured
@cached_property
def resolved_model_config_path(self) -> Path:
"""解析模型配置文件路径。
如果配置文件不存在,尝试使用示例配置文件。
"""
configured = self.resolve_from_backend(self.model_config_path)
if configured.exists():
return configured
# 如果配置文件不存在,尝试使用示例配置
example = self.project_root / "config" / "models.example.yaml"
if example.exists():
return example
return configured
@cached_property
def resolved_avatar_dir(self) -> Path:
"""解析角色头像存储目录为绝对路径。"""
path = self.resolve_from_backend(self.avatar_dir)
path.mkdir(parents=True, exist_ok=True)
return path
# 全局配置实例 # 全局配置实例
settings = Settings() settings = Settings()
+10 -1
View File
@@ -11,7 +11,16 @@ from starlette.exceptions import HTTPException as StarletteHTTPException
from app.core.ids import new_trace_id from app.core.ids import new_trace_id
from app.core.logging import log_with_context from app.core.logging import log_with_context
from app.core.security import redact_value from app.core.security import redact_value
from app.modules.safety.exceptions import SafetyError class SafetyError(Exception):
"""司命决策异常(错误码映射契约字典 §15)。"""
def __init__(self, code: str, message: str) -> None:
super().__init__(message)
self.code = code
self.message = message
def __str__(self) -> str:
return self.message
trace_id_var: ContextVar[str | None] = ContextVar("trace_id", default=None) trace_id_var: ContextVar[str | None] = ContextVar("trace_id", default=None)
+5
View File
@@ -47,6 +47,11 @@ def new_personality_proposal_id() -> str:
return new_id("personality_proposal_") return new_id("personality_proposal_")
def new_confirmation_id() -> str:
"""确认项 ID(前缀 confirm_,契约词典 §6)。"""
return new_id("confirm")
def new_audit_id() -> str: def new_audit_id() -> str:
return new_id("audit_") return new_id("audit_")
Loaded 100 of 448 files, more files were not shown because too many files have changed in this diff. Show more