完善文档
This commit is contained in:
1 parent
192ea96eb4
commit
efc08dab5e
82 files changed
+1455
-109
No files matched your search
@@ -0,0 +1,16 @@
|
||||
# Chunking
|
||||
|
||||
## 原则
|
||||
|
||||
优先保持语义完整性,而不是机械固定长度切分。
|
||||
|
||||
考虑:
|
||||
|
||||
- 标题层级
|
||||
- 段落
|
||||
- 代码块
|
||||
- 表格
|
||||
- 文档结构
|
||||
- Token 长度
|
||||
|
||||
Chunk 必须携带可追溯 metadata。
|
||||
Reference in new issue
Block a user