Skip to content

fix:vision_api.template.py cannot handle <thinking> and Add url support#420

Open
tqjason wants to merge 2 commits into
lsdefine:mainfrom
tqjason:fix/vision_api_template
Open

fix:vision_api.template.py cannot handle <thinking> and Add url support#420
tqjason wants to merge 2 commits into
lsdefine:mainfrom
tqjason:fix/vision_api_template

Conversation

@tqjason
Copy link
Copy Markdown
Contributor

@tqjason tqjason commented May 18, 2026

vision_api.py 调用的时候,如果模型返回的有 thinking 块,就不能处理,修复了这个问题。同时,现在逻辑应该处理不了流式输出,所以强制 stream=False

@tqjason tqjason force-pushed the fix/vision_api_template branch from b8c8813 to e34079a Compare May 21, 2026 00:41
@tqjason tqjason changed the title fix:vision_api.template.py cannot handle <thinking> fix:vision_api.template.py cannot handle <thinking> and Add url support May 21, 2026
@tqjason
Copy link
Copy Markdown
Contributor Author

tqjason commented May 21, 2026

图片理解除了能够处理本地图片文件以外,增加了支持网络 URL。这个场景是用浏览器上网查找资料,然后发现图片需要理解。这个时候呢,直接把图片的链接发给 ask_vision 函数就可以了,不用下载到本地再提交。很多多模态模型是支持提交 URL 的。

同时呢,我还写了一个 简单的 SOP,用来指导怎么用图像识别模型?但考虑不是所有人都需要,我没有提交上来。需要的自己创建这个 SOP 吧。

image_sop.md

# Image API SOP

图片理解(image understanding) API 用法,网络URL可直传,不用下载到本地

库 memory/vision_api.py

## 快速用法

```python
from vision_api import ask_vision
result = ask_vision(image)
```

## ask_vision 函数签名
```
def ask_vision(image_input, prompt="详细描述这张图片的内容", backend="claude", timeout=60, max_pixels=1_440_000):
# image_input: URL(str) 或 本地文件路径(str/Path) 或 PIL Image
# backend: 'claude'(默认) | 'openai' | 'modelscope'
# 返回 str:成功为模型回复,失败为 'Error: ...'
```

相应的在 L1 记忆里边,需要加一条索引。

global_mem_insight.txt

# [Global Memory Insight]
...

L3: memory_cleanup_sop | subagent | plan_sop | tmwebdriver_sop | autonomous_operation_sop | scheduled_task_sop 
| image_sop

...
...

Image understanding: image_sop

...
...

[RULES]
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant