Initial commit: Market-Assistant standalone (Django + Vue + JD crawler copy)

Made-with: Cursor

.env.example

@@ -0,0 +1,35 @@
+# =============================================================================
+# 唯一环境变量模板：复制为同目录 .env 后填写（.env 勿提交仓库）
+#   copy .env.example .env   （Windows）
+#   cp .env.example .env     （Linux/macOS）
+# =============================================================================
+
+# --- 数据工作区根目录（可选）---
+# 不填时默认为本仓库根目录（market_assistant），数据写在 ./data/JD/。
+# 若数据盘与代码分离，可设为绝对路径（须可写，运行时会自动创建 data/JD）。
+# LOW_GI_PROJECT_ROOT=D:\data\low-gi-workspace
+
+# --- Django ---
+DJANGO_SECRET_KEY=please-change-me
+DJANGO_DEBUG=True
+DJANGO_ALLOWED_HOSTS=localhost,127.0.0.1
+
+# 可选：SQLite 绝对路径；不填则使用 backend/db.sqlite3
+# DJANGO_SQLITE_PATH=D:\PythonProject\Low GI\market_assistant\backend\db.sqlite3
+
+# --- 浏览器访问前端时的 Origin（开发：Vite 5173；生产改为实际域名）---
+CORS_ALLOWED_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
+CSRF_TRUSTED_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
+
+# --- LLM（配料图识别、竞品报告、策略稿；OpenAI 兼容接口，与 AI_crawler 共用本文件）---
+# OPENAI_API_KEY=sk-your-key-here
+# OPENAI_BASE_URL=https://llm.example.com/v1
+# OPENAI_VISION_MODEL=Qwen/Qwen3-Omni-30B-A3B
+# 纯文本优先：OPENAI_TEXT_MODEL 或 LLM_TEXT_MODEL；未设则回退到视觉模型名
+# OPENAI_TEXT_MODEL=
+# 别名：LLM_API_KEY、LLM_BASE_URL、LLM_MODEL
+
+# --- 可选：流水线侧 LLM 开关 ---
+# MA_SKIP_LLM_KEYWORD_SUGGEST=1
+# MA_ENABLE_LLM_COMMENT_SENTIMENT=1
+# MA_SKIP_LLM_COMMENT_SENTIMENT=1

.gitignore

@@ -0,0 +1,34 @@
+# 独立成库时使用本文件即可（勿提交密钥与本地产物）
+.env
+.env.local
+.env.*.local
+
+backend/db.sqlite3
+*.sqlite3
+
+backend/**/__pycache__/
+**/__pycache__/
+*.py[cod]
+*.pyc
+*.egg-info/
+.eggs/
+
+backend/runtime_cookies/
+
+# 京东登录 Cookie（本地放置，勿提交）
+backend/crawler_copy/jd_pc_search/common/jd_cookie.txt
+
+# 流水线 / 京东采集产出（默认在仓库根下 data/JD）
+/data/
+
+venv/
+.venv/
+
+.idea/
+.vscode/
+.cursor/
+
+frontend/node_modules/
+frontend/dist/
+
+backend/crawler_copy/jd_pc_search/node_modules/

README.md

@@ -0,0 +1,120 @@
+# Market-Assistant（低 GI / 京东采集与竞品分析）
+
+**本目录可作为独立 Git 仓库根目录**：克隆后只需配置 `.env` 与数据区，即可部署并实现当前工作台全部能力（任务、入库、浏览、报告、策略、LLM 等）。代码不依赖仓库外的 `crawler/` 等目录；爬虫副本在 `backend/crawler_copy/jd_pc_search`。
+
+面向「前台事业部」的 Web 工作台：提交京东关键词采集任务、查看流水线产出、**库内分页浏览**已入库的搜索/商详/评价数据、生成竞品分析报告，并支持导出 JSON / CSV / Excel。
+
+跑批 CSV、`pipeline_runs` 等默认写在**本仓库根目录**下的 `data/JD/`；若数据需放在其它磁盘，可在 `.env` 中设置 **LOW_GI_PROJECT_ROOT** 为绝对路径。
+
+**环境变量**：全栈**只使用一份** `market_assistant/.env`（模板为 `.env.example`），勿在仓库根或其它子目录再建第二份 `.env`。
+
+**研发对接**：任务产物、状态与 REST 能力见项目内 **流水线输出说明** 与 **OpenAPI 子集**；部署与 Git 整理见 **docs/DEPLOY_AND_GIT.md**。
+
+---
+
+## 技术栈
+
+| 部分 | 说明 |
+|------|------|
+| 后端 | Django 5 + Django REST Framework，SQLite（可配置路径） |
+| 前端 | Vue 3 + Vite 5 + Vue Router，开发时通过 Vite 代理访问 API |
+| 采集 | 京东 PC 搜索侧脚本副本（与流水线任务联动） |
+
+---
+
+## 环境准备
+
+- **Python** 3.11+（建议虚拟环境）
+- **Node.js** 18+（用于前端）
+- **唯一**环境文件：在本目录（`market_assistant/`）执行：
+
+```bash
+copy .env.example .env
+```
+
+编辑 `.env`，至少设置：
+
+- **DJANGO_SECRET_KEY**；生产环境将 **DJANGO_DEBUG** 设为 False，并配置 **ALLOWED_HOSTS** 与 CORS/CSRF。
+- **LOW_GI_PROJECT_ROOT**（可选）：不填则数据写在仓库根下 `data/JD/`；单独数据盘时再填绝对路径。
+- 若使用配料识别、报告/策略 LLM：在同一文件填写 **OPENAI_*** 或 **LLM_***（与 `AI_crawler` 共用，无需另建 `.env`）。
+
+---
+
+## 启动后端
+
+在后端子目录下执行：
+
+```bash
+cd market_assistant/backend
+
+# 安装依赖（建议在 venv 中）
+pip install -r requirements.txt
+
+# 数据库迁移
+python manage.py migrate
+
+# 开发服务（默认 http://127.0.0.1:8000）
+python manage.py runserver
+```
+
+管理后台（可选）：创建超级用户后访问 Django 管理地址。
+
+---
+
+## 启动前端
+
+在前端子目录下执行：
+
+```bash
+cd market_assistant/frontend
+
+# 首次安装依赖
+npm install
+
+# 开发模式（默认 http://127.0.0.1:5173）
+npm run dev
+```
+
+浏览器打开本地开发地址。开发环境下，前端将 **API** 代理到后端端口，因此需**先启动后端**，再启动前端。
+
+其他脚本：
+
+```bash
+npm run build    # 生产构建
+npm run preview  # 本地预览构建结果
+```
+
+---
+
+## 常用功能说明
+
+1. **搜索采集**：创建京东关键词流水线任务（翻页、SKU 上限、Cookie 等；报告统计规则在「报告生成」）。  
+2. **任务与结果**：查看任务状态；成功任务可 **库内浏览**、文件预览与下载、导出。  
+3. **报告生成**：配置统计规则并重新生成分析报告文件。  
+4. **报告查看**：在线预览、单文件下载、加载结构化摘要、**一键下载简报包**（ZIP）。  
+5. **结构化摘要**：与报告同口径的规则化 JSON，供联调或其它工具使用。  
+6. **市场策略制定**：选成功任务，可选填业务备注，生成策略向 Markdown（目标、战场、定位选项、支柱与行动；规则版、非大模型）。  
+
+任务**成功结束后**会自动执行入库；也可在「库内浏览」里从批次目录重新入库。
+
+---
+
+## API 前缀
+
+开发时 REST 接口默认在后端根地址下的 **/api**；前端通过同源代理访问即可。
+
+---
+
+## 相关文档
+
+均在项目 **docs** 目录下，主要包括：项目进展与里程碑、流水线输出说明、演示与脱敏、工程说明等。
+
+**部署、Git 仓库整理、单 `.env` 约定**：见 [docs/DEPLOY_AND_GIT.md](docs/DEPLOY_AND_GIT.md)。
+
+---
+
+## 目录结构（简要）
+
+- **backend**：Django 与任务流水线 API  
+- **frontend**：Vue + Vite 工作台  
+- **docs**：说明、模板、演示与 OpenAPI 等  

backend/config/asgi.py

@@ -0,0 +1,16 @@
+"""
+ASGI config for config project.
+
+It exposes the ASGI callable as a module-level variable named ``application``.
+
+For more information on this file, see
+https://docs.djangoproject.com/en/5.2/howto/deployment/asgi/
+"""
+
+import os
+
+from django.core.asgi import get_asgi_application
+
+os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
+
+application = get_asgi_application()

backend/config/settings.py

@@ -0,0 +1,112 @@
+"""
+Django settings for config project.
+
+环境变量**仅**从 ``market_assistant/.env`` 加载（与 ``.env.example`` 同目录），
+勿再在 ``backend/.env`` 或仓库根目录使用第二份 .env，以免部署混淆。
+"""
+from pathlib import Path
+
+from dotenv import load_dotenv
+import os
+
+BASE_DIR = Path(__file__).resolve().parent.parent
+MA_ROOT = BASE_DIR.parent
+load_dotenv(MA_ROOT / ".env")
+
+_raw_low = (os.environ.get("LOW_GI_PROJECT_ROOT") or "").strip().strip('"').strip("'")
+if _raw_low:
+    LOW_GI_PROJECT_ROOT = str(Path(_raw_low).expanduser().resolve())
+else:
+    LOW_GI_PROJECT_ROOT = str(MA_ROOT.resolve())
+os.environ["LOW_GI_PROJECT_ROOT"] = LOW_GI_PROJECT_ROOT
+Path(LOW_GI_PROJECT_ROOT).joinpath("data", "JD").mkdir(parents=True, exist_ok=True)
+
+CRAWLER_JD_ROOT = BASE_DIR / "crawler_copy" / "jd_pc_search"
+
+SECRET_KEY = os.environ.get("DJANGO_SECRET_KEY", "django-insecure-dev-only-change-in-env")
+DEBUG = os.environ.get("DJANGO_DEBUG", "True").lower() in ("1", "true", "yes")
+ALLOWED_HOSTS = [
+    h.strip()
+    for h in os.environ.get("DJANGO_ALLOWED_HOSTS", "localhost,127.0.0.1").split(",")
+    if h.strip()
+]
+
+INSTALLED_APPS = [
+    "django.contrib.admin",
+    "django.contrib.auth",
+    "django.contrib.contenttypes",
+    "django.contrib.sessions",
+    "django.contrib.messages",
+    "django.contrib.staticfiles",
+    "rest_framework",
+    "corsheaders",
+    "pipeline",
+]
+
+MIDDLEWARE = [
+    "django.middleware.security.SecurityMiddleware",
+    "corsheaders.middleware.CorsMiddleware",
+    "django.contrib.sessions.middleware.SessionMiddleware",
+    "django.middleware.common.CommonMiddleware",
+    "django.middleware.csrf.CsrfViewMiddleware",
+    "django.contrib.auth.middleware.AuthenticationMiddleware",
+    "django.contrib.messages.middleware.MessageMiddleware",
+    "django.middleware.clickjacking.XFrameOptionsMiddleware",
+]
+
+ROOT_URLCONF = "config.urls"
+
+TEMPLATES = [
+    {
+        "BACKEND": "django.template.backends.django.DjangoTemplates",
+        "DIRS": [],
+        "APP_DIRS": True,
+        "OPTIONS": {
+            "context_processors": [
+                "django.template.context_processors.request",
+                "django.contrib.auth.context_processors.auth",
+                "django.contrib.messages.context_processors.messages",
+            ],
+        },
+    },
+]
+
+WSGI_APPLICATION = "config.wsgi.application"
+
+_sqlite = os.environ.get("DJANGO_SQLITE_PATH", "").strip()
+DATABASES = {
+    "default": {
+        "ENGINE": "django.db.backends.sqlite3",
+        "NAME": Path(_sqlite).expanduser().resolve()
+        if _sqlite
+        else (BASE_DIR / "db.sqlite3"),
+    }
+}
+
+AUTH_PASSWORD_VALIDATORS = [
+    {"NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator"},
+    {"NAME": "django.contrib.auth.password_validation.MinimumLengthValidator"},
+    {"NAME": "django.contrib.auth.password_validation.CommonPasswordValidator"},
+    {"NAME": "django.contrib.auth.password_validation.NumericPasswordValidator"},
+]
+
+LANGUAGE_CODE = "zh-hans"
+TIME_ZONE = "Asia/Shanghai"
+USE_I18N = True
+USE_TZ = True
+
+STATIC_URL = "static/"
+DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
+
+REST_FRAMEWORK = {
+    "DEFAULT_RENDERER_CLASSES": [
+        "rest_framework.renderers.JSONRenderer",
+    ],
+}
+
+_cors = os.environ.get("CORS_ALLOWED_ORIGINS", "http://localhost:5173,http://127.0.0.1:5173")
+CORS_ALLOWED_ORIGINS = [x.strip() for x in _cors.split(",") if x.strip()]
+CORS_ALLOW_CREDENTIALS = True
+
+_csrf = os.environ.get("CSRF_TRUSTED_ORIGINS", "http://localhost:5173,http://127.0.0.1:5173")
+CSRF_TRUSTED_ORIGINS = [x.strip() for x in _csrf.split(",") if x.strip()]

backend/config/urls.py

@@ -0,0 +1,7 @@
+from django.contrib import admin
+from django.urls import include, path
+
+urlpatterns = [
+    path("admin/", admin.site.urls),
+    path("api/", include("pipeline.urls")),
+]

backend/config/wsgi.py

@@ -0,0 +1,16 @@
+"""
+WSGI config for config project.
+
+It exposes the WSGI callable as a module-level variable named ``application``.
+
+For more information on this file, see
+https://docs.djangoproject.com/en/5.2/howto/deployment/wsgi/
+"""
+
+import os
+
+from django.core.wsgi import get_wsgi_application
+
+os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
+
+application = get_wsgi_application()

backend/crawler_copy/jd_pc_search/AI_crawler.py

@@ -0,0 +1,804 @@
+# -*- coding: utf-8 -*-
+"""
+从本地图片路径或图片 URL 调用 OpenAI 兼容多模态接口，提取配料表等；并提供**纯文本** ``chat/completions`` 供报告/策略等场景复用。
+
+**密钥与网关仅通过环境变量配置**（勿写入代码）：
+
+- ``OPENAI_API_KEY``：API Key（必填）
+- ``OPENAI_BASE_URL``：网关根地址，如 ``https://llm.example.com/v1``（必填，勿尾斜杠多余路径）
+- ``OPENAI_VISION_MODEL``：视觉模型名（可选，默认 ``Qwen/Qwen3-Omni-30B-A3B``）
+
+**纯文本调用**（``chat_completion_text``）优先使用：
+
+- ``OPENAI_TEXT_MODEL`` 或 ``LLM_TEXT_MODEL``；未设置时依次回退到 ``OPENAI_VISION_MODEL``、``LLM_MODEL``、上述默认。
+
+兼容别名（二选一即可）：``LLM_API_KEY``、``LLM_BASE_URL``、``LLM_MODEL``。
+
+上述变量与 Django 共用 **一份** ``market_assistant/.env``（与本脚本所在 ``backend`` 的上三级目录下的 ``.env``；需 ``pip install python-dotenv``）。
+
+**运行方式**：在下方「运行配置」里改好 ``IMAGE_SOURCE`` 等变量后，直接执行 ``python AI_crawler.py``，无需命令行参数。
+"""
+
+from __future__ import annotations
+
+import base64
+import os
+import re
+import sys
+from pathlib import Path
+from typing import Any
+
+import requests
+
+_SCRIPT_DIR = Path(__file__).resolve().parent
+# backend/crawler_copy/jd_pc_search -> parents[3] == market_assistant
+_MA_ROOT = Path(__file__).resolve().parents[3]
+
+
+def _load_market_assistant_dotenv() -> None:
+    """先于 LOW_GI_PROJECT_ROOT 解析加载 ``market_assistant/.env``（唯一配置源）。"""
+    try:
+        from dotenv import load_dotenv
+    except ImportError:
+        return
+    p = _MA_ROOT / ".env"
+    if p.is_file():
+        load_dotenv(p)
+
+
+_load_market_assistant_dotenv()
+
+from _low_gi_root import low_gi_project_root  # noqa: E402
+
+_PROJECT_ROOT = low_gi_project_root()
+
+# ---------------------------------------------------------------------------
+# 运行配置（按需修改；启动时不要求命令行参数）
+# ---------------------------------------------------------------------------
+# 必填：本地图片路径，或 http(s) 图片链接（如京东主图 / 详情图）
+IMAGE_SOURCE = "https://img30.360buyimg.com/sku/jfs/t1/390444/8/13018/103574/6982e951Fc44d9d7b/00d62ee56189d75d.jpg.avif"
+# IMAGE_SOURCE = "https://img30.360buyimg.com/sku/jfs/t1/382894/31/7432/241977/694cf41aFa27be91e/00d63164ffeb8b46.jpg.avif"
+
+# 提示词：留空则使用 ``PROMPT_DEFAULT``
+USER_PROMPT = ""
+PROMPT_DEFAULT = (
+    "请识别图片中的配料表，只输出配料列表本身，不要将菜品做法、步骤、用料等认为是配料表，不要误识别为食谱；用中文逗号或顿号分隔，"
+    "输出为连续一段文字，不要使用多行换行（避免与食谱、做法步骤混淆）。"
+    "每种原料名称只出现一次，禁止重复罗列同一添加剂（如磷酸三钾、磷酸三钠等勿循环抄写多遍）；"
+    "若图为表格中多行同类添加剂，可概括为「食品添加剂（按国家标准使用）」或合并为一句，勿展开成数百字重复。"
+    "【禁止猜测】必须严格依据图中清晰可见的印刷文字归纳；不得根据商品品类、常识或模糊字迹推测、补全、编造任何原料。"
+    "若本图无配料表、仅有产品信息/广告、文字被裁切、过小、模糊到无法逐字确认，或你只能「猜」出部分内容，则禁止输出配料列表："
+    "请只输出且仅输出一句「无法识别图片中的配料表」（不要解释、不要道歉长文、不要列出疑似项）。"
+)
+
+# 拉取远程图时的 Referer（京东图床一般需类似商城域名）
+IMAGE_REFERER = "https://www.jd.com/"
+
+TEMPERATURE = 0.0
+MAX_TOKENS = 2048
+# 部分 Qwen 网关需要关闭 thinking
+QWEN_OMNI_TEMPLATE = False
+# ---------------------------------------------------------------------------
+
+DEFAULT_MODEL = "Qwen/Qwen3-Omni-30B-A3B"
+DEFAULT_USER_AGENT = (
+    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
+    "(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+)
+
+
+def _normalize_chat_content(content: Any) -> str:
+    """
+    兼容 OpenAI 兼容网关：``message.content`` 可能是 str，也可能是
+    ``[{type:text, text:...}, ...]``；避免对 list 误用 ``.strip()`` 或得到怪异字符串。
+    """
+    if content is None:
+        return ""
+    if isinstance(content, str):
+        return content.strip()
+    if isinstance(content, list):
+        parts: list[str] = []
+        for item in content:
+            if isinstance(item, dict):
+                if item.get("type") == "text":
+                    parts.append(str(item.get("text") or ""))
+                elif "text" in item:
+                    parts.append(str(item.get("text") or ""))
+            elif isinstance(item, str):
+                parts.append(item)
+        return "".join(parts).strip()
+    return str(content).strip()
+
+
+def normalize_ingredients_text_for_csv(text: str) -> str:
+    """
+    将配料 OCR 结果压成**单行**，便于 ``detail_ware_export.csv`` / 合并表展示。
+    模型常按「一行一项」输出食谱或列表，会产生多换行；合并为非换行文本，行间用中文分号分隔。
+    """
+    t = (text or "").replace("\r\n", "\n").replace("\r", "\n").strip()
+    if not t:
+        return ""
+    lines = [ln.strip() for ln in t.split("\n") if ln.strip()]
+    if len(lines) <= 1:
+        return lines[0] if lines else ""
+    return "；".join(lines)
+
+
+def _split_ingredient_segments(text: str) -> list[str]:
+    """按常见分隔符拆成原料小段（用于检测尾部循环复读）。"""
+    t = (text or "").strip()
+    if not t:
+        return []
+    return [p.strip() for p in re.split(r"[；、,，]+", t) if p.strip()]
+
+
+def sanitize_vision_ingredients_output(text: str) -> str:
+    """
+    清洗多模态配料识别结果：去掉尾部引号、切除「仅两三种词循环数百次」的模型复读尾巴、超长截断。
+
+    典型故障：真实配料后无限重复「磷酸三钾、磷酸三钠…」，仍因前半段通过业务校验。
+    """
+    t = (text or "").strip()
+    _trail_q = frozenset({'"', "'", "\u201c", "\u201d", "\u2018", "\u2019", "\uff02"})
+    while t and t[-1] in _trail_q:
+        t = t[:-1].strip()
+
+    segs = _split_ingredient_segments(t)
+    if not segs:
+        return ""
+
+    min_spam_run = 28
+    cut_i = len(segs)
+    for i in range(0, max(0, len(segs) - min_spam_run + 1)):
+        suf = segs[i:]
+        if len(suf) >= min_spam_run and len(set(suf)) <= 3:
+            cut_i = i
+            break
+    segs = segs[:cut_i]
+    if not segs:
+        return ""
+
+    t = "、".join(segs)
+
+    # 字符级兜底：同一短词组高频重复（未按顿号切分时）
+    t = re.sub(
+        r"(磷酸三[钾钠][、,，]?\s*){35,}",
+        "磷酸三钾、磷酸三钠等（按国家标准使用）",
+        t,
+    )
+
+    max_chars = 3200
+    if len(t) > max_chars:
+        cut = t[:max_chars]
+        last = max(cut.rfind("、"), cut.rfind("，"), cut.rfind(","), cut.rfind("；"))
+        if last > max_chars // 2:
+            t = cut[: last + 1] + "…（已截断）"
+        else:
+            t = cut + "…（已截断）"
+    return t.strip()
+
+
+def _resolve_credentials(
+    api_key: str | None,
+    base_url: str | None,
+    model: str | None,
+) -> tuple[str, str, str]:
+    """凭证只从环境变量（及可选函数参数）读取，不在代码中写死。"""
+    key = (
+        (api_key or "").strip()
+        or (os.environ.get("OPENAI_API_KEY") or os.environ.get("LLM_API_KEY") or "").strip()
+    )
+    base = (
+        (base_url or "").strip().rstrip("/")
+        or (
+            os.environ.get("OPENAI_BASE_URL") or os.environ.get("LLM_BASE_URL") or ""
+        ).strip().rstrip("/")
+    )
+    m = (
+        (model or "").strip()
+        or (
+            os.environ.get("OPENAI_VISION_MODEL")
+            or os.environ.get("LLM_MODEL")
+            or DEFAULT_MODEL
+        ).strip()
+    )
+    if not key:
+        raise ValueError("请设置环境变量 OPENAI_API_KEY（或 LLM_API_KEY）")
+    if not base:
+        raise ValueError(
+            "请设置环境变量 OPENAI_BASE_URL（或 LLM_BASE_URL），例如 https://your-gateway.com/v1"
+        )
+    return key, base, m
+
+
+def resolve_text_model_name(model: str | None = None) -> str:
+    """
+    文本补全所用模型：显式 ``model`` 优先，否则读环境变量（见模块文档）。
+    """
+    m = (model or "").strip()
+    if m:
+        return m
+    for env in (
+        "OPENAI_TEXT_MODEL",
+        "LLM_TEXT_MODEL",
+        "OPENAI_VISION_MODEL",
+        "LLM_MODEL",
+    ):
+        v = (os.environ.get(env) or "").strip()
+        if v:
+            return v
+    return DEFAULT_MODEL
+
+
+def strip_outer_markdown_fence(text: str) -> str:
+    """若模型用 ``` / ```markdown 包裹全文，去掉最外层围栏。"""
+    t = (text or "").strip()
+    if not t.startswith("```"):
+        return t
+    lines = t.split("\n")
+    if lines and lines[0].strip().startswith("```"):
+        lines = lines[1:]
+    while lines and lines[-1].strip() == "```":
+        lines = lines[:-1]
+    return "\n".join(lines).strip()
+
+
+def chat_completion_text(
+    *,
+    system_prompt: str,
+    user_prompt: str,
+    api_key: str | None = None,
+    base_url: str | None = None,
+    model: str | None = None,
+    temperature: float = 0.2,
+    max_tokens: int = 8192,
+    timeout: int = 300,
+    extra_json: dict[str, Any] | None = None,
+) -> str:
+    """
+    OpenAI 兼容网关的**纯文本**多轮占位为 system + user 各一条，与 ``extract_ingredients_from_image`` 共用凭证与端点。
+    返回助手消息正文（已 ``strip`` / 兼容 list 型 content）。
+    """
+    k, b, _ = _resolve_credentials(api_key, base_url, None)
+    m = resolve_text_model_name(model)
+    body: dict[str, Any] = {
+        "model": m,
+        "messages": [
+            {"role": "system", "content": system_prompt},
+            {"role": "user", "content": user_prompt},
+        ],
+        "temperature": temperature,
+        "max_tokens": max_tokens,
+    }
+    if extra_json:
+        body.update(extra_json)
+    r = requests.post(
+        f"{b}/chat/completions",
+        headers={
+            "Authorization": f"Bearer {k}",
+            "Content-Type": "application/json",
+        },
+        json=body,
+        timeout=timeout,
+    )
+    r.raise_for_status()
+    data = r.json()
+    msg = (data.get("choices") or [{}])[0].get("message") or {}
+    return _normalize_chat_content(msg.get("content"))
+
+
+def _mime_for_path(path: str) -> str:
+    ext = path.lower().rsplit(".", 1)[-1]
+    return {
+        "jpg": "image/jpeg",
+        "jpeg": "image/jpeg",
+        "png": "image/png",
+        "webp": "image/webp",
+        "gif": "image/gif",
+        "avif": "image/avif",
+    }.get(ext, "image/jpeg")
+
+
+def _mime_from_response(url: str, content_type: str | None) -> str:
+    if content_type and content_type.lower().startswith("image/"):
+        return content_type.split(";")[0].strip().lower()
+    u = url.lower().split("?")[0]
+    for suf, mime in (
+        (".png", "image/png"),
+        (".webp", "image/webp"),
+        (".avif", "image/avif"),
+        (".gif", "image/gif"),
+        (".jpg", "image/jpeg"),
+        (".jpeg", "image/jpeg"),
+    ):
+        if u.endswith(suf):
+            return mime
+    return "image/jpeg"
+
+
+def image_to_data_url(
+    source: str,
+    *,
+    referer: str = "https://www.jd.com/",
+    timeout: int = 60,
+) -> tuple[str, str]:
+    """
+    ``source`` 为本地路径或以 http(s) 开头的 URL。
+    返回 (data_url, 来源说明)。
+    """
+    s = source.strip()
+    if s.lower().startswith(("http://", "https://")):
+        headers = {
+            "User-Agent": DEFAULT_USER_AGENT,
+            "Accept": "image/avif,image/webp,image/*,*/*;q=0.8",
+            "Referer": referer,
+        }
+        r = requests.get(s, headers=headers, timeout=timeout, allow_redirects=True)
+        r.raise_for_status()
+        mime = _mime_from_response(s, r.headers.get("Content-Type"))
+        b64 = base64.standard_b64encode(r.content).decode("ascii")
+        return f"data:{mime};base64,{b64}", f"url:{s[:80]}"
+
+    with open(s, "rb") as f:
+        raw = f.read()
+    mime = _mime_for_path(s)
+    b64 = base64.standard_b64encode(raw).decode("ascii")
+    return f"data:{mime};base64,{b64}", f"file:{s}"
+
+
+def extract_ingredients_from_image(
+    image_path_or_url: str,
+    *,
+    api_key: str | None = None,
+    base_url: str | None = None,
+    model: str | None = None,
+    user_prompt: str | None = None,
+    temperature: float = 0.0,
+    max_tokens: int = 2048,
+    referer: str = "https://www.jd.com/",
+    extra_json: dict[str, Any] | None = None,
+    prompt_default: str | None = None,
+) -> str:
+    """
+    从本地图片路径或图片 URL 识别配料表（可改 ``user_prompt`` 扩展为营养成分表等）。
+    未传入 ``api_key`` / ``base_url`` / ``model`` 时从环境变量读取。
+    返回值为经 ``normalize_ingredients_text_for_csv`` 处理后的**单行**文本，便于写入 CSV。
+    """
+    k, b, m = _resolve_credentials(api_key, base_url, model)
+
+    data_url, _src = image_to_data_url(image_path_or_url, referer=referer)
+
+    _fallback = (
+        prompt_default
+        or "请识别图片中的配料表，只输出配料列表，不要误识别为做法用料；用逗号或顿号分隔为一段，不要换行分段。"
+    )
+    prompt = user_prompt if user_prompt is not None and str(user_prompt).strip() else _fallback
+
+    body: dict[str, Any] = {
+        "model": m,
+        "messages": [
+            {
+                "role": "user",
+                "content": [
+                    {"type": "image_url", "image_url": {"url": data_url}},
+                    {"type": "text", "text": prompt},
+                ],
+            }
+        ],
+        "temperature": temperature,
+        "max_tokens": max_tokens,
+    }
+    if extra_json:
+        body.update(extra_json)
+
+    r = requests.post(
+        f"{b}/chat/completions",
+        headers={
+            "Authorization": f"Bearer {k}",
+            "Content-Type": "application/json",
+        },
+        json=body,
+        timeout=120,
+    )
+    r.raise_for_status()
+    data = r.json()
+    msg = (data.get("choices") or [{}])[0].get("message") or {}
+    raw = normalize_ingredients_text_for_csv(_normalize_chat_content(msg.get("content")))
+    return sanitize_vision_ingredients_output(raw)
+
+
+def parse_joined_image_urls(joined: str) -> list[str]:
+    """
+    解析详情 DOM 拼出的 URL 串（与列 ``detail_body_ingredients`` 在「仅 URL」阶段同形：分号、换行分隔的 http(s) 链接）。
+    保持从前到后的顺序；去重不在这里做（上游已去重）。
+    """
+    t = (joined or "").strip()
+    if not t:
+        return []
+    t = t.replace("\r\n", "\n").replace("\r", "\n")
+    parts = re.split(r"\s*;\s*|\s*\n\s*", t)
+    out: list[str] = []
+    for p in parts:
+        u = p.strip()
+        if u.startswith(("http://", "https://")):
+            out.append(u)
+    return out
+
+
+def _looks_like_recipe_or_dish_prep(text: str) -> bool:
+    """
+    判断模型输出是否更像**菜谱/做法备料**（详情图里常见），而非包装「配料表」。
+    命中则不应写入 ``detail_body_ingredients``，继续尝试其它长图。
+    """
+    t = (text or "").strip()
+    if not t:
+        return False
+
+    recipe_kw = (
+        "做法",
+        "制作步骤",
+        "烹饪步骤",
+        "第一步",
+        "第二步",
+        "第三步",
+        "教程",
+        "准备食材",
+        "食材准备",
+        "下锅",
+        "翻炒",
+        "煮熟",
+        "大火烧开",
+        "转小火",
+        "装盘",
+        "小贴士",
+        "腌制",
+        "爆香",
+        "焯水",
+        "切丝",
+        "切丁",
+        "切片",
+        "打匀",
+        "搅拌均匀",
+        "油热",
+        "调味",
+    )
+    if any(k in t for k in recipe_kw):
+        return True
+
+    # 「葱花蒜末 各1勺」类菜谱用量
+    if re.search(r"各[一二两三四五六七八九十\d零]+勺", t):
+        return True
+
+    # 多条「短名称 + 数量 + 料理常用单位」并列（典型备料清单）
+    dish_qty = re.findall(
+        r"[^\n；。,，、]{1,14}\s+\d+(?:\.\d+)?\s*[个只根块片勺条袋包杯碗适量克gG毫升mlML]{1,4}",
+        t,
+    )
+    if len(dish_qty) >= 2:
+        return True
+
+    # 半块/半根等家常分量词 + 生鲜食材名（包装配料表极少这样写）
+    if "半块" in t and re.search(r"鸡胸|鸡腿|牛肉|猪肉|黄瓜|番茄|土豆|豆腐", t):
+        return True
+    if "半根" in t and re.search(r"黄瓜|胡萝卜|玉米|香肠|葱", t):
+        return True
+
+    # 规范化后的「A；B；C；…」若多段都很短且多段含数字，多为做法用料枚举
+    parts = [p.strip() for p in t.split("；") if p.strip()]
+    if len(parts) >= 4:
+        short_with_digit = [p for p in parts if len(p) <= 24 and re.search(r"\d", p)]
+        if len(short_with_digit) >= 4:
+            return True
+
+    # 多行/多段里至少 3 条「短句 + 数字 + 个根块勺克」
+    lines = [ln.strip() for ln in re.split(r"[\n；]", t) if ln.strip()]
+    if len(lines) >= 3:
+        n_short_qty = sum(
+            1
+            for ln in lines
+            if len(ln) <= 22
+            and re.search(r"\d", ln)
+            and re.search(r"[个只根块片勺克gG]", ln)
+        )
+        if n_short_qty >= 3:
+            return True
+
+    return False
+
+
+def _looks_like_packaged_ingredient_enumeration(text: str) -> bool:
+    """
+    视觉模型常把包装图上的「配料表」整段压成**逗号/顿号分隔的原料枚举**，丢掉标题与含量行。
+    此类文本与菜谱备料（鸡胸、黄瓜、葱花等）可区分时，视为有效配料信号。
+    """
+    t = (text or "").strip()
+    if not t:
+        return False
+
+    parts = [p.strip() for p in re.split(r"[,，、;；]", t) if p.strip()]
+    if len(parts) < 3:
+        return False
+
+    # 多段像「家常备料」则不走此路（避免鸡胸、鸡蛋、黄瓜…误过）
+    recipe_seg = re.compile(
+        r"鸡胸|鸡腿|牛腩|牛肉|五花肉|里脊|鸡蛋|鸭蛋|皮蛋|黄瓜|番茄|西红柿|土豆|马铃薯|"
+        r"葱花|蒜末|姜丝|小米椒|青椒|洋葱|胡萝卜|生菜|菠菜|白菜|芹菜|香菜|小葱|"
+        r"面条$|挂面|粉条|粉丝"
+    )
+    n_recipe_like = sum(1 for p in parts if recipe_seg.search(p))
+    if n_recipe_like >= 2:
+        return False
+
+    # 工业化配料常见子串（粉体、纤维、添加剂类别、粮谷原料等）
+    industrial = re.compile(
+        r"食用|食品添加|麦麸|纤维|淀粉|魔芋|提取物|谷朊|谷胱|麸皮|糖浆|山梨|麦芽|柠檬酸|碳酸|"
+        r"酵母|乳粉|全脂|脱脂|果胶|黄原|卡拉胶|海藻酸|小麦|面粉|荞麦|燕麦|藜麦|青稞|糙米|黑米|"
+        r"棕榈|植物油|精炼油|氢化|起酥|可可脂"
+    )
+    n_industrial = sum(1 for p in parts if industrial.search(p))
+    if len(parts) >= 4 and n_industrial >= 2:
+        return True
+    if len(parts) >= 3 and n_industrial >= 3:
+        return True
+    return False
+
+
+def _has_packaged_ingredient_table_signals(text: str) -> bool:
+    """
+    正向判断：是否像**包装配料表**——标题+含量、行内含量、或（多段工业化原料枚举）。
+
+    仅 OCR 出一段家常食材名、无上述结构时，返回 False。
+    """
+    t = (text or "").strip()
+    if not t:
+        return False
+
+    # 行内「××（含量≥50%）」等，常见于包装，不强制出现「配料表」标题
+    if re.search(
+        r"[\u4e00-\u9fff\w·．\d]{1,18}[（(]\s*含量\s*[≥>=＝]?\s*[\d.]+\s*%?\s*[)）]",
+        t,
+    ):
+        return True
+
+    label = bool(
+        re.search(r"配料表", t)
+        or re.search(r"配\s*料\s*[:：]", t)
+        or re.search(r"原\s*料\s*[:：]", t)
+        or re.search(r"食品添加剂", t)
+        or re.search(r"产品\s*配\s*料", t)
+    )
+
+    # 「含量」相关信息：百分比、不等式、法规用语、添加量表述等（不含单独「50克」类菜谱用量）
+    content = bool(
+        re.search(r"含量", t)
+        or re.search(r"添加量", t)
+        or re.search(r"\d+(?:\.\d+)?\s*[％%]", t)
+        or re.search(r"[≥＞>]\s*[\d.]+", t)
+        or re.search(r"按\s*添\s*加\s*量\s*递\s*减", t)
+    )
+
+    if label and content:
+        return True
+
+    # 模型只输出「原料1,原料2,…」时仍可能是正规配料表
+    if _looks_like_packaged_ingredient_enumeration(t):
+        return True
+
+    return False
+
+
+def _ingredient_extraction_acceptable(text: str) -> bool:
+    """粗判模型输出是否像有效配料信息（过滤拒识句、伪列表、过短碎片、菜谱备料）。
+
+    通过条件之一：配料表标题+含量类信号；行内「××（含量≥x%）」；或多段工业化原料枚举（见
+    ``_looks_like_packaged_ingredient_enumeration``，用于模型只输出逗号分隔原料、丢掉标题时）。
+    """
+    t = (text or "").strip()
+    if len(t) < 6:
+        return False
+    # 模型偶发输出类似 Python 列表的字符串，或 JSON 数组形态
+    if re.match(r"^\s*\[.*\]\s*$", t):
+        return False
+    refuse = (
+        "无法识别",
+        "没有配料",
+        "看不清",
+        "不存在配料",
+        "未在图中",
+        "未在图片",
+        "抱歉，我",
+        "抱歉，无法",
+        "不能识别",
+        "没有识别到",
+        "图中没有",
+        "图片中没有",
+        "无配料",
+        "未见配料",
+    )
+    if any(x in t for x in refuse):
+        return False
+    # 真配料表通常含分隔符或足够长；避免「无」「暂无」等被当成命中
+    if t in ("无", "暂无", "没有", "无。", "无，"):
+        return False
+    if _looks_like_recipe_or_dish_prep(t):
+        return False
+    if not _has_packaged_ingredient_table_signals(t):
+        return False
+    tail = _split_ingredient_segments(t)
+    if len(tail) >= 32:
+        if len(set(tail[-32:])) <= 3:
+            return False
+    sep_chars = "，、,;；"
+    if len(t) < 18 and not any(c in t for c in sep_chars):
+        return False
+    return True
+
+
+REASON_NO_BODY_URLS = "【未识别到配料】未解析到任何详情长图 URL。"
+REASON_NO_VISION_API = (
+    "【未识别到配料】未配置多模态 API（需环境变量 OPENAI_API_KEY + OPENAI_BASE_URL，"
+    "或 LLM_API_KEY + LLM_BASE_URL）。"
+)
+
+
+def extract_ingredients_from_body_image_urls_reversed_with_source(
+    urls_joined: str,
+    *,
+    referer: str | None = None,
+    user_prompt: str | None = None,
+    prompt_default: str | None = None,
+    temperature: float | None = None,
+    max_tokens: int | None = None,
+    extra_json: dict[str, Any] | None = None,
+) -> tuple[str, str | None]:
+    """
+    与 ``extract_ingredients_from_body_image_urls_reversed`` 相同逻辑；额外返回命中配料时所用的**图片 URL**
+    （自后向前首次通过校验的那张）。未命中或失败时第二项为 ``None``。
+    """
+    urls = parse_joined_image_urls(urls_joined)
+    if not urls:
+        return REASON_NO_BODY_URLS, None
+    try:
+        _resolve_credentials(None, None, None)
+    except ValueError:
+        return REASON_NO_VISION_API, None
+
+    ref = (referer if referer is not None else IMAGE_REFERER) or "https://www.jd.com/"
+    temp = float(temperature) if temperature is not None else float(TEMPERATURE)
+    mt = int(max_tokens) if max_tokens is not None else int(MAX_TOKENS)
+    extra = extra_json
+    if extra is None and QWEN_OMNI_TEMPLATE:
+        extra = {"chat_template_kwargs": {"enable_thinking": False}}
+
+    pu = user_prompt if user_prompt is not None else ((USER_PROMPT or "").strip() or None)
+    pd = prompt_default if prompt_default is not None else PROMPT_DEFAULT
+
+    n = len(urls)
+    n_err = 0
+    n_rejected = 0
+    for url in reversed(urls):
+        try:
+            text = extract_ingredients_from_image(
+                url,
+                user_prompt=pu,
+                referer=ref.strip(),
+                temperature=temp,
+                max_tokens=mt,
+                extra_json=extra,
+                prompt_default=pd,
+            )
+        except Exception:
+            n_err += 1
+            continue
+        t = (text or "").strip()
+        if _ingredient_extraction_acceptable(t):
+            return t, url
+        if t:
+            n_rejected += 1
+
+    parts = [
+        f"【未识别到配料】已对 {n} 张详情长图自后向前依次尝试（命中即停），未得到有效配料表。"
+    ]
+    if n_err:
+        parts.append(f" 请求异常 {n_err} 次。")
+    if n_rejected:
+        parts.append(f" 有 {n_rejected} 次返回未通过配料校验。")
+    if not n_err and not n_rejected:
+        parts.append(" 模型返回均为空或过短。")
+    return "".join(parts), None
+
+
+def extract_ingredients_from_body_image_urls_reversed(
+    urls_joined: str,
+    *,
+    referer: str | None = None,
+    user_prompt: str | None = None,
+    prompt_default: str | None = None,
+    temperature: float | None = None,
+    max_tokens: int | None = None,
+    extra_json: dict[str, Any] | None = None,
+) -> str:
+    """
+    对 URL 串拆出的链接 **从后往前**依次调用视觉模型：**首次**通过校验的配料文本立即返回（省时间）。
+
+    若始终无命中：返回以 ``【未识别到配料】`` 开头的原因说明（**不再返回空串**）。
+    未配置 API 时返回 ``REASON_NO_VISION_API``。
+
+    命中条件（见 ``_ingredient_extraction_acceptable``）：须像**包装配料表**——「配料/含量」标题结构、
+    ``××（含量≥x%）``、或多段工业化原料逗号/顿号枚举（模型常省略标题）；纯家常备料（鸡胸、黄瓜、葱花等）
+    仍丢弃并试下一张图。
+
+    若需同时得到所用图片 URL，请用 ``extract_ingredients_from_body_image_urls_reversed_with_source``。
+    """
+    text, _ = extract_ingredients_from_body_image_urls_reversed_with_source(
+        urls_joined,
+        referer=referer,
+        user_prompt=user_prompt,
+        prompt_default=prompt_default,
+        temperature=temperature,
+        max_tokens=max_tokens,
+        extra_json=extra_json,
+    )
+    return text
+
+
+def main() -> None:
+    try:
+        if hasattr(sys.stdout, "reconfigure"):
+            sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+        if hasattr(sys.stderr, "reconfigure"):
+            sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+    except Exception:
+        pass
+
+    src = (IMAGE_SOURCE or "").strip()
+    if not src:
+        print(
+            "[AI_crawler] 请在文件顶部设置 IMAGE_SOURCE（图片路径或 URL）后重试。",
+            file=sys.stderr,
+        )
+        sys.exit(2)
+
+    prompt_use = (USER_PROMPT or "").strip() or None
+    extra = None
+    if QWEN_OMNI_TEMPLATE:
+        extra = {"chat_template_kwargs": {"enable_thinking": False}}
+
+    try:
+        text = extract_ingredients_from_image(
+            src,
+            user_prompt=prompt_use,
+            referer=(IMAGE_REFERER or "https://www.jd.com/").strip(),
+            temperature=float(TEMPERATURE),
+            max_tokens=int(MAX_TOKENS),
+            extra_json=extra,
+            prompt_default=PROMPT_DEFAULT,
+        )
+    except ValueError as e:
+        print(f"[AI_crawler] {e}", file=sys.stderr)
+        sys.exit(2)
+    except requests.HTTPError as e:
+        err_body = ""
+        if e.response is not None and e.response.text:
+            err_body = e.response.text[:1500]
+        print(f"[AI_crawler] HTTP 错误: {e}\n{err_body}", file=sys.stderr)
+        sys.exit(1)
+    except Exception as e:
+        print(f"[AI_crawler] 失败: {e}", file=sys.stderr)
+        sys.exit(1)
+
+    t = (text or "").strip()
+    if _ingredient_extraction_acceptable(t):
+        print(t)
+    else:
+        print(
+            "【未通过配料表校验】输出须同时包含包装配料表常见结构（如「配料/配料表/原料/食品添加剂」）"
+            "与含量或百分比等信息，或为「××（含量≥x%）」形态；纯食材/菜谱备料枚举不会采纳。"
+            "与 extract_ingredients_from_body_image_urls_reversed 流水线规则一致。"
+        )
+        if t:
+            print(f"[AI_crawler] 模型原始输出（未采纳）: {t}", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

backend/crawler_copy/jd_pc_search/_low_gi_root.py

@@ -0,0 +1,27 @@
+# -*- coding: utf-8 -*-
+"""
+数据工作区根目录：流水线与爬虫副本在其下读写 ``data/JD/`` 等。
+
+- 若已设置 ``LOW_GI_PROJECT_ROOT``（如 Django settings 或 ``market_assistant/.env``），使用该路径。
+- 未设置时默认为 **本仓库根**（``market_assistant``），便于独立克隆后无需再指向上级目录。
+"""
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+
+def _market_assistant_root() -> Path:
+    """本文件位于 backend/crawler_copy/jd_pc_search/_low_gi_root.py → 上溯 3 级为 MA 根。"""
+    return Path(__file__).resolve().parents[3]
+
+
+def low_gi_project_root() -> Path:
+    raw = (os.environ.get("LOW_GI_PROJECT_ROOT") or "").strip().strip('"').strip("'")
+    if raw:
+        p = Path(raw).expanduser().resolve()
+    else:
+        p = _market_assistant_root().resolve()
+    if not p.is_dir():
+        raise RuntimeError(f"LOW_GI_PROJECT_ROOT 不是有效目录: {p}")
+    return p

backend/crawler_copy/jd_pc_search/comment/jd_export_item_comment_page_request.js

@@ -0,0 +1,99 @@
+/**
+ * stdout 一行 JSON：{ url, method, form, headers } — POST client.action（与 item.jd.com 抓包一致）。
+ *
+ *   node jd_export_item_comment_page_request.js --sku 10145684793035 --category 36574;44419;44439 \\
+ *     --first-guid T6NdP8N2gZgtdRyCXKCcwHaB --page-num 1 --is-first true
+ */
+const path = require("path");
+const { loadJdSearchAuth } = require("../common/jd_search_common.js");
+const {
+  get_h5st_item_comment_page,
+  build_item_comment_page_client_action_form,
+} = require("./jd_h5st_item_comment_page.js");
+const { buildJdPcItemCommentHeaders } = require("./jd_pc_item_comment_headers.js");
+
+const DEFAULT_COOKIE = path.join(__dirname, "..", "common", "jd_cookie.txt");
+
+function parseCli(argv = process.argv.slice(2)) {
+  const out = {
+    sku: null,
+    category: null,
+    firstGuid: null,
+    pageNum: "1",
+    isFirst: true,
+    shopType: "0",
+    spuId: null,
+    style: "1",
+    functionId:
+      process.env.JD_COMMENT_LIST_FUNCTION_ID || "getCommentListPage",
+    cookiePath: DEFAULT_COOKIE,
+  };
+  const a = argv;
+  for (let i = 0; i < a.length; i++) {
+    const x = a[i];
+    const take = (key) => {
+      if (a[i + 1]) out[key] = a[++i];
+    };
+    if (x === "--sku") take("sku");
+    else if (x.startsWith("--sku=")) out.sku = x.slice(6);
+    else if (x === "--category") take("category");
+    else if (x.startsWith("--category=")) out.category = x.slice(11);
+    else if (x === "--first-guid") take("firstGuid");
+    else if (x.startsWith("--first-guid=")) out.firstGuid = x.slice(13);
+    else if (x === "--page-num") take("pageNum");
+    else if (x.startsWith("--page-num=")) out.pageNum = x.slice(11);
+    else if (x === "--is-first") take("isFirst");
+    else if (x.startsWith("--is-first=")) {
+      const v = x.slice(11).toLowerCase();
+      out.isFirst = v === "true" || v === "1" || v === "yes";
+    } else if (x === "--shop-type") take("shopType");
+    else if (x.startsWith("--shop-type=")) out.shopType = x.slice(12);
+    else if (x === "--spu-id") take("spuId");
+    else if (x.startsWith("--spu-id=")) out.spuId = x.slice(9);
+    else if (x === "--style") take("style");
+    else if (x.startsWith("--style=")) out.style = x.slice(8);
+    else if (x === "--function-id") take("functionId");
+    else if (x.startsWith("--function-id=")) out.functionId = x.slice(14);
+    else if (x === "--cookie-file") take("cookiePath");
+    else if (x.startsWith("--cookie-file=")) out.cookiePath = x.slice(14);
+  }
+  if (typeof out.isFirst === "string") {
+    const v = String(out.isFirst).toLowerCase();
+    out.isFirst = v === "true" || v === "1" || v === "yes";
+  }
+  return out;
+}
+
+try {
+  const cli = parseCli();
+  if (!cli.sku) throw new Error("需要 --sku");
+  if (!cli.category) throw new Error("需要 --category（如 36574;44419;44439）");
+  if (!cli.firstGuid) throw new Error("需要 --first-guid（首条评价 guid）");
+  if (!cli.functionId) throw new Error("需要 --function-id 或环境变量 JD_COMMENT_LIST_FUNCTION_ID");
+
+  const { cookie, uuid } = loadJdSearchAuth(cli.cookiePath);
+  if (!cookie) throw new Error("Cookie 为空或不存在");
+  if (!uuid) throw new Error("缺少 uuid（Cookie 中 __jdu / mba_muid）");
+
+  const pack = get_h5st_item_comment_page({
+    sku: cli.sku,
+    category: cli.category,
+    firstCommentGuid: cli.firstGuid,
+    pageNum: cli.pageNum,
+    isFirstRequest: cli.isFirst,
+    shopType: cli.shopType,
+    spuId: cli.spuId || undefined,
+    style: cli.style,
+    functionId: cli.functionId,
+  });
+  const { url, form } = build_item_comment_page_client_action_form(pack, {
+    uuid,
+  });
+  const headers = buildJdPcItemCommentHeaders({ cookie, sku: cli.sku });
+  process.stdout.write(
+    JSON.stringify({ url, method: "POST", form, headers })
+  );
+} catch (e) {
+  console.error(e.message || String(e));
+  process.exit(1);
+}

backend/crawler_copy/jd_pc_search/comment/jd_export_item_comment_request.js

@@ -0,0 +1,100 @@
+/**
+ * stdout 输出一行 JSON：{ url, headers }，供 jd_h5_item_comment_requests.py 使用。
+ *
+ *   cd crawler/jd_pc_search/comment && node jd_export_item_comment_request.js --sku 10145684793035
+ *   node jd_export_item_comment_request.js --sku 10145684793035 --comment-num 5 --shop-type 0
+ */
+const path = require("path");
+const { loadJdSearchAuth } = require("../common/jd_search_common.js");
+const {
+  get_h5st_item_lego_detail_comment,
+  build_item_lego_comment_api_url,
+} = require("./jd_h5st_item_comment.js");
+const { buildJdPcItemCommentHeaders } = require("./jd_pc_item_comment_headers.js");
+
+const DEFAULT_COOKIE = path.join(__dirname, "..", "common", "jd_cookie.txt");
+
+function parseItemCommentCliArgs(argv = process.argv.slice(2)) {
+  const out = {
+    sku: null,
+    commentNum: 5,
+    shopType: "0",
+    source: "pc",
+    cookiePath: DEFAULT_COOKIE,
+  };
+  const a = argv;
+  for (let i = 0; i < a.length; i++) {
+    const x = a[i];
+    if (x === "--sku" && a[i + 1]) {
+      out.sku = String(a[++i]).trim();
+      continue;
+    }
+    if (x.startsWith("--sku=")) {
+      out.sku = x.slice(6).trim();
+      continue;
+    }
+    if (x === "--comment-num" && a[i + 1]) {
+      out.commentNum = Math.max(1, parseInt(a[++i], 10) || 5);
+      continue;
+    }
+    if (x.startsWith("--comment-num=")) {
+      out.commentNum = Math.max(1, parseInt(x.slice(14), 10) || 5);
+      continue;
+    }
+    if (x === "--shop-type" && a[i + 1]) {
+      out.shopType = String(a[++i]);
+      continue;
+    }
+    if (x.startsWith("--shop-type=")) {
+      out.shopType = x.slice(12);
+      continue;
+    }
+    if (x === "--source" && a[i + 1]) {
+      out.source = String(a[++i]);
+      continue;
+    }
+    if (x.startsWith("--source=")) {
+      out.source = x.slice(9);
+      continue;
+    }
+    if (x === "--cookie-file" && a[i + 1]) {
+      out.cookiePath = String(a[++i]);
+      continue;
+    }
+    if (x.startsWith("--cookie-file=")) {
+      out.cookiePath = x.slice(14);
+      continue;
+    }
+  }
+  return out;
+}
+
+try {
+  const cli = parseItemCommentCliArgs();
+  if (!cli.sku) throw new Error("需要 --sku（商品 SKU，与 item.jd.com/{sku}.html 一致）");
+
+  const { cookie, uuid, xApiEidToken } = loadJdSearchAuth(cli.cookiePath);
+  if (!cookie) throw new Error("Cookie 为空或不存在（jd_cookie.txt 或 --cookie-file）");
+  if (!uuid || !xApiEidToken)
+    throw new Error("缺少 uuid 或 x-api-eid-token（Cookie 中 __jdu/mba_muid 与 3AB9D23F7A4B3CSS）");
+
+  const pack = get_h5st_item_lego_detail_comment({
+    sku: cli.sku,
+    commentNum: cli.commentNum,
+    shopType: cli.shopType,
+    source: cli.source,
+  });
+  const url = build_item_lego_comment_api_url(pack, {
+    uuid,
+    xApiEidToken,
+    bodyMode: "json",
+  });
+  const headers = buildJdPcItemCommentHeaders({
+    cookie,
+    sku: cli.sku,
+  });
+  process.stdout.write(JSON.stringify({ url, headers }));
+} catch (e) {
+  console.error(e.message || String(e));
+  process.exit(1);
+}

backend/crawler_copy/jd_pc_search/comment/jd_h5_item_comment_requests.py

@@ -0,0 +1,666 @@
+# -*- coding: utf-8 -*-
+"""
+京东商品详情页评论（getLegoWareDetailComment）。
+
+由同目录 Node ``jd_export_item_comment_request.js`` 生成 **url + headers**
+（含 ParamsSign / h5st，见 ``jd_h5st_item_comment.js``），再用 **Playwright/Chromium**
+发 GET（与 ``search/jd_search_playwright.py`` 相同方式，浏览器 TLS，减轻 jfe 403）。
+
+依赖: pip install playwright && playwright install chromium
+
+鉴权 Cookie：由 Node 读入请求头；路径见下方配置项 ``COOKIE_FILE`` / ``COOKIE_OVERRIDE``。
+
+
+用法（本仓库默认）: 修改下方「运行配置」后 ``python jd_h5_item_comment_requests.py``（无命令行参数）。
+"""
+
+from __future__ import annotations
+
+import csv
+import json
+import random
+import subprocess
+import sys
+import time
+from io import StringIO
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Any
+
+from playwright.sync_api import sync_playwright
+
+_JD_PKG_ROOT = Path(__file__).resolve().parent.parent
+if str(_JD_PKG_ROOT) not in sys.path:
+    sys.path.insert(0, str(_JD_PKG_ROOT))
+from common.jd_delay_utils import parse_request_delay_range
+from _low_gi_root import low_gi_project_root  # noqa: E402
+
+_JD_COMMENT_DIR = Path(__file__).resolve().parent
+
+# ---------------------------------------------------------------------------
+# 运行配置（按需改这里）
+# ---------------------------------------------------------------------------
+# 路径：副本通过 LOW_GI_PROJECT_ROOT 指向「Low GI」根目录
+_PROJECT_ROOT = low_gi_project_root()
+_PROJECT_DATA = _PROJECT_ROOT / "data" / "JD"
+_JD_COMMON_COOKIE = Path(__file__).resolve().parents[1] / "common" / "jd_cookie.txt"
+
+# SKU：单个商品 ID；与 SKU_FILE 二选一
+SKU = "10166848058665"
+# SKU_FILE：每行一个 SKU（# 注释）；与 SKU 二选一
+SKU_FILE = ""
+# COMMENT_NUM：首屏 Lego 请求 body.commentNum（评价条数相关）
+COMMENT_NUM = 5
+# SHOP_TYPE：body.shopType，一般 "0"
+SHOP_TYPE = "0"
+# COOKIE_FILE：传给 Node 读 Cookie 的文件路径（与搜索/详情共用 jd_cookie.txt）
+COOKIE_FILE = str(_JD_COMMON_COOKIE)
+# COOKIE_OVERRIDE：非空则覆盖请求头中的 Cookie
+COOKIE_OVERRIDE = ""
+# TIMEOUT_SEC：单次 Playwright GET/POST 超时（秒）
+TIMEOUT_SEC = 30.0
+# REQUEST_DELAY：每次发起新 HTTP 前的随机等待，如 "30-60"；"0-0" 可关闭
+REQUEST_DELAY = "30-60"
+# OUT_JSONL：网络采集时每条接口一行 JSONL；空则打印 stdout
+OUT_JSONL = str(_PROJECT_DATA / "jd_comments.jsonl")
+# PRETTY：单 SKU 且无 OUT_JSONL 时是否缩进打印业务 JSON（parsed）
+PRETTY = False
+# RAISE_HTTP：True 时 HTTP 非 2xx 直接退出进程
+RAISE_HTTP = False
+# HEADED：True 有头浏览器
+HEADED = False
+# COMMENTS_OUT：解析后的扁平评价，扩展名 .csv 或 .jsonl；可与采集同时写
+COMMENTS_OUT = str(_PROJECT_DATA / "jd_comments_flat.csv")
+# FROM_JSONL：非空则离线模式，仅从已存 JSONL 抽评价；须同时设 COMMENTS_OUT，不走浏览器
+FROM_JSONL = ""
+# WITH_COMMENT_LIST：首屏 Lego 成功后是否继续请求分页评价列表（POST client.action）
+WITH_COMMENT_LIST = False
+# LIST_PAGES：列表分页规格，如 "1"、"1-5"、"1,3,5"
+LIST_PAGES = "1"
+# LIST_FUNCTION_ID：列表接口 functionId，须与抓包一致
+LIST_FUNCTION_ID = "getCommentListPage"
+# LIST_STYLE：非首包分页请求的 style
+LIST_STYLE = "1"
+# LIST_CATEGORY：可选，手动 body.category（默认从首条评价 maidianInfo 解析）
+LIST_CATEGORY = ""
+# LIST_FIRST_GUID：可选，手动 firstCommentGuid（默认首屏 commentInfoList[0].guid）
+LIST_FIRST_GUID = ""
+# ---------------------------------------------------------------------------
+
+
+def export_item_comment_request_json(
+    sku: str,
+    *,
+    comment_num: int = 5,
+    shop_type: str = "0",
+    cookie_file: str | None = None,
+) -> dict[str, Any]:
+    """Node 输出 {url, headers}；h5st 与 body 与商品页抓包一致。"""
+    cmd = [
+        "node",
+        str(_JD_COMMENT_DIR / "jd_export_item_comment_request.js"),
+        "--sku",
+        str(sku).strip(),
+        "--comment-num",
+        str(max(1, int(comment_num))),
+        "--shop-type",
+        str(shop_type),
+    ]
+    cf = (cookie_file or "").strip()
+    if cf:
+        cmd.extend(["--cookie-file", cf])
+    r = subprocess.run(
+        cmd,
+        cwd=str(_JD_COMMENT_DIR),
+        capture_output=True,
+        text=True,
+        encoding="utf-8",
+    )
+    if r.returncode != 0:
+        print(r.stderr or r.stdout, file=sys.stderr)
+        sys.exit(r.returncode or 1)
+    return json.loads(r.stdout)
+
+
+def export_item_comment_page_request_json(
+    sku: str,
+    *,
+    category: str,
+    first_guid: str,
+    page_num: str,
+    is_first: bool,
+    function_id: str,
+    shop_type: str = "0",
+    spu_id: str | None = None,
+    style: str = "1",
+    cookie_file: str | None = None,
+) -> dict[str, Any]:
+    """Node 输出 POST client.action：{ url, method, form, headers }。"""
+    cmd = [
+        "node",
+        str(_JD_COMMENT_DIR / "jd_export_item_comment_page_request.js"),
+        "--sku",
+        str(sku).strip(),
+        "--category",
+        str(category).strip(),
+        "--first-guid",
+        str(first_guid).strip(),
+        "--page-num",
+        str(page_num).strip(),
+        "--is-first",
+        "true" if is_first else "false",
+        "--function-id",
+        str(function_id).strip(),
+        "--shop-type",
+        str(shop_type),
+        "--style",
+        str(style),
+    ]
+    if spu_id and str(spu_id).strip():
+        cmd.extend(["--spu-id", str(spu_id).strip()])
+    cf = (cookie_file or "").strip()
+    if cf:
+        cmd.extend(["--cookie-file", cf])
+    r = subprocess.run(
+        cmd,
+        cwd=str(_JD_COMMENT_DIR),
+        capture_output=True,
+        text=True,
+        encoding="utf-8",
+    )
+    if r.returncode != 0:
+        print(r.stderr or r.stdout, file=sys.stderr)
+        sys.exit(r.returncode or 1)
+    return json.loads(r.stdout)
+
+
+def _sleep_between_jd_requests(
+    delay_range: tuple[float, float], label: str = "请求间隔"
+) -> None:
+    """在「上一包已完成、即将发起下一包」时调用；0-0 视为不等待。"""
+    lo, hi = delay_range
+    if lo <= 0 and hi <= 0:
+        return
+    sec = random.uniform(lo, hi)
+    print(
+        f"[京东] {label} sleep {sec:.1f}s（区间 {lo:g}–{hi:g}）",
+        file=sys.stderr,
+    )
+    time.sleep(sec)
+
+
+def parse_list_pages_spec(spec: str) -> list[str]:
+    """
+    --list-pages：``1-5`` → 1..5；``1,3,5`` → 单页序列；单数字 ``2`` → [\"2\"]。
+    """
+    s = (spec or "").strip()
+    if not s:
+        return ["1"]
+    if "," in s:
+        return [p.strip() for p in s.split(",") if p.strip()]
+    if "-" in s:
+        parts = s.split("-", 1)
+        lo, hi = int(parts[0].strip()), int(parts[1].strip())
+        if lo > hi:
+            lo, hi = hi, lo
+        return [str(i) for i in range(lo, hi + 1)]
+    return [s]
+
+
+def category_and_first_guid_from_lego(parsed: Any) -> tuple[str, str]:
+    """从 getLegoWareDetailComment 的 commentInfoList[0] 取 category（maidianInfo 前缀）与 guid。"""
+    if not isinstance(parsed, dict):
+        return "", ""
+    lst = parsed.get("commentInfoList")
+    if not isinstance(lst, list) or not lst:
+        return "", ""
+    first = lst[0]
+    if not isinstance(first, dict):
+        return "", ""
+    guid = str(first.get("guid") or "").strip()
+    maidian = str(first.get("maidianInfo") or "").strip()
+    category = maidian.split("_", 1)[0].strip() if maidian else ""
+    return category, guid
+
+
+def _read_sku_lines(path: str) -> list[str]:
+    p = Path(path)
+    if not p.is_file():
+        print(f"文件不存在: {path}", file=sys.stderr)
+        sys.exit(2)
+    out: list[str] = []
+    for line in p.read_text(encoding="utf-8").splitlines():
+        s = line.strip()
+        if not s or s.startswith("#"):
+            continue
+        out.append(s)
+    return out
+
+
+def _loads_jd_plain_json(text: str) -> Any:
+    s = (text or "").strip()
+    if not s:
+        return None
+    try:
+        return json.loads(s)
+    except json.JSONDecodeError:
+        return None
+
+
+def _jd_business_ok(parsed: Any) -> bool:
+    if not isinstance(parsed, dict):
+        return False
+    if parsed.get("success") is False:
+        return False
+    c = parsed.get("code")
+    if c is None:
+        return True
+    return c == 0 or str(c) == "0"
+
+
+def _clean_text(v: Any) -> str:
+    if v is None:
+        return ""
+    s = str(v).strip()
+    return " ".join(s.split()) if s else ""
+
+
+def _large_pic_urls_from_picture_list(pil: Any) -> list[str]:
+    out: list[str] = []
+    if not isinstance(pil, list):
+        return out
+    for p in pil:
+        if not isinstance(p, dict):
+            continue
+        u = p.get("largePicURL") or p.get("largePicUrl")
+        if u:
+            t = str(u).strip()
+            if t and t not in out:
+                out.append(t)
+    return out
+
+
+def _is_jd_single_comment_dict(d: dict) -> bool:
+    """区分「一条评价」与标签/楼层等对象（getCommentListPage 里多为 commentInfo 扁平结构）。"""
+    cid = d.get("commentId")
+    if cid is None or str(cid).strip() == "":
+        return False
+    if d.get("userNickName") is None and not (
+        d.get("tagCommentContent") or d.get("commentData")
+    ):
+        return False
+    return True
+
+
+def _walk_collect_comment_dicts(obj: Any, acc: list[dict[str, Any]]) -> None:
+    """深度遍历 JSON，收集所有像单条评价的 dict（含 Lego 的 commentInfoList 项与列表页的 commentInfo）。"""
+    if isinstance(obj, dict):
+        if _is_jd_single_comment_dict(obj):
+            acc.append(obj)
+        for v in obj.values():
+            _walk_collect_comment_dicts(v, acc)
+    elif isinstance(obj, list):
+        for x in obj:
+            _walk_collect_comment_dicts(x, acc)
+
+
+def _row_from_comment_dict(sku: str, item: dict[str, Any]) -> dict[str, Any]:
+    text = _clean_text(
+        item.get("tagCommentContent") or item.get("commentData")
+    )
+    buy = _clean_text(
+        item.get("buyCountText") or item.get("repurchaseInfo")
+    )
+    date = _clean_text(
+        item.get("commentDate") or item.get("newCommentDate")
+    )
+    return {
+        "sku": str(sku).strip(),
+        "commentId": str(item.get("commentId") or "").strip(),
+        "userNickName": _clean_text(item.get("userNickName")),
+        "tagCommentContent": text,
+        "commentDate": date,
+        "buyCountText": buy,
+        "largePicURLs": _large_pic_urls_from_picture_list(
+            item.get("pictureInfoList")
+        ),
+        "commentScore":str(item.get("commentScore") or "").strip(),
+    }
+
+
+def extract_comment_rows_from_parsed(sku: str, parsed: Any) -> list[dict[str, Any]]:
+    """
+    从整段 parsed 深度遍历抽取评价：
+    - getLegoWareDetailComment：commentInfoList / lastCommentInfoList
+    - getCommentListPage：result.floors → data 里 { commentInfo: {...} } 已拍平为内层字段，同上
+    """
+    if not isinstance(parsed, dict):
+        return []
+    acc: list[dict[str, Any]] = []
+    _walk_collect_comment_dicts(parsed, acc)
+    seen: set[str] = set()
+    rows: list[dict[str, Any]] = []
+    for item in acc:
+        cid = str(item.get("commentId") or "").strip()
+        dedup_key = f"{sku}:{cid}" if cid else f"{sku}:{id(item)}"
+        if dedup_key in seen:
+            continue
+        seen.add(dedup_key)
+        rows.append(_row_from_comment_dict(sku, item))
+    return rows
+
+
+def _comment_flat_fieldnames() -> list[str]:
+    return [
+        "sku",
+        "commentId",
+        "userNickName",
+        "tagCommentContent",
+        "commentDate",
+        "buyCountText",
+        "largePicURLs",
+        "commentScore",
+    ]
+
+
+def _write_comments_csv(path: Path, rows: list[dict[str, Any]]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    buf = StringIO()
+    fn = _comment_flat_fieldnames()
+    w = csv.DictWriter(buf, fieldnames=fn, extrasaction="ignore")
+    w.writeheader()
+    for r in rows:
+        line = {k: r.get(k, "") for k in fn}
+        line["largePicURLs"] = json.dumps(
+            r.get("largePicURLs") or [], ensure_ascii=False
+        )
+        w.writerow(line)
+    path.write_text("\ufeff" + buf.getvalue(), encoding="utf-8")
+
+
+def write_comments_flat_csv(path: Path | str, rows: list[dict[str, Any]]) -> None:
+    """与 ``COMMENTS_OUT`` 为 ``.csv`` 时相同格式（UTF-8 BOM），供流水线等复用。"""
+    _write_comments_csv(Path(path), rows)
+
+
+def _append_comments_jsonl(f, rows: list[dict[str, Any]]) -> None:
+    for r in rows:
+        f.write(json.dumps(r, ensure_ascii=False) + "\n")
+
+
+def _extract_rows_from_jsonl_file(path: Path) -> list[dict[str, Any]]:
+    all_rows: list[dict[str, Any]] = []
+    for line in path.read_text(encoding="utf-8").splitlines():
+        s = line.strip()
+        if not s or s.startswith("#"):
+            continue
+        try:
+            rec = json.loads(s)
+        except json.JSONDecodeError:
+            continue
+        sku = str(rec.get("sku") or "").strip()
+        parsed = rec.get("parsed")
+        all_rows.extend(extract_comment_rows_from_parsed(sku, parsed))
+    return all_rows
+
+
+def main() -> None:
+    args = SimpleNamespace(
+        sku=(SKU or "").strip(),
+        sku_file=(SKU_FILE or "").strip(),
+        comment_num=int(COMMENT_NUM),
+        shop_type=str(SHOP_TYPE),
+        cookie_file=(COOKIE_FILE or "").strip(),
+        cookie=(COOKIE_OVERRIDE or "").strip(),
+        timeout=float(TIMEOUT_SEC),
+        request_delay=(REQUEST_DELAY or "").strip() or "30-60",
+        out=(OUT_JSONL or "").strip() or None,
+        pretty=bool(PRETTY),
+        raise_http=bool(RAISE_HTTP),
+        headed=bool(HEADED),
+        comments_out=(COMMENTS_OUT or "").strip(),
+        from_jsonl=(FROM_JSONL or "").strip(),
+        with_comment_list=bool(WITH_COMMENT_LIST),
+        list_pages=(LIST_PAGES or "1").strip(),
+        list_function_id=(LIST_FUNCTION_ID or "getCommentListPage").strip(),
+        list_style=(LIST_STYLE or "1").strip(),
+        list_category=(LIST_CATEGORY or "").strip(),
+        list_first_guid=(LIST_FIRST_GUID or "").strip(),
+    )
+
+    comments_out = args.comments_out
+    from_jsonl = args.from_jsonl
+
+    if from_jsonl:
+        if not comments_out:
+            print("离线模式：请配置 FROM_JSONL 与 COMMENTS_OUT", file=sys.stderr)
+            sys.exit(2)
+        co_path = Path(comments_out)
+        rows = _extract_rows_from_jsonl_file(Path(from_jsonl))
+        suf = co_path.suffix.lower()
+        if suf == ".csv":
+            _write_comments_csv(co_path, rows)
+        elif suf == ".jsonl":
+            co_path.parent.mkdir(parents=True, exist_ok=True)
+            with co_path.open("w", encoding="utf-8") as cf:
+                _append_comments_jsonl(cf, rows)
+        else:
+            print("COMMENTS_OUT 请使用 .csv 或 .jsonl 扩展名", file=sys.stderr)
+            sys.exit(2)
+        print(f"[京东] 已从 JSONL 抽取 {len(rows)} 条评价 → {co_path}", file=sys.stderr)
+        return
+
+    sku_one = args.sku
+    sku_file = args.sku_file
+    if bool(sku_one) == bool(sku_file):
+        print("请只配置其一：SKU 或 SKU_FILE", file=sys.stderr)
+        sys.exit(2)
+
+    skus = [sku_one] if sku_one else _read_sku_lines(sku_file)
+    if not skus:
+        print("SKU 列表为空", file=sys.stderr)
+        sys.exit(2)
+
+    rd = args.request_delay
+    try:
+        delay_range = parse_request_delay_range(rd)
+    except ValueError as e:
+        print(f"[京东] REQUEST_DELAY 无效: {e}", file=sys.stderr)
+        sys.exit(2)
+
+    cookie_file_node = (args.cookie_file or "").strip()
+    if cookie_file_node:
+        cookie_file_node = str(Path(cookie_file_node).resolve())
+
+    timeout_ms = max(1000, int(args.timeout * 1000))
+    cookie_override = (args.cookie or "").strip()
+
+    out_f = None
+    if args.out:
+        outp = Path(args.out)
+        outp.parent.mkdir(parents=True, exist_ok=True)
+        out_f = outp.open("w", encoding="utf-8")
+
+    comments_path = Path(comments_out) if comments_out else None
+    comments_csv_rows: list[dict[str, Any]] = []
+    comments_jsonl_f = None
+    if comments_path is not None:
+        suf = comments_path.suffix.lower()
+        if suf not in (".csv", ".jsonl"):
+            print("COMMENTS_OUT 请使用 .csv 或 .jsonl 扩展名", file=sys.stderr)
+            sys.exit(2)
+        comments_path.parent.mkdir(parents=True, exist_ok=True)
+        if suf == ".jsonl":
+            comments_jsonl_f = comments_path.open("w", encoding="utf-8")
+
+    try:
+        with sync_playwright() as pw:
+            browser = pw.chromium.launch(headless=not args.headed)
+            context = browser.new_context()
+            try:
+                gap_before_next = False
+                for sku in skus:
+                    if gap_before_next:
+                        _sleep_between_jd_requests(delay_range, "请求间隔")
+
+                    pack = export_item_comment_request_json(
+                        sku,
+                        comment_num=args.comment_num,
+                        shop_type=str(args.shop_type),
+                        cookie_file=cookie_file_node or None,
+                    )
+                    url = pack["url"]
+                    hdrs = {str(k): str(v) for k, v in pack["headers"].items()}
+                    if cookie_override:
+                        hdrs["Cookie"] = cookie_override
+
+                    resp = context.request.get(url, headers=hdrs, timeout=timeout_ms)
+                    status = resp.status
+                    text = resp.text()
+                    print(
+                        f"[京东] sku={sku} HTTP {status} {resp.status_text or ''}",
+                        file=sys.stderr,
+                    )
+
+                    if args.raise_http and status // 100 != 2:
+                        print(f"[京东] HTTP {status}，--raise-http 已启用", file=sys.stderr)
+                        sys.exit(1)
+
+                    parsed = _loads_jd_plain_json(text)
+                    http_ok = 200 <= status < 300
+                    row = {
+                        "sku": sku,
+                        "http_status": status,
+                        "http_ok": http_ok,
+                        "ok": http_ok and _jd_business_ok(parsed),
+                        "parsed": parsed,
+                        "raw": text if parsed is None else None,
+                    }
+
+                    line = json.dumps(row, ensure_ascii=False)
+                    if out_f:
+                        out_f.write(line + "\n")
+                    elif len(skus) == 1 and args.pretty and parsed is not None:
+                        sys.stdout.write(
+                            json.dumps(parsed, ensure_ascii=False, indent=2) + "\n"
+                        )
+                    elif len(skus) == 1:
+                        sys.stdout.write(line + "\n")
+                    else:
+                        sys.stdout.write(line + "\n")
+
+                    flat = extract_comment_rows_from_parsed(sku, parsed)
+                    if comments_path is not None:
+                        if comments_jsonl_f is not None:
+                            _append_comments_jsonl(comments_jsonl_f, flat)
+                        else:
+                            comments_csv_rows.extend(flat)
+
+                    gap_before_next = True
+
+                    if args.with_comment_list:
+                        cat, fguid = category_and_first_guid_from_lego(parsed)
+                        if (args.list_category or "").strip():
+                            cat = (args.list_category or "").strip()
+                        if (args.list_first_guid or "").strip():
+                            fguid = (args.list_first_guid or "").strip()
+                        if not cat or not fguid:
+                            print(
+                                "[京东] --with-comment-list 跳过：缺少 category 或 firstCommentGuid；"
+                                "请确认首屏有评价，或使用 --list-category / --list-first-guid",
+                                file=sys.stderr,
+                            )
+                        else:
+                            pages = parse_list_pages_spec(args.list_pages or "1")
+                            lfid = (args.list_function_id or "getCommentListPage").strip()
+                            lstyle = (args.list_style or "1").strip()
+                            for pi, pnum in enumerate(pages):
+                                if gap_before_next:
+                                    _sleep_between_jd_requests(
+                                        delay_range, "列表分页请求间隔"
+                                    )
+                                is_first = pi == 0
+                                pack_p = export_item_comment_page_request_json(
+                                    sku,
+                                    category=cat,
+                                    first_guid=fguid,
+                                    page_num=pnum,
+                                    is_first=is_first,
+                                    function_id=lfid,
+                                    shop_type=str(args.shop_type),
+                                    cookie_file=cookie_file_node or None,
+                                    style=lstyle,
+                                )
+                                url_p = pack_p["url"]
+                                hdrs_p = {
+                                    str(k): str(v)
+                                    for k, v in pack_p["headers"].items()
+                                }
+                                if cookie_override:
+                                    hdrs_p["Cookie"] = cookie_override
+                                form_p = pack_p.get("form") or {}
+                                form_pw = {
+                                    str(k): str(v) for k, v in form_p.items()
+                                }
+                                resp_p = context.request.post(
+                                    url_p,
+                                    headers=hdrs_p,
+                                    form=form_pw,
+                                    timeout=timeout_ms,
+                                )
+                                st_p = resp_p.status
+                                text_p = resp_p.text()
+                                print(
+                                    f"[京东] sku={sku} 列表页 pageNum={pnum} "
+                                    f"HTTP {st_p} {resp_p.status_text or ''}",
+                                    file=sys.stderr,
+                                )
+                                if args.raise_http and st_p // 100 != 2:
+                                    print(
+                                        "[京东] 列表分页 --raise-http 已启用",
+                                        file=sys.stderr,
+                                    )
+                                    sys.exit(1)
+                                parsed_p = _loads_jd_plain_json(text_p)
+                                http_ok_p = 200 <= st_p < 300
+                                row_p = {
+                                    "sku": sku,
+                                    "kind": "comment_list_page",
+                                    "page_num": pnum,
+                                    "http_status": st_p,
+                                    "http_ok": http_ok_p,
+                                    "ok": http_ok_p
+                                    and _jd_business_ok(parsed_p),
+                                    "parsed": parsed_p,
+                                    "raw": text_p if parsed_p is None else None,
+                                }
+                                line_p = json.dumps(row_p, ensure_ascii=False)
+                                if out_f:
+                                    out_f.write(line_p + "\n")
+                                else:
+                                    sys.stdout.write(line_p + "\n")
+
+                                flat_p = extract_comment_rows_from_parsed(
+                                    sku, parsed_p
+                                )
+                                if comments_path is not None:
+                                    if comments_jsonl_f is not None:
+                                        _append_comments_jsonl(
+                                            comments_jsonl_f, flat_p
+                                        )
+                                    else:
+                                        comments_csv_rows.extend(flat_p)
+                                gap_before_next = True
+            finally:
+                browser.close()
+    finally:
+        if out_f:
+            out_f.close()
+        if comments_jsonl_f:
+            comments_jsonl_f.close()
+        if comments_path is not None and comments_path.suffix.lower() == ".csv":
+            _write_comments_csv(comments_path, comments_csv_rows)
+
+
+if __name__ == "__main__":
+    main()

backend/crawler_copy/jd_pc_search/comment/jd_h5st_item_comment.js

@@ -0,0 +1,114 @@
+/**
+ * 商品详情 getLegoWareDetailComment 的 h5st（ParamsSign appId fb5df）。
+ * 与 jd_h5st.js 分离，避免改动搜索列表 pc_search 链路。
+ */
+require("../common/jd_browser_env.js");
+require("../common/code.js");
+const CryptoJS = require("crypto-js");
+
+const ITEM_COMMENT_PARAMS_SIGN_APP_ID = "fb5df";
+
+let _psign = null;
+function _ensurePsignItemComment() {
+  if (!_psign) {
+    _psign = new window.ParamsSign({
+      appId: ITEM_COMMENT_PARAMS_SIGN_APP_ID,
+      preRequest: false,
+      onSign: () => {},
+      onRequestTokenRemotely: () => {},
+    });
+  }
+  return _psign;
+}
+
+/**
+ * @param {object} opt
+ * @param {number|string} opt.sku
+ * @param {number} [opt.commentNum=5]
+ * @param {string} [opt.shopType='0']
+ * @param {string} [opt.source='pc']
+ * @param {number} [opt.t]
+ */
+function get_h5st_item_lego_detail_comment(opt) {
+  const o = opt || {};
+  const sku = o.sku != null ? Number(o.sku) : NaN;
+  if (!Number.isFinite(sku) || sku <= 0) {
+    throw new Error("get_h5st_item_lego_detail_comment: 需要有效 opt.sku");
+  }
+  const commentNum = Math.max(
+    1,
+    parseInt(String(o.commentNum != null ? o.commentNum : 5), 10) || 5
+  );
+  const shopType = o.shopType != null ? String(o.shopType) : "0";
+  const source = o.source != null ? String(o.source) : "pc";
+  const time = o.t != null ? Number(o.t) : Date.now();
+
+  const bodyObj = {
+    shopType,
+    sku,
+    commentNum,
+    source,
+  };
+  const bodyJson = JSON.stringify(bodyObj);
+  const bodySha = CryptoJS.SHA256(bodyJson).toString();
+  const functionId = "getLegoWareDetailComment";
+  const paramsH5sign = {
+    appid: "item-v3",
+    functionId,
+    client: "pc",
+    clientVersion: "1.0.0",
+    t: time,
+    body: bodySha,
+  };
+  const signed = _ensurePsignItemComment()._$sdnmd({
+    ...paramsH5sign,
+  });
+
+  return {
+    h5st: signed.h5st,
+    signed,
+    bodyJson,
+    bodySha256: signed.body,
+    bodyObj,
+    tQuerySecond: String(signed.t),
+  };
+}
+
+/**
+ * @param {object} pack get_h5st_item_lego_detail_comment 返回值
+ * @param {object} opts
+ * @param {string} opts.uuid
+ * @param {string} opts.xApiEidToken
+ * @param {'json'|'sha256'} [opts.bodyMode='json']
+ */
+function build_item_lego_comment_api_url(pack, opts) {
+  const uuid = opts.uuid != null ? String(opts.uuid) : "";
+  const xApiEidToken =
+    opts.xApiEidToken != null ? String(opts.xApiEidToken) : "";
+  const bodyMode = opts.bodyMode === "sha256" ? "sha256" : "json";
+  const signed = pack.signed;
+  const bodyValue = bodyMode === "sha256" ? pack.bodySha256 : pack.bodyJson;
+  const build = opts.build != null ? String(opts.build) : "100000";
+  const qParts = [
+    ["functionId", signed.functionId],
+    ["body", bodyValue],
+    ["h5st", signed.h5st],
+    ["uuid", uuid],
+    ["loginType", "3"],
+    ["appid", signed.appid],
+    ["clientVersion", signed.clientVersion],
+    ["client", signed.client],
+    ["t", pack.tQuerySecond],
+    ["x-api-eid-token", xApiEidToken],
+    ["build", build],
+  ];
+  const qs = qParts
+    .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`)
+    .join("&");
+  return `https://api.m.jd.com/?${qs}`;
+}
+
+module.exports = {
+  get_h5st_item_lego_detail_comment,
+  build_item_lego_comment_api_url,
+};

backend/crawler_copy/jd_pc_search/comment/jd_h5st_item_comment_page.js

@@ -0,0 +1,172 @@
+/**
+ * 商品页「评价列表」分页：POST https://api.m.jd.com/client.action
+ *（application/x-www-form-urlencoded，与 item.jd.com 抓包一致）。
+ *
+ * - 参与签名的 appid：**pc-rate-qa**（非 item-v3）
+ * - ParamsSign 构造器 appId：**01a47**（h5st 第三段，非 fb5df）
+ * - 表单字段：appid、body、client、clientVersion、functionId、h5st、loginType、t、uuid
+ *
+ * 首包 isFirstRequest:true 无 style；后续包 isFirstRequest:false 且含 style:"1"。
+ */
+require("../common/jd_browser_env.js");
+require("../common/code.js");
+const CryptoJS = require("crypto-js");
+
+const COMMENT_LIST_CLIENT_ACTION_APPID = "pc-rate-qa";
+/** 与浏览器 h5st 中第三段一致 */
+const COMMENT_LIST_PARAMS_SIGN_APP_ID = "01a47";
+
+let _psign = null;
+function _ensurePsign() {
+  if (!_psign) {
+    _psign = new window.ParamsSign({
+      appId: COMMENT_LIST_PARAMS_SIGN_APP_ID,
+      preRequest: false,
+      onSign: () => {},
+      onRequestTokenRemotely: () => {},
+    });
+  }
+  return _psign;
+}
+
+function _extInfoBlock(spuId) {
+  const s = String(spuId != null ? spuId : "");
+  return {
+    isQzc: "0",
+    spuId: s,
+    commentRate: "1",
+    needTopAlbum: "1",
+    bbtf: "",
+    userGroupComment: "1",
+  };
+}
+
+/**
+ * @param {object} opt
+ * @param {string} opt.sku
+ * @param {string} opt.category 如 36574;44419;44439（来自首屏 maidianInfo 等）
+ * @param {string} opt.firstCommentGuid 首条评价 guid
+ * @param {string|number} opt.pageNum
+ * @param {boolean} opt.isFirstRequest
+ * @param {string} [opt.shopType='0']
+ * @param {string} [opt.spuId] 默认与 sku 字符串相同
+ * @param {string} [opt.style='1'] 仅 isFirstRequest 为 false 时写入 body
+ * @param {string} [opt.num='10']
+ * @param {string} [opt.pageSize='10']
+ * @param {string} [opt.sortType='5']
+ * @param {number} [opt.t]
+ */
+function build_item_comment_page_body(opt) {
+  const o = opt || {};
+  const skuStr = String(o.sku != null ? o.sku : "").trim();
+  if (!skuStr) throw new Error("build_item_comment_page_body: 需要 opt.sku");
+  const categoryStr = String(o.category != null ? o.category : "").trim();
+  if (!categoryStr) throw new Error("build_item_comment_page_body: 需要 opt.category");
+  const guid = String(o.firstCommentGuid != null ? o.firstCommentGuid : "").trim();
+  if (!guid) throw new Error("build_item_comment_page_body: 需要 opt.firstCommentGuid");
+  const shopTypeStr = o.shopType != null ? String(o.shopType) : "0";
+  const spuId = o.spuId != null ? String(o.spuId) : skuStr;
+  const pageNum = String(o.pageNum != null ? o.pageNum : "1");
+  const isFirst = Boolean(o.isFirstRequest);
+  const num = o.num != null ? String(o.num) : "10";
+  const pageSize = o.pageSize != null ? String(o.pageSize) : "10";
+  const sortType = o.sortType != null ? String(o.sortType) : "5";
+  const extInfo = _extInfoBlock(spuId);
+
+  /** @type {Record<string, unknown>} */
+  const base = {
+    requestSource: "pc",
+    shopComment: 0,
+    sameComment: 0,
+    channel: null,
+    extInfo,
+    num,
+    pictureCommentType: "A",
+    scval: null,
+    shadowMainSku: "0",
+    shopType: shopTypeStr,
+    firstCommentGuid: guid,
+    sku: skuStr,
+    category: categoryStr,
+    shieldCurrentComment: "1",
+    pageSize,
+    isFirstRequest: isFirst,
+  };
+  if (!isFirst) {
+    base.style = o.style != null ? String(o.style) : "1";
+  }
+  base.isCurrentSku = false;
+  base.sortType = sortType;
+  base.tagId = "";
+  base.tagType = "";
+  base.type = "0";
+  base.pageNum = pageNum;
+  return base;
+}
+
+/**
+ * @param {object} opt 同 build_item_comment_page_body，另需 functionId
+ * @param {string} opt.functionId
+ */
+function get_h5st_item_comment_page(opt) {
+  const o = opt || {};
+  const functionId = o.functionId != null ? String(o.functionId).trim() : "";
+  if (!functionId) throw new Error("get_h5st_item_comment_page: 需要 opt.functionId");
+  const time = o.t != null ? Number(o.t) : Date.now();
+  const bodyObj = build_item_comment_page_body(o);
+  const bodyJson = JSON.stringify(bodyObj);
+  const bodySha = CryptoJS.SHA256(bodyJson).toString();
+  const paramsH5sign = {
+    appid: COMMENT_LIST_CLIENT_ACTION_APPID,
+    functionId,
+    client: "pc",
+    clientVersion: "1.0.0",
+    t: time,
+    body: bodySha,
+  };
+  const signed = _ensurePsign()._$sdnmd({ ...paramsH5sign });
+  return {
+    h5st: signed.h5st,
+    signed,
+    bodyJson,
+    bodySha256: signed.body,
+    bodyObj,
+    tQuerySecond: String(signed.t),
+  };
+}
+
+const CLIENT_ACTION_URL = "https://api.m.jd.com/client.action";
+
+/**
+ * POST client.action 的 x-www-form-urlencoded 字段（顺序与常见抓包一致）。
+ * @param {object} pack get_h5st_item_comment_page 返回值
+ * @param {object} opts
+ * @param {string} opts.uuid
+ */
+function build_item_comment_page_client_action_form(pack, opts) {
+  const uuid = opts.uuid != null ? String(opts.uuid) : "";
+  if (!uuid) throw new Error("build_item_comment_page_client_action_form: 需要 opts.uuid");
+  const signed = pack.signed;
+  return {
+    url: CLIENT_ACTION_URL,
+    form: {
+      appid: COMMENT_LIST_CLIENT_ACTION_APPID,
+      body: pack.bodyJson,
+      client: signed.client,
+      clientVersion: signed.clientVersion,
+      functionId: signed.functionId,
+      h5st: signed.h5st,
+      loginType: "3",
+      t: pack.tQuerySecond,
+      uuid,
+    },
+  };
+}
+
+module.exports = {
+  build_item_comment_page_body,
+  get_h5st_item_comment_page,
+  build_item_comment_page_client_action_form,
+  COMMENT_LIST_CLIENT_ACTION_APPID,
+  COMMENT_LIST_PARAMS_SIGN_APP_ID,
+};

backend/crawler_copy/jd_pc_search/comment/jd_pc_item_comment_headers.js

@@ -0,0 +1,34 @@
+/**
+ * 与 Chrome 访问 item.jd.com → api.m.jd.com getLegoWareDetailComment 的请求头对齐。
+ */
+function buildJdPcItemCommentHeaders(opts) {
+  const sku = opts.sku != null ? String(opts.sku).trim() : "";
+  const referer = sku
+    ? `https://item.jd.com/${encodeURIComponent(sku)}.html`
+    : "https://item.jd.com/";
+  const h = {
+    Accept: "application/json, text/plain, */*",
+    "Accept-Encoding": "gzip, deflate, br",
+    "Accept-Language": "zh-CN,zh;q=0.9",
+    "Cache-Control": "no-cache",
+    Pragma: "no-cache",
+    Priority: "u=1, i",
+    "User-Agent":
+      "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
+    Referer: referer,
+    Origin: "https://item.jd.com",
+    "sec-ch-ua":
+      '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"',
+    "sec-ch-ua-mobile": "?0",
+    "sec-ch-ua-platform": '"Windows"',
+    "sec-fetch-dest": "empty",
+    "sec-fetch-mode": "cors",
+    "sec-fetch-site": "same-site",
+    "x-referer-page": referer,
+    "x-rp-client": "h5_1.0.0",
+  };
+  if (opts.cookie) h.Cookie = opts.cookie;
+  return h;
+}
+
+module.exports = { buildJdPcItemCommentHeaders };

backend/crawler_copy/jd_pc_search/common/__init__.py

@@ -0,0 +1 @@
+# 京东 PC 爬虫共享：Cookie、签名环境、请求间隔工具等。

backend/crawler_copy/jd_pc_search/common/jd_browser_env.js

@@ -0,0 +1,131 @@
+/**
+ * 在 Node 下为 crawler/code.js（ParamsSign / h5st 相关）补齐最小「浏览器」全局环境。
+ *
+ * 用法（必须在 require("./code.js") 之前）：
+ *   require("./jd_browser_env.js");
+ *
+ * 说明：仅满足当前 bundle 在加载期访问到的 DOM/BOM；若京东更新脚本仍可能缺字段，再按报错补桩。
+ */
+(function applyJDBrowserEnv() {
+  const g = globalThis;
+  if (g.__JD_BROWSER_ENV_APPLIED__) {
+    return;
+  }
+  g.__JD_BROWSER_ENV_APPLIED__ = true;
+
+  g.window = g;
+
+  function Element() {}
+  Element.prototype.scrollIntoViewIfNeeded = function () {};
+  g.Element = Element;
+
+  const memStore = Object.create(null);
+  const storage = {
+    getItem(k) {
+      return Object.prototype.hasOwnProperty.call(memStore, k)
+        ? memStore[k]
+        : null;
+    },
+    setItem(k, v) {
+      memStore[String(k)] = String(v);
+    },
+    removeItem(k) {
+      delete memStore[String(k)];
+    },
+    clear() {
+      for (const k of Object.keys(memStore)) {
+        delete memStore[k];
+      }
+    },
+  };
+
+  g.document = {
+    all: null,
+    cookie: "",
+    domain: "jd.com",
+    referrer: "https://search.jd.com/",
+    createElement() {
+      return Object.assign(new Element(), {
+        style: {},
+        appendChild() {},
+        setAttribute() {},
+        remove() {},
+      });
+    },
+    getElementsByTagName() {
+      return [{ appendChild() {} }];
+    },
+    querySelector() {
+      return null;
+    },
+  };
+
+  g.navigator = {
+    userAgent:
+      "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
+    language: "zh-CN",
+    languages: ["zh-CN", "zh", "en"],
+    mimeTypes: { length: 0 },
+    plugins: { length: 0 },
+    appVersion: "5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
+    platform: "Win32",
+    webdriver: false,
+    hardwareConcurrency: 8,
+  };
+
+  g.location = {
+    href: "https://search.jd.com/Search?keyword=&enc=utf-8",
+    origin: "https://search.jd.com",
+    protocol: "https:",
+    host: "search.jd.com",
+    pathname: "/Search",
+    search: "",
+  };
+
+  g.history = {
+    replaceState() {},
+    pushState() {},
+    back() {},
+  };
+
+  g.screen = {
+    width: 1920,
+    height: 1080,
+    availWidth: 1920,
+    availHeight: 1040,
+  };
+  g.outerWidth = 1920;
+  g.outerHeight = 1080;
+  g.innerWidth = 1920;
+  g.innerHeight = 969;
+  g.devicePixelRatio = 1;
+
+  g.chrome = {};
+
+  g.localStorage = { ...storage };
+  g.sessionStorage = { ...storage };
+
+  function XMLHttpRequest() {
+    this.readyState = 0;
+    this.status = 0;
+    this.responseText = "";
+  }
+  XMLHttpRequest.prototype.open = function () {};
+  XMLHttpRequest.prototype.setRequestHeader = function () {};
+  XMLHttpRequest.prototype.send = function () {};
+  XMLHttpRequest.prototype.abort = function () {};
+  g.XMLHttpRequest = XMLHttpRequest;
+
+  g.getComputedStyle = function () {
+    return {};
+  };
+
+  g.MutationObserver = function () {
+    this.observe = function () {};
+    this.disconnect = function () {};
+  };
+
+  g.WebKitMutationObserver = g.MutationObserver;
+})();
+
+module.exports = {};

backend/crawler_copy/jd_pc_search/common/jd_delay_utils.py

@@ -0,0 +1,40 @@
+# -*- coding: utf-8 -*-
+"""搜索/评论等脚本共用的「请求间隔」解析与 sleep（避免 comment 依赖整份 jd_h5_search_requests）。"""
+from __future__ import annotations
+
+import random
+import sys
+import time
+
+
+def parse_request_delay_range(s: str | None) -> tuple[float, float]:
+    """
+    解析 CLI「MIN-MAX」为随机等待区间（秒）。
+    例：``30-60`` → uniform(30, 60)。
+    """
+    t = (s or "").strip()
+    if not t:
+        raise ValueError("空字符串")
+    parts = t.split("-", 1)
+    if len(parts) != 2:
+        raise ValueError(f"应为 MIN-MAX（秒），如 30-60，收到: {t!r}")
+    lo = float(parts[0].strip())
+    hi = float(parts[1].strip())
+    if lo < 0 or hi < 0:
+        raise ValueError("延迟不能为负")
+    if lo > hi:
+        lo, hi = hi, lo
+    return (lo, hi)
+
+
+def sleep_pc_search_request_gap(delay_range: tuple[float, float] | None) -> None:
+    """在已有至少一次请求之后、发起下一次之前调用。"""
+    if not delay_range:
+        return
+    lo, hi = delay_range
+    sec = random.uniform(lo, hi)
+    print(
+        f"[京东] pc_search 间隔 sleep {sec:.1f}s（区间 {lo:g}–{hi:g}）",
+        file=sys.stderr,
+    )
+    time.sleep(sec)

backend/crawler_copy/jd_pc_search/common/jd_https_fetch.js

@@ -0,0 +1,60 @@
+/**
+ * 直连 HTTPS GET（不走代理）。自动按 Content-Encoding 解压 gzip/deflate/br。
+ */
+const https = require("https");
+const zlib = require("zlib");
+const { URL } = require("url");
+
+function decodeBody(buf, headers) {
+  if (!buf || !buf.length) return "";
+  const enc = String(headers["content-encoding"] || "").toLowerCase();
+  try {
+    if (enc.includes("br")) return zlib.brotliDecompressSync(buf).toString("utf8");
+    if (enc.includes("gzip")) return zlib.gunzipSync(buf).toString("utf8");
+    if (enc.includes("deflate")) return zlib.inflateSync(buf).toString("utf8");
+  } catch {
+    /* 非压缩或损坏时按原文 UTF-8 */
+  }
+  return buf.toString("utf8");
+}
+
+function httpsGet(urlString, headers) {
+  return new Promise((resolve, reject) => {
+    const u = new URL(urlString);
+    const opt = {
+      hostname: u.hostname,
+      port: u.port || 443,
+      path: u.pathname + u.search,
+      method: "GET",
+      headers: { ...headers, Host: u.hostname },
+    };
+    const req = https.request(opt, (res) => {
+      const chunks = [];
+      res.on("data", (c) => chunks.push(c));
+      res.on("end", () => {
+        const buf = Buffer.concat(chunks);
+        const body = decodeBody(buf, res.headers);
+        resolve({
+          status: res.statusCode || 0,
+          statusMessage: res.statusMessage || "",
+          headers: res.headers,
+          body,
+        });
+      });
+    });
+    req.on("error", (err) => {
+      if (err && err.name === "AggregateError" && err.errors && err.errors[0]) {
+        reject(err.errors[0]);
+      } else {
+        reject(err);
+      }
+    });
+    req.setTimeout(45000, () => {
+      req.destroy();
+      reject(new Error("timeout"));
+    });
+    req.end();
+  });
+}
+
+module.exports = { httpsGet };

backend/crawler_copy/jd_pc_search/common/jd_search_common.js

@@ -0,0 +1,110 @@
+const fs = require("fs");
+const path = require("path");
+
+const DEFAULT_COOKIE = path.join(__dirname, "jd_cookie.txt");
+
+function readCookieFile(p = DEFAULT_COOKIE) {
+  if (!fs.existsSync(p)) return "";
+  const chunks = [];
+  for (const line of fs.readFileSync(p, "utf8").split(/\r?\n/)) {
+    const s = line.trim();
+    if (!s || s.startsWith("#")) continue;
+    chunks.push(s);
+  }
+  return chunks.join("; ").trim();
+}
+
+function cookieGet(cookie, name) {
+  const m = new RegExp(`(?:^|;\\s*)${name}=([^;]*)`).exec(cookie);
+  return m ? decodeURIComponent(m[1].trim()) : "";
+}
+
+function parseSearchCliArgs(argv = process.argv.slice(2)) {
+  const out = {
+    q: process.env.JD_KEYWORD || "低GI",
+    page: Math.max(1, parseInt(process.env.JD_PAGE || "1", 10) || 1),
+    s: Math.max(1, parseInt(process.env.JD_S || "1", 10) || 1),
+    pvid: (process.env.JD_PVID || "").trim() || null,
+    cookiePath: DEFAULT_COOKIE,
+  };
+  const a = argv;
+  for (let i = 0; i < a.length; i++) {
+    const x = a[i];
+    if (x === "--q" && a[i + 1]) {
+      out.q = a[++i];
+      continue;
+    }
+    if (x.startsWith("--q=")) {
+      out.q = x.slice(4);
+      continue;
+    }
+    // --page：写入 pc_search **body.page**（与 Python 脚本「逻辑页 L」不同；L 的首包为 2L-1）
+    if (x === "--page" && a[i + 1]) {
+      out.page = Math.max(1, parseInt(a[++i], 10) || 1);
+      continue;
+    }
+    if (x.startsWith("--page=")) {
+      out.page = Math.max(1, parseInt(x.slice(7), 10) || 1);
+      continue;
+    }
+    if (x === "--s" && a[i + 1]) {
+      out.s = Math.max(1, parseInt(a[++i], 10) || 1);
+      continue;
+    }
+    if (x.startsWith("--s=")) {
+      out.s = Math.max(1, parseInt(x.slice(4), 10) || 1);
+      continue;
+    }
+    if (x === "--pvid" && a[i + 1]) {
+      out.pvid = String(a[++i]).trim() || null;
+      continue;
+    }
+    if (x.startsWith("--pvid=")) {
+      const v = x.slice(7).trim();
+      out.pvid = v || null;
+      continue;
+    }
+    if (x === "--cookie-file" && a[i + 1]) {
+      out.cookiePath = String(a[++i]);
+      continue;
+    }
+    if (x.startsWith("--cookie-file=")) {
+      out.cookiePath = x.slice(14);
+      continue;
+    }
+  }
+  // 兼容：node script.js 低GI 2（首参词、次参为 **body.page**）。不得在有 --q 时把 a[1]（往往是关键词）当成 page。
+  if (a[0] && !a[0].startsWith("-")) {
+    out.q = a[0];
+  }
+  if (a[1] && !a[1].startsWith("-")) {
+    const pi = parseInt(a[1], 10);
+    if (!Number.isNaN(pi)) {
+      out.page = Math.max(1, pi);
+    }
+  }
+  return out;
+}
+
+function loadJdSearchAuth(cookiePath = DEFAULT_COOKIE) {
+  const cookie = readCookieFile(cookiePath);
+  const uuid =
+    process.env.JD_UUID ||
+    cookieGet(cookie, "__jdu") ||
+    cookieGet(cookie, "mba_muid") ||
+    "";
+  const xApiEidToken =
+    process.env.JD_X_API_EID_TOKEN ||
+    cookieGet(cookie, "3AB9D23F7A4B3CSS") ||
+    cookieGet(cookie, "cd_eid") ||
+    "";
+  return { cookie, uuid, xApiEidToken };
+}
+
+module.exports = {
+  DEFAULT_COOKIE,
+  readCookieFile,
+  cookieGet,
+  parseSearchCliArgs,
+  loadJdSearchAuth,
+};

backend/crawler_copy/jd_pc_search/detail/jd_export_detail_ware_business_request.js

@@ -0,0 +1,135 @@
+/**
+ * stdout 一行 JSON：{ url, headers }，供 jd_detail_ware_business_requests.py 使用。
+ *
+ *   cd crawler/jd_pc_search/detail && node jd_export_detail_ware_business_request.js --sku 10166848058665
+ *   node jd_export_detail_ware_business_request.js --sku 10166848058665 --area 19_1601_50258_129167
+ *
+ * 与浏览器 URL 对齐：可用 --uuid、--x-api-eid-token 覆盖（同次抓包里 Query 可能与 Cookie 中 3CSS 不一致）；
+ * 亦可设环境变量 JD_UUID、JD_X_API_EID_TOKEN（见 common/jd_search_common.js）。
+ */
+const path = require("path");
+const { loadJdSearchAuth } = require("../common/jd_search_common.js");
+const {
+  get_h5st_detail_ware_business,
+  build_detail_ware_business_api_url,
+} = require("./jd_h5st_detail_ware_business.js");
+const {
+  buildJdPcDetailWareBusinessHeaders,
+} = require("./jd_pc_detail_ware_business_headers.js");
+
+const DEFAULT_COOKIE = path.join(__dirname, "..", "common", "jd_cookie.txt");
+
+function parseCliArgs(argv = process.argv.slice(2)) {
+  const out = {
+    skuId: null,
+    area: null,
+    num: "1",
+    sfTime: "1,0,0",
+    cookiePath: DEFAULT_COOKIE,
+    uuid: null,
+    xApiEidToken: null,
+  };
+  const a = argv;
+  for (let i = 0; i < a.length; i++) {
+    const x = a[i];
+    if ((x === "--sku" || x === "--sku-id") && a[i + 1]) {
+      out.skuId = String(a[++i]).trim();
+      continue;
+    }
+    if (x.startsWith("--sku=")) {
+      out.skuId = x.slice(6).trim();
+      continue;
+    }
+    if (x.startsWith("--sku-id=")) {
+      out.skuId = x.slice(9).trim();
+      continue;
+    }
+    if (x === "--area" && a[i + 1]) {
+      out.area = String(a[++i]).trim();
+      continue;
+    }
+    if (x.startsWith("--area=")) {
+      out.area = x.slice(7).trim();
+      continue;
+    }
+    if (x === "--num" && a[i + 1]) {
+      out.num = String(a[++i]).trim();
+      continue;
+    }
+    if (x.startsWith("--num=")) {
+      out.num = x.slice(6).trim();
+      continue;
+    }
+    if (x === "--sf-time" && a[i + 1]) {
+      out.sfTime = String(a[++i]).trim();
+      continue;
+    }
+    if (x.startsWith("--sf-time=")) {
+      out.sfTime = x.slice(10).trim();
+      continue;
+    }
+    if (x === "--cookie-file" && a[i + 1]) {
+      out.cookiePath = String(a[++i]);
+      continue;
+    }
+    if (x.startsWith("--cookie-file=")) {
+      out.cookiePath = x.slice(14);
+      continue;
+    }
+    if (x === "--uuid" && a[i + 1]) {
+      out.uuid = String(a[++i]).trim();
+      continue;
+    }
+    if (x.startsWith("--uuid=")) {
+      out.uuid = x.slice(7).trim();
+      continue;
+    }
+    if (x === "--x-api-eid-token" && a[i + 1]) {
+      out.xApiEidToken = String(a[++i]).trim();
+      continue;
+    }
+    if (x.startsWith("--x-api-eid-token=")) {
+      out.xApiEidToken = x.slice(18).trim();
+      continue;
+    }
+  }
+  return out;
+}
+
+try {
+  const cli = parseCliArgs();
+  if (!cli.skuId) {
+    throw new Error("需要 --sku 或 --sku-id（商品 SKU）");
+  }
+
+  if (cli.uuid) process.env.JD_UUID = cli.uuid;
+  if (cli.xApiEidToken) process.env.JD_X_API_EID_TOKEN = cli.xApiEidToken;
+
+  const { cookie, uuid, xApiEidToken } = loadJdSearchAuth(cli.cookiePath);
+  if (!cookie) throw new Error("Cookie 为空或不存在（jd_cookie.txt 或 --cookie-file）");
+  if (!uuid || !xApiEidToken) {
+    throw new Error(
+      "缺少 uuid 或 x-api-eid-token（Cookie 中 __jdu/mba_muid 与 3AB9D23F7A4B3CSS）"
+    );
+  }
+
+  const pack = get_h5st_detail_ware_business({
+    skuId: cli.skuId,
+    area: cli.area || undefined,
+    num: cli.num,
+    sfTime: cli.sfTime,
+  });
+  const url = build_detail_ware_business_api_url(pack, {
+    uuid,
+    xApiEidToken,
+    bodyMode: "json",
+  });
+  const headers = buildJdPcDetailWareBusinessHeaders({
+    cookie,
+    skuId: cli.skuId,
+  });
+  process.stdout.write(JSON.stringify({ url, headers }));
+} catch (e) {
+  console.error(e.message || String(e));
+  process.exit(1);
+}

backend/crawler_copy/jd_pc_search/detail/jd_h5st_detail_ware_business.js

@@ -0,0 +1,109 @@
+/**
+ * 商品详情 pc_detailpage_wareBusiness（appid=pc-item-soa）的 h5st。
+ * 与 jd_h5st.js（搜索）、jd_h5st_item_comment.js（评论）分离。
+ * ParamsSign 使用抓包第二段 appId：fb5df（与 getLegoWareDetailComment 一致）。
+ */
+require("../common/jd_browser_env.js");
+require("../common/code.js");
+const CryptoJS = require("crypto-js");
+
+const DETAIL_WARE_BUSINESS_SIGN_APP_ID = "fb5df";
+const DEFAULT_AREA = "19_1601_50258_129167";
+
+let _psign = null;
+function _ensurePsign() {
+  if (!_psign) {
+    _psign = new window.ParamsSign({
+      appId: DETAIL_WARE_BUSINESS_SIGN_APP_ID,
+      preRequest: false,
+      onSign: () => {},
+      onRequestTokenRemotely: () => {},
+    });
+  }
+  return _psign;
+}
+
+/**
+ * @param {object} opt
+ * @param {string|number} opt.skuId 商品 SKU（与 item.jd.com/{sku}.html 一致，body 里为字符串）
+ * @param {string} [opt.area]
+ * @param {string} [opt.num='1']
+ * @param {string} [opt.sfTime='1,0,0'] 与 PC 详情抓包一致
+ * @param {number} [opt.t]
+ */
+function get_h5st_detail_ware_business(opt) {
+  const o = opt || {};
+  const skuId = o.skuId != null ? String(o.skuId).trim() : "";
+  if (!skuId || !/^\d+$/.test(skuId)) {
+    throw new Error("get_h5st_detail_ware_business: 需要有效 opt.skuId（数字 SKU）");
+  }
+  const area = o.area != null ? String(o.area) : DEFAULT_AREA;
+  const num = o.num != null ? String(o.num) : "1";
+  const sfTime = o.sfTime != null ? String(o.sfTime) : "1,0,0";
+  const time = o.t != null ? Number(o.t) : Date.now();
+
+  const bodyObj = {
+    skuId,
+    area,
+    num,
+    sfTime,
+  };
+  const bodyJson = JSON.stringify(bodyObj);
+  const bodySha = CryptoJS.SHA256(bodyJson).toString();
+  const functionId = "pc_detailpage_wareBusiness";
+  const appid = "pc-item-soa";
+  const paramsH5sign = {
+    appid,
+    functionId,
+    client: "pc",
+    clientVersion: "1.0.0",
+    t: time,
+    body: bodySha,
+  };
+  const signed = _ensurePsign()._$sdnmd({ ...paramsH5sign });
+
+  return {
+    h5st: signed.h5st,
+    signed,
+    bodyJson,
+    bodySha256: signed.body,
+    bodyObj,
+    tQuerySecond: String(signed.t),
+  };
+}
+
+/**
+ * 拼 https://api.m.jd.com/?functionId=...（与 DevTools 路径一致，无 /api 前缀）。
+ * Query 键顺序与 Chrome 一致：functionId, body, h5st, uuid, loginType, appid,
+ * clientVersion, client, t, x-api-eid-token。
+ * body 为 JSON 字符串（非 SHA256），键顺序 skuId → area → num → sfTime，值均为字符串。
+ */
+function build_detail_ware_business_api_url(pack, opts) {
+  const uuid = opts.uuid != null ? String(opts.uuid) : "";
+  const xApiEidToken = opts.xApiEidToken != null ? String(opts.xApiEidToken) : "";
+  const bodyMode = opts.bodyMode === "sha256" ? "sha256" : "json";
+  const signed = pack.signed;
+  const bodyValue = bodyMode === "sha256" ? pack.bodySha256 : pack.bodyJson;
+  const qParts = [
+    ["functionId", signed.functionId],
+    ["body", bodyValue],
+    ["h5st", signed.h5st],
+    ["uuid", uuid],
+    ["loginType", "3"],
+    ["appid", signed.appid],
+    ["clientVersion", signed.clientVersion],
+    ["client", signed.client],
+    ["t", pack.tQuerySecond],
+    ["x-api-eid-token", xApiEidToken],
+  ];
+  const qs = qParts
+    .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`)
+    .join("&");
+  return `https://api.m.jd.com/?${qs}`;
+}
+
+module.exports = {
+  get_h5st_detail_ware_business,
+  build_detail_ware_business_api_url,
+  DEFAULT_AREA,
+};

backend/crawler_copy/jd_pc_search/detail/jd_pc_detail_ware_business_headers.js

@@ -0,0 +1,40 @@
+/**
+ * 与 Chrome 146 访问 item.jd.com → api.m.jd.com pc_detailpage_wareBusiness 的请求头对齐
+ *（含 Accept-Encoding: zstd，与 DevTools 抓包一致）。
+ *
+ * Playwright 的 APIRequestContext 对部分 sec-ch-* 会忽略 per-request headers；
+ * Python 侧在 ``browser.new_context({ userAgent, extraHTTPHeaders })`` 中重复注入同组
+ * Client Hints（见 jd_detail_ware_business_requests.py），与真实请求一致。
+ */
+function buildJdPcDetailWareBusinessHeaders(opts) {
+  const sku = opts.skuId != null ? String(opts.skuId).trim() : "";
+  const itemPage = sku
+    ? `https://item.jd.com/${encodeURIComponent(sku)}.html`
+    : "https://item.jd.com/";
+  const h = {
+    Accept: "application/json, text/plain, */*",
+    "Accept-Encoding": "gzip, deflate, br, zstd",
+    "Accept-Language": "zh-CN,zh;q=0.9",
+    "Cache-Control": "no-cache",
+    Pragma: "no-cache",
+    Priority: "u=1, i",
+    "Content-Type": "application/x-www-form-urlencoded",
+    "User-Agent":
+      "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
+    Referer: "https://item.jd.com/",
+    Origin: "https://item.jd.com",
+    "sec-ch-ua":
+      '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"',
+    "sec-ch-ua-mobile": "?0",
+    "sec-ch-ua-platform": '"Windows"',
+    "sec-fetch-dest": "empty",
+    "sec-fetch-mode": "cors",
+    "sec-fetch-site": "same-site",
+    "x-referer-page": itemPage,
+    "x-rp-client": "h5_1.0.0",
+  };
+  if (opts.cookie) h.Cookie = opts.cookie;
+  return h;
+}
+
+module.exports = { buildJdPcDetailWareBusinessHeaders };

backend/crawler_copy/jd_pc_search/jd_keyword_pipeline.py

@@ -0,0 +1,886 @@
+# -*- coding: utf-8 -*-
+"""
+关键词 → 京东 PC 搜索 → 对若干 SKU 拉取详情（pc_detailpage_wareBusiness）与评论（首屏 Lego，
+可选继续 ``getCommentListPage`` 分页，与同目录 ``jd_h5_item_comment_requests`` 一致），
+合并为一行 CSV（搜索列 + 详情摘要 + 评价摘要）。
+
+依赖：Node（搜索/评论签 h5st）、Playwright、本仓库 ``common/jd_cookie.txt``。
+
+用法：修改下方「运行配置」后，在项目任意目录执行::
+
+  python crawler/jd_pc_search/jd_keyword_pipeline.py
+
+或::
+
+  cd crawler/jd_pc_search && python jd_keyword_pipeline.py
+
+每次运行默认在 ``data/JD/pipeline_runs/<时间戳>_<关键词>/`` 下集中写入：合并表、
+PC 搜索导出 CSV、评价扁平 CSV、详情汇总 CSV（``detail_ware_export.csv``）、
+各 SKU 规整 JSON（``detail/ware_{sku}_response.json``），以及（可选）pc_search 原始包与请求记录。
+
+合并表 ``keyword_pipeline_merged.csv`` 默认 ``MERGED_CSV_MODE=lean``：搜索全列 + **竞品报告/入库实际用到的商详子集**（见 ``_MERGED_LEAN_DETAIL_FIELDNAMES``）+ 评论摘要；全量商详扁平请设 ``MERGED_CSV_MODE="full"``（``WARE_BUSINESS_MERGE_FIELDNAMES``）。
+``detail_ware_export.csv`` 默认 ``DETAIL_WARE_CSV_MODE=lean``，为 ``skuId`` + 与合并表一致的商详子集（品牌/到手价/店铺/类目/参数/配料）；全列请设 ``DETAIL_WARE_CSV_MODE="full"``。
+若 ``EXTRACT_INGREDIENTS_FROM_DETAIL_BODY_IMAGES=True``，配料列为**配料表文本**（``detail_body_ingredients_source_url`` 仅在内存/全量详情 CSV 使用，**不写** lean 合并表）；关闭视觉提取时合并表配料列为 **#detail-main 长图 URL 串**。
+默认启用 **应用场景筛选**（``brief_content.txt`` 4.1 中式面点/主食 + 4.2 烘焙）：仅命中关键词的 SKU 进入详情与评论队列；词表见 ``scenario_filter.py``。``SCENARIO_FILTER_ENABLED=False`` 可关闭；``SCENARIO_FILTER_PC_SEARCH_CSV="filtered"`` 可使导出 CSV 与筛选后列表一致。
+各 SKU 完整接口 JSON 仍在 ``detail/ware_{sku}_response.json``。
+
+端到端竞品速览 Markdown：配置 ``jd_competitor_report.py`` 顶部 ``KEYWORD`` 后执行 ``python jd_competitor_report.py``（内部调用本模块 ``main(keyword=...)``）。
+"""
+
+from __future__ import annotations
+
+import csv
+import json
+import random
+import sys
+import time
+from io import StringIO
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Any
+
+from playwright.sync_api import sync_playwright
+
+# ---------------------------------------------------------------------------
+# 路径与运行配置
+# ---------------------------------------------------------------------------
+_ROOT = Path(__file__).resolve().parent
+from _low_gi_root import low_gi_project_root  # noqa: E402
+
+_PROJECT_ROOT = low_gi_project_root()
+# 京东采集统一目录（与 search / detail / comment 脚本默认一致；副本依赖 LOW_GI_PROJECT_ROOT）
+_PROJECT_DATA = _PROJECT_ROOT / "data" / "JD"
+_COOKIE_FILE = str((_ROOT / "common" / "jd_cookie.txt").resolve())
+# 运行时覆盖（由 Market-Assistant 等在 main() 前设置）：非空则优先于 _COOKIE_FILE / 默认空 override
+PIPELINE_COOKIE_FILE = ""
+PIPELINE_COOKIE_OVERRIDE = ""
+
+# KEYWORD：搜索词（与 pc_search 一致）
+KEYWORD = "低GI"
+# PAGE_START / PAGE_TO：逻辑页范围（与 jd_search_playwright 含义相同）
+PAGE_START = 1
+PAGE_TO = 2
+# PVID：可选，与搜索结果页 URL 中 pvid 一致时填写
+PVID = ""
+# REQUEST_DELAY：pc_search 包间随机等待，如 "30-60"；None 关闭
+REQUEST_DELAY = "30-60"
+PAGE_DELAY_SEC = 1.2
+FETCH_RETRIES = 3
+FETCH_RETRY_DELAY_SEC = 3.0
+# PIPELINE_RUN_DIR：本次运行输出根目录。空则自动创建
+# ``data/JD/pipeline_runs/<YYYYMMDD_HHMMSS>_<关键词>/``；非空则用该路径（相对路径相对 data/JD）
+PIPELINE_RUN_DIR = ""
+# 工作台「终止任务」：可调用无参，返回 True 时在可停点结束并写出已采集部分（协作式，非杀进程）
+PIPELINE_CANCEL_CHECK = None  # Callable[[], bool] | None
+# 是否将 pc_search 原始响应 / 请求记录写入运行目录子文件夹（与 jd_search_playwright 一致）
+PIPELINE_SAVE_PC_SEARCH_RAW = True
+PIPELINE_SAVE_PC_SEARCH_RECORDS = True
+# 非空时覆盖上面两项，直接指定目录（与单独跑搜索脚本相同）
+SAVE_SEARCH_RAW_DIR = ""
+RECORD_SEARCH_REQ_DIR = ""
+
+# MAX_SKUS：搜索去重后，最多对多少个 SKU 继续拉详情+评论（控制总耗时）
+MAX_SKUS = 5
+# COMMENT_NUM：Lego 接口 body.commentNum（仅首屏条数；更多评价靠分页）
+COMMENT_NUM = 5
+SHOP_TYPE = "0"
+# WITH_COMMENT_LIST：首屏 Lego 成功后是否继续请求评价列表分页（POST client.action）
+WITH_COMMENT_LIST = True
+# LIST_PAGES：分页规格，如 "1"、"1-5"、"1,3,5"（与 jd_h5_item_comment_requests 相同）
+LIST_PAGES = "1-2"
+LIST_FUNCTION_ID = "getCommentListPage"
+LIST_STYLE = "1"
+# LIST_CATEGORY / LIST_FIRST_GUID：一般留空，从首屏 parsed 解析；抓包不一致时再填
+LIST_CATEGORY = ""
+LIST_FIRST_GUID = ""
+# COMMENT_LIST_DELAY：分页请求之间的随机等待；空字符串表示沿用 SKU_STEP_DELAY
+COMMENT_LIST_DELAY = ""
+# SKU_STEP_DELAY：每个 SKU 内「详情→首评」及步骤间随机等待（秒）
+SKU_STEP_DELAY = "4-10"
+# 详情：与 jd_detail_ware_business_requests 一致，结果为空或无效时重试
+DETAIL_FETCH_MAX_ATTEMPTS = 3
+DETAIL_FETCH_RETRY_DELAY_SEC = 2.0
+# USE_CHROME：True 使用本机 Chrome
+USE_CHROME = True
+HEADED = False
+
+# 运行目录内固定文件名（一般无需改）
+FILE_MERGED_CSV = "keyword_pipeline_merged.csv"
+FILE_PC_SEARCH_CSV = "pc_search_export.csv"
+FILE_COMMENTS_FLAT_CSV = "comments_flat.csv"
+FILE_DETAIL_WARE_CSV = "detail_ware_export.csv"
+FILE_RUN_META_JSON = "run_meta.json"
+# MERGED_CSV_MODE：``lean`` 时合并表为搜索全列 + 商详子集（``_MERGED_LEAN_DETAIL_FIELDNAMES``）+ 评论摘要；``full`` 为搜索全列 + ``WARE_BUSINESS_MERGE_FIELDNAMES`` 全量
+MERGED_CSV_MODE = "lean"
+# DETAIL_WARE_CSV_MODE：``lean`` 时 ``detail_ware_export.csv`` 为 ``skuId`` + lean 商详子集；``full`` 为完整详情扁平列（含 http_status 与各 detail_*）
+DETAIL_WARE_CSV_MODE = "lean"
+# 应用场景筛选（对齐 brief 4.1 中式面点/主食 + 4.2 烘焙）：仅命中关键词的商品进入详情/评论队列
+SCENARIO_FILTER_ENABLED = True
+# ``pc_search_export.csv``：``full`` 保留搜索全量；``filtered`` 仅写入命中场景的行（与详情样本一致）
+SCENARIO_FILTER_PC_SEARCH_CSV = "full"
+# 若启用筛选后无命中行，是否回退为未筛选列表（避免跑空）；False 则仍按空列表继续
+SCENARIO_FILTER_FALLBACK_TO_UNFILTERED = True
+# True：对 ``meta`` 中 ``detail_body_image_urls`` 从后往前调用 ``AI_crawler``，**首次**校验通过即写入配料；
+# 未命中时列内为 ``【未识别到配料】…`` 原因说明（非空串）。需 .env；未配置 API 时写入对应提示。关此开关时该列仍为长图 URL 串。
+EXTRACT_INGREDIENTS_FROM_DETAIL_BODY_IMAGES = True
+# ---------------------------------------------------------------------------
+
+# 保证可导入 search / comment / detail 下脚本与 common
+if str(_ROOT) not in sys.path:
+    sys.path.insert(0, str(_ROOT))
+_SEARCH_DIR = _ROOT / "search"
+_COMMENT_DIR = _ROOT / "comment"
+_DETAIL_DIR = _ROOT / "detail"
+for _p in (_SEARCH_DIR, _COMMENT_DIR, _DETAIL_DIR):
+    s = str(_p.resolve())
+    if s not in sys.path:
+        sys.path.insert(0, s)
+
+from collect_pc_search_items import (  # noqa: E402
+    SearchCollectionCancelled,
+    collect_pc_search_export_rows,
+)
+from common.jd_delay_utils import parse_request_delay_range  # noqa: E402
+from scenario_filter import filter_rows_by_scenario  # noqa: E402
+from jd_detail_ware_business_requests import (  # noqa: E402
+    DETAIL_WARE_LEAN_CSV_FIELDNAMES,
+    WARE_BUSINESS_MERGE_FIELDNAMES,
+    WARE_PARSED_CSV_FIELDNAMES,
+    _JD_DETAIL_CONTEXT_EXTRA_HEADERS,
+    _JD_DETAIL_UA,
+    detail_ware_lean_csv_row,
+    fetch_ware_business,
+    format_ware_response_for_save,
+    parse_ware_business_response_text,
+    ware_parsed_row,
+)
+from jd_h5_item_comment_requests import (  # noqa: E402
+    category_and_first_guid_from_lego,
+    export_item_comment_page_request_json,
+    export_item_comment_request_json,
+    extract_comment_rows_from_parsed,
+    parse_list_pages_spec,
+    write_comments_flat_csv,
+)
+from jd_h5_search_requests import (  # noqa: E402
+    CSV_FIELDS,
+    JD_EXPORT_COLUMN_HEADERS,
+    jd_row_to_export,
+)
+
+
+_SKU_CSV_HEADER = JD_EXPORT_COLUMN_HEADERS["sku_id"]
+
+_MERGED_EXTRA_FIELDS = (
+    ["pipeline_keyword"]
+    + list(WARE_BUSINESS_MERGE_FIELDNAMES)
+    + ["comment_count", "comment_preview"]
+)
+
+# lean 合并表·商详块（jd_competitor_report + ingest + 配料）；须与 pipeline/csv_schema.MERGED_LEAN_DETAIL_KEYS 一致
+_MERGED_LEAN_DETAIL_FIELDNAMES: tuple[str, ...] = (
+    "detail_brand",
+    "detail_price_final",
+    "detail_shop_name",
+    "detail_category_path",
+    "detail_product_attributes",
+    "detail_body_ingredients",
+)
+
+# 合并表精简列：搜索列与 jd_h5_search_requests 一致 + 上表商详子集 + 评论摘要
+_MERGED_LEAN_FIELDNAMES: tuple[str, ...] = (
+    "pipeline_keyword",
+    "SKU(skuId)",
+    "主商品ID(wareId)",
+    "标题(wareName)",
+    "标价(jdPrice,jdPriceText,realPrice)",
+    "券后到手价(couponPrice,subsidyPrice,finalPrice.estimatedPrice,priceShow)",
+    "原价(oriPrice,originalPrice,marketPrice)",
+    "卖点(sellingPoint)",
+    "榜单类文案(标签/腰带/标题数组中的榜、TOP 等)",
+    "评价量(commentFuzzy)",
+    "销量楼层(commentSalesFloor)",
+    "店铺名(shopName)",
+    "商品链接(toUrl,clickUrl,item.m.jd.com)",
+    "主图(imageurl,imageUrl)",
+    "规格属性(propertyList,color,catid,shortName)",
+    "类目(leafCategory,cid3Name,catid)",
+    "搜索词(keyword)",
+    "页码(page)",
+    *_MERGED_LEAN_DETAIL_FIELDNAMES,
+    "comment_count",
+    "comment_preview",
+)
+
+
+def _merged_csv_fieldnames() -> list[str]:
+    if (MERGED_CSV_MODE or "lean").strip().lower() == "full":
+        return list(CSV_FIELDS) + [
+            f for f in _MERGED_EXTRA_FIELDS if f not in CSV_FIELDS
+        ]
+    return list(_MERGED_LEAN_FIELDNAMES)
+
+
+def _detail_ware_csv_fieldnames() -> list[str]:
+    if (DETAIL_WARE_CSV_MODE or "lean").strip().lower() == "full":
+        return list(WARE_PARSED_CSV_FIELDNAMES)
+    return list(DETAIL_WARE_LEAN_CSV_FIELDNAMES)
+
+
+def _sleep_range(spec: str, label: str) -> None:
+    try:
+        lo, hi = parse_request_delay_range(spec)
+    except ValueError:
+        return
+    if hi <= 0 and lo <= 0:
+        return
+    t = random.uniform(lo, hi)
+    print(f"[流水线] {label} 等待 {t:.1f}s", file=sys.stderr)
+    time.sleep(t)
+
+
+def _dedupe_comment_rows(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    """按 commentId 去重（跨首屏 + 多页列表）。"""
+    seen: set[str] = set()
+    out: list[dict[str, Any]] = []
+    for r in rows:
+        cid = str(r.get("commentId") or "").strip()
+        if cid:
+            if cid in seen:
+                continue
+            seen.add(cid)
+        out.append(r)
+    return out
+
+
+def _comment_fields_from_rows(rows: list[dict[str, Any]]) -> dict[str, str]:
+    previews: list[str] = []
+    for r in rows[:8]:
+        t = str(r.get("tagCommentContent") or "").strip()
+        if t:
+            previews.append(t[:400])
+    joined = " | ".join(previews)[:4000]
+    return {
+        "comment_count": str(len(rows)),
+        "comment_preview": joined,
+    }
+
+
+def _loads_json(text: str) -> Any:
+    try:
+        return json.loads(text)
+    except json.JSONDecodeError:
+        return None
+
+
+def _safe_dir_segment(s: str, max_len: int = 48) -> str:
+    bad = '<>:"/\\|?*\n\r\t'
+    t = "".join("_" if c in bad else c for c in (s or "").strip())[:max_len]
+    t = t.strip(" .") or "run"
+    return t
+
+
+def _resolve_pipeline_run_dir(kw: str) -> Path:
+    raw = (PIPELINE_RUN_DIR or "").strip()
+    if raw:
+        p = Path(raw).expanduser()
+        if not p.is_absolute():
+            p = _PROJECT_DATA / p
+        return p.resolve()
+    stamp = time.strftime("%Y%m%d_%H%M%S")
+    seg = _safe_dir_segment(kw)
+    return (_PROJECT_DATA / "pipeline_runs" / f"{stamp}_{seg}").resolve()
+
+
+class PipelineCancelled(Exception):
+    """工作台请求终止本次流水线；携带已分配的运行目录（可有部分产出）。"""
+
+    def __init__(self, run_dir: Path) -> None:
+        self.run_dir = run_dir.resolve()
+        super().__init__("pipeline cancelled")
+
+
+def _pipeline_cancel_requested() -> bool:
+    fn = PIPELINE_CANCEL_CHECK
+    try:
+        return fn is not None and callable(fn) and bool(fn())
+    except Exception:
+        return False
+
+
+def main(keyword: str | None = None) -> Path:
+    """
+    跑完整条流水线。``keyword`` 非空时覆盖文件内 ``KEYWORD``；返回本次运行目录。
+
+    供 ``jd_competitor_report`` 等脚本 ``import`` 调用；命令行仍执行 ``main()`` 无参。
+    """
+    try:
+        if hasattr(sys.stdout, "reconfigure"):
+            sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+        if hasattr(sys.stderr, "reconfigure"):
+            sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+    except Exception:
+        pass
+
+    kw = (keyword if keyword is not None else KEYWORD) or ""
+    kw = str(kw).strip()
+    if not kw:
+        print("[流水线] 请配置 KEYWORD 或传入 keyword 参数", file=sys.stderr)
+        sys.exit(2)
+
+    page_start = max(1, int(PAGE_START))
+    page_to = PAGE_TO if PAGE_TO is not None else page_start
+    page_to = max(page_start, int(page_to))
+
+    req_delay_range: tuple[float, float] | None = None
+    if REQUEST_DELAY:
+        try:
+            req_delay_range = parse_request_delay_range(str(REQUEST_DELAY).strip())
+        except ValueError as e:
+            print(f"[流水线] REQUEST_DELAY 无效: {e}", file=sys.stderr)
+            sys.exit(2)
+
+    run_dir = _resolve_pipeline_run_dir(kw)
+    run_dir.mkdir(parents=True, exist_ok=True)
+    stop_pipeline = False
+    print(f"[流水线] 本次输出目录: {run_dir}", file=sys.stderr)
+
+    if (SAVE_SEARCH_RAW_DIR or "").strip():
+        save_js = Path(SAVE_SEARCH_RAW_DIR).expanduser().resolve()
+    elif PIPELINE_SAVE_PC_SEARCH_RAW:
+        save_js = run_dir / "pc_search_raw"
+    else:
+        save_js = None
+
+    if (RECORD_SEARCH_REQ_DIR or "").strip():
+        record_req = Path(RECORD_SEARCH_REQ_DIR).expanduser().resolve()
+    elif PIPELINE_SAVE_PC_SEARCH_RECORDS:
+        record_req = run_dir / "pc_search_requests"
+    else:
+        record_req = None
+
+    node_pvid = (PVID or "").strip() or None
+
+    search_args = SimpleNamespace(
+        q=kw,
+        page_delay=float(PAGE_DELAY_SEC),
+        fetch_retries=int(FETCH_RETRIES),
+        fetch_retry_delay=float(FETCH_RETRY_DELAY_SEC),
+        pretty_raw_json=True,
+        csv=True,
+        out="",
+    )
+
+    print(f"[流水线] 搜索词={kw!r} 逻辑页 {page_start}–{page_to}", file=sys.stderr)
+
+    merged_rows: list[dict[str, str]] = []
+    all_comment_rows: list[dict[str, Any]] = []
+    detail_csv_rows: list[dict[str, str]] = []
+    launch_kw: dict[str, Any] = {"headless": not HEADED}
+    if USE_CHROME:
+        launch_kw["channel"] = "chrome"
+
+    _ac_mod: Any = None
+    _ingredient_vision_ok = False
+    if EXTRACT_INGREDIENTS_FROM_DETAIL_BODY_IMAGES:
+        try:
+            import AI_crawler as _ac_mod  # noqa: WPS433
+
+            _ac_mod._resolve_credentials(None, None, None)
+            _ingredient_vision_ok = True
+        except Exception as e:
+            print(
+                f"[流水线] 已开启配料视觉提取但未就绪（{e}），"
+                f"各 SKU 列 detail_body_ingredients 将写入「未配置 API」类原因说明（非 URL）",
+                file=sys.stderr,
+            )
+
+    _pcf = (PIPELINE_COOKIE_FILE or "").strip()
+    if _pcf:
+        _pcfp = Path(_pcf).expanduser().resolve()
+        cookie_path = str(_pcfp) if _pcfp.is_file() else None
+        if cookie_path is None:
+            print(
+                f"[流水线] 警告：PIPELINE_COOKIE_FILE 不是有效文件，将回退 common/jd_cookie.txt："
+                f"{_pcf!r}",
+                file=sys.stderr,
+            )
+    else:
+        cookie_path = None
+    if cookie_path is None:
+        cookie_path = _COOKIE_FILE if Path(_COOKIE_FILE).is_file() else None
+    _cookie_override = (PIPELINE_COOKIE_OVERRIDE or "").strip()
+
+    with sync_playwright() as pw:
+        browser = pw.chromium.launch(**launch_kw)
+        ctx = browser.new_context()
+        try:
+            export_rows_full = collect_pc_search_export_rows(
+                ctx,
+                search_args,
+                page_start=page_start,
+                pe=page_to,
+                req_delay_range=req_delay_range,
+                save_js_dir=save_js,
+                record_req_dir=record_req,
+                node_pvid=node_pvid,
+                cancel_check=_pipeline_cancel_requested,
+                node_cookie_file=cookie_path,
+            )
+        except SearchCollectionCancelled as e:
+            export_rows_full = [jd_row_to_export(r) for r in e.partial_rows]
+            stop_pipeline = True
+            print(
+                "[流水线] 已按请求在下一请求前终止 PC 搜索（保留已得行）",
+                file=sys.stderr,
+            )
+        if _pipeline_cancel_requested():
+            stop_pipeline = True
+
+        scenario_filter_on = bool(SCENARIO_FILTER_ENABLED)
+        scenario_stats: dict[str, Any] | None = None
+        export_rows_for_skus: list[dict[str, str]] = list(export_rows_full)
+        if scenario_filter_on:
+            fr, scenario_stats = filter_rows_by_scenario(export_rows_full)
+            export_rows_for_skus = fr
+            print(
+                f"[流水线] 应用场景筛选：全量 {scenario_stats['input_rows']} 行 → "
+                f"保留 {scenario_stats['kept_rows']} 行（剔除 {scenario_stats['dropped_rows']}），"
+                f"标签分布 {scenario_stats.get('tag_counts')!r}",
+                file=sys.stderr,
+            )
+            if not export_rows_for_skus and SCENARIO_FILTER_FALLBACK_TO_UNFILTERED:
+                print(
+                    "[流水线] 筛选后无命中行，已按 SCENARIO_FILTER_FALLBACK_TO_UNFILTERED "
+                    "回退为未筛选列表",
+                    file=sys.stderr,
+                )
+                export_rows_for_skus = list(export_rows_full)
+                scenario_stats = {
+                    **(scenario_stats or {}),
+                    "fallback_unfiltered": True,
+                }
+
+        csv_mode = (SCENARIO_FILTER_PC_SEARCH_CSV or "full").strip().lower()
+        if csv_mode == "filtered":
+            rows_for_search_csv = (
+                export_rows_for_skus
+                if scenario_filter_on
+                else list(export_rows_full)
+            )
+        else:
+            rows_for_search_csv = list(export_rows_full)
+
+        skus_ordered: list[str] = []
+        seen: set[str] = set()
+        for row in export_rows_for_skus:
+            sid = str(row.get(_SKU_CSV_HEADER) or "").strip()
+            if not sid or sid in seen:
+                continue
+            seen.add(sid)
+            skus_ordered.append(sid)
+            if len(skus_ordered) >= max(1, int(MAX_SKUS)):
+                break
+
+        print(
+            f"[流水线] 搜索导出 {len(export_rows_full)} 行（写入 CSV {len(rows_for_search_csv)} 行），"
+            f"取前 {len(skus_ordered)} 个 SKU 拉详情+评论",
+            file=sys.stderr,
+        )
+        if _pipeline_cancel_requested():
+            stop_pipeline = True
+
+        search_csv_path = run_dir / FILE_PC_SEARCH_CSV
+        sbuf = StringIO()
+        sw = csv.DictWriter(
+            sbuf, fieldnames=list(CSV_FIELDS), extrasaction="ignore"
+        )
+        sw.writeheader()
+        sw.writerows(rows_for_search_csv)
+        search_csv_path.write_text("\ufeff" + sbuf.getvalue(), encoding="utf-8")
+        print(
+            f"[流水线] 已写 PC 搜索导出 {search_csv_path}",
+            file=sys.stderr,
+        )
+
+        detail_dir = run_dir / "detail"
+        detail_dir.mkdir(parents=True, exist_ok=True)
+
+        detail_ctx = browser.new_context(
+            user_agent=_JD_DETAIL_UA,
+            locale="zh-CN",
+            timezone_id="Asia/Shanghai",
+            extra_http_headers=dict(_JD_DETAIL_CONTEXT_EXTRA_HEADERS),
+        )
+        page = detail_ctx.new_page()
+
+        for idx, sku in enumerate(skus_ordered):
+            if stop_pipeline or _pipeline_cancel_requested():
+                stop_pipeline = True
+                break
+            if idx > 0:
+                if stop_pipeline or _pipeline_cancel_requested():
+                    stop_pipeline = True
+                    break
+                _sleep_range(SKU_STEP_DELAY, "SKU 间隔")
+
+            search_row = next(
+                (
+                    r
+                    for r in export_rows_full
+                    if str(r.get(_SKU_CSV_HEADER) or "").strip() == sku
+                ),
+                {},
+            )
+            merged: dict[str, str] = {k: str(search_row.get(k) or "") for k in CSV_FIELDS}
+            merged["pipeline_keyword"] = kw
+
+            if stop_pipeline or _pipeline_cancel_requested():
+                stop_pipeline = True
+                break
+
+            d_code, d_text, d_meta = fetch_ware_business(
+                detail_ctx,
+                page,
+                sku,
+                cookie_file=cookie_path,
+                timeout_ms=45_000,
+                cookie_override=_cookie_override,
+                max_attempts=int(DETAIL_FETCH_MAX_ATTEMPTS),
+                retry_delay_sec=float(DETAIL_FETCH_RETRY_DELAY_SEC),
+                cancel_check=_pipeline_cancel_requested,
+            )
+            body_for_parse = d_text if d_code == 200 else ""
+            ware_flat, _wok = parse_ware_business_response_text(body_for_parse)
+            merged.update(ware_flat)
+            raw_body_urls = str(d_meta.get("detail_body_image_urls") or "").strip()
+            _skip_vision_and_later_network = (
+                stop_pipeline or _pipeline_cancel_requested()
+            )
+            if _skip_vision_and_later_network:
+                stop_pipeline = True
+            if (
+                EXTRACT_INGREDIENTS_FROM_DETAIL_BODY_IMAGES
+                and _ingredient_vision_ok
+                and not _skip_vision_and_later_network
+            ):
+                try:
+                    _ex_src = getattr(
+                        _ac_mod,
+                        "extract_ingredients_from_body_image_urls_reversed_with_source",
+                        None,
+                    )
+                    if callable(_ex_src):
+                        di, src_u = _ex_src(raw_body_urls)
+                        merged["detail_body_ingredients"] = di
+                        merged["detail_body_ingredients_source_url"] = (
+                            str(src_u).strip() if src_u else ""
+                        )
+                    else:
+                        merged["detail_body_ingredients"] = (
+                            _ac_mod.extract_ingredients_from_body_image_urls_reversed(
+                                raw_body_urls
+                            )
+                        )
+                        merged["detail_body_ingredients_source_url"] = ""
+                    di = merged["detail_body_ingredients"]
+                    if str(di).startswith("【未识别"):
+                        print(f"[流水线] sku={sku} {di}", file=sys.stderr)
+                    else:
+                        su = str(
+                            merged.get("detail_body_ingredients_source_url") or ""
+                        ).strip()
+                        if su:
+                            print(
+                                f"[流水线] sku={sku} 已从详情长图解析配料表，图源: {su}",
+                                file=sys.stderr,
+                            )
+                        else:
+                            print(
+                                f"[流水线] sku={sku} 已从详情长图（自后向前，首次命中）解析配料表",
+                                file=sys.stderr,
+                            )
+                except Exception as e:
+                    print(
+                        f"[流水线] sku={sku} 配料视觉提取异常: {e}",
+                        file=sys.stderr,
+                    )
+                    merged["detail_body_ingredients"] = (
+                        f"【未识别到配料】识别过程异常：{e}"[:800]
+                    )
+                    merged["detail_body_ingredients_source_url"] = ""
+            elif EXTRACT_INGREDIENTS_FROM_DETAIL_BODY_IMAGES and not _ingredient_vision_ok:
+                merged["detail_body_ingredients"] = (
+                    "【未识别到配料】未配置或无效的多模态 API，已跳过识别。"
+                )
+                merged["detail_body_ingredients_source_url"] = ""
+            else:
+                merged["detail_body_ingredients"] = raw_body_urls
+                merged["detail_body_ingredients_source_url"] = ""
+
+            response_body = format_ware_response_for_save(
+                d_text or "",
+                normalize=True,
+                sort_keys=True,
+                indent=2,
+            )
+            (detail_dir / f"ware_{sku}_response.json").write_text(
+                response_body, encoding="utf-8"
+            )
+            _d_ing = str(merged.get("detail_body_ingredients") or "").strip()
+            _d_src = str(
+                merged.get("detail_body_ingredients_source_url") or ""
+            ).strip()
+            if (DETAIL_WARE_CSV_MODE or "lean").strip().lower() == "full":
+                detail_csv_rows.append(
+                    ware_parsed_row(
+                        sku,
+                        d_code,
+                        d_text or "",
+                        detail_body_ingredients=_d_ing,
+                        detail_body_ingredients_source_url=_d_src,
+                    )
+                )
+            else:
+                detail_csv_rows.append(
+                    detail_ware_lean_csv_row(
+                        sku,
+                        d_code,
+                        d_text or "",
+                        detail_body_ingredients=_d_ing,
+                        detail_body_ingredients_source_url=_d_src,
+                    )
+                )
+
+            if stop_pipeline or _pipeline_cancel_requested():
+                stop_pipeline = True
+                merged["comment_count"] = "0"
+                merged["comment_preview"] = ""
+                merged_rows.append(merged)
+                break
+
+            _sleep_range(SKU_STEP_DELAY, "详情→评论")
+
+            if stop_pipeline or _pipeline_cancel_requested():
+                stop_pipeline = True
+                merged["comment_count"] = "0"
+                merged["comment_preview"] = ""
+                merged_rows.append(merged)
+                break
+
+            try:
+                pack = export_item_comment_request_json(
+                    sku,
+                    comment_num=int(COMMENT_NUM),
+                    shop_type=str(SHOP_TYPE),
+                    cookie_file=cookie_path,
+                )
+            except SystemExit:
+                merged["comment_count"] = "0"
+                merged["comment_preview"] = ""
+                merged_rows.append(merged)
+                continue
+
+            if stop_pipeline or _pipeline_cancel_requested():
+                stop_pipeline = True
+                merged["comment_count"] = "0"
+                merged["comment_preview"] = ""
+                merged_rows.append(merged)
+                break
+
+            try:
+                url = pack["url"]
+                hdrs = {str(k): str(v) for k, v in pack["headers"].items()}
+                resp = ctx.request.get(url, headers=hdrs, timeout=45_000)
+                c_st = resp.status
+                c_text = resp.text()
+                parsed = _loads_json(c_text)
+                comment_rows: list[dict[str, Any]] = []
+                if isinstance(parsed, dict):
+                    comment_rows.extend(
+                        extract_comment_rows_from_parsed(sku, parsed)
+                    )
+                list_delay = (
+                    (COMMENT_LIST_DELAY or "").strip() or SKU_STEP_DELAY
+                )
+                if (
+                    WITH_COMMENT_LIST
+                    and isinstance(parsed, dict)
+                    and 200 <= c_st < 300
+                ):
+                    cat, fguid = category_and_first_guid_from_lego(parsed)
+                    if (LIST_CATEGORY or "").strip():
+                        cat = LIST_CATEGORY.strip()
+                    if (LIST_FIRST_GUID or "").strip():
+                        fguid = LIST_FIRST_GUID.strip()
+                    if cat and fguid:
+                        pages = parse_list_pages_spec(LIST_PAGES or "1")
+                        lfid = (LIST_FUNCTION_ID or "getCommentListPage").strip()
+                        lstyle = (LIST_STYLE or "1").strip()
+                        for pi, pnum in enumerate(pages):
+                            if stop_pipeline or _pipeline_cancel_requested():
+                                stop_pipeline = True
+                                break
+                            _sleep_range(list_delay, "评论列表分页")
+                            if stop_pipeline or _pipeline_cancel_requested():
+                                stop_pipeline = True
+                                break
+                            try:
+                                pack_p = export_item_comment_page_request_json(
+                                    sku,
+                                    category=cat,
+                                    first_guid=fguid,
+                                    page_num=pnum,
+                                    is_first=(pi == 0),
+                                    function_id=lfid,
+                                    shop_type=str(SHOP_TYPE),
+                                    cookie_file=cookie_path,
+                                    style=lstyle,
+                                )
+                            except SystemExit:
+                                print(
+                                    "[流水线] 评论分页 Node 失败，已停止该 SKU 后续分页",
+                                    file=sys.stderr,
+                                )
+                                break
+                            if stop_pipeline or _pipeline_cancel_requested():
+                                stop_pipeline = True
+                                break
+                            url_p = pack_p["url"]
+                            hdrs_p = {
+                                str(k): str(v)
+                                for k, v in pack_p["headers"].items()
+                            }
+                            form_p = {
+                                str(k): str(v)
+                                for k, v in (pack_p.get("form") or {}).items()
+                            }
+                            try:
+                                resp_p = ctx.request.post(
+                                    url_p,
+                                    headers=hdrs_p,
+                                    form=form_p,
+                                    timeout=45_000,
+                                )
+                                st_p = resp_p.status
+                                text_p = resp_p.text()
+                            except Exception as e:
+                                print(
+                                    f"[流水线] 评论分页 POST 异常: {e}",
+                                    file=sys.stderr,
+                                )
+                                break
+                            parsed_p = _loads_json(text_p)
+                            if isinstance(parsed_p, dict):
+                                comment_rows.extend(
+                                    extract_comment_rows_from_parsed(
+                                        sku, parsed_p
+                                    )
+                                )
+                    else:
+                        print(
+                            "[流水线] WITH_COMMENT_LIST 已开但缺少 category 或 "
+                            "firstCommentGuid，仅保留首屏评价",
+                            file=sys.stderr,
+                        )
+                comment_rows = _dedupe_comment_rows(comment_rows)
+                merged.update(_comment_fields_from_rows(comment_rows))
+                all_comment_rows.extend(comment_rows)
+            except Exception as e:
+                print(
+                    f"[流水线] sku={sku} 评论请求异常: {e}",
+                    file=sys.stderr,
+                )
+                merged["comment_count"] = "0"
+                merged["comment_preview"] = ""
+
+            merged_rows.append(merged)
+            print(f"[流水线] [{idx + 1}/{len(skus_ordered)}] sku={sku} OK", file=sys.stderr)
+            if stop_pipeline:
+                break
+
+        try:
+            page.close()
+        except Exception:
+            pass
+        try:
+            detail_ctx.close()
+        except Exception:
+            pass
+        browser.close()
+
+    out_path = run_dir / FILE_MERGED_CSV
+    fieldnames = _merged_csv_fieldnames()
+    buf = StringIO()
+    w = csv.DictWriter(buf, fieldnames=fieldnames, extrasaction="ignore")
+    w.writeheader()
+    w.writerows(merged_rows)
+    out_path.write_text("\ufeff" + buf.getvalue(), encoding="utf-8")
+    print(
+        f"[流水线] 已写合并表 {out_path} 共 {len(merged_rows)} 行 "
+        f"（MERGED_CSV_MODE={MERGED_CSV_MODE!r}，{len(fieldnames)} 列）",
+        file=sys.stderr,
+    )
+
+    detail_csv_path = run_dir / FILE_DETAIL_WARE_CSV
+    detail_csv_path.parent.mkdir(parents=True, exist_ok=True)
+    detail_fn = _detail_ware_csv_fieldnames()
+    with detail_csv_path.open("w", encoding="utf-8-sig", newline="") as dcf:
+        dw = csv.DictWriter(
+            dcf,
+            fieldnames=detail_fn,
+            extrasaction="ignore",
+        )
+        dw.writeheader()
+        dw.writerows(detail_csv_rows)
+    print(
+        f"[流水线] 已写详情扁平表 {detail_csv_path} 共 {len(detail_csv_rows)} 行 "
+        f"（DETAIL_WARE_CSV_MODE={DETAIL_WARE_CSV_MODE!r}，{len(detail_fn)} 列）",
+        file=sys.stderr,
+    )
+
+    comments_path = run_dir / FILE_COMMENTS_FLAT_CSV
+    write_comments_flat_csv(comments_path, all_comment_rows)
+    print(
+        f"[流水线] 已写评价扁平表 {comments_path} 共 {len(all_comment_rows)} 条",
+        file=sys.stderr,
+    )
+
+    meta = {
+        "keyword": kw,
+        "page_start": page_start,
+        "page_to": page_to,
+        "max_skus_config": int(MAX_SKUS),
+        "extract_ingredients_from_detail_body_images": bool(
+            EXTRACT_INGREDIENTS_FROM_DETAIL_BODY_IMAGES
+        ),
+        "ingredient_vision_api_ready": bool(_ingredient_vision_ok),
+        "scenario_filter_enabled": bool(SCENARIO_FILTER_ENABLED),
+        "scenario_filter_pc_search_csv": (SCENARIO_FILTER_PC_SEARCH_CSV or "full")
+        .strip()
+        .lower(),
+        "scenario_filter_stats": scenario_stats,
+        "pc_search_export_rows": len(rows_for_search_csv),
+        "pc_search_export_rows_full": len(export_rows_full),
+        "merged_rows": len(merged_rows),
+        "merged_csv_mode": (MERGED_CSV_MODE or "lean").strip().lower(),
+        "merged_csv_column_count": len(fieldnames),
+        "detail_ware_csv_mode": (DETAIL_WARE_CSV_MODE or "lean").strip().lower(),
+        "detail_ware_csv_column_count": len(detail_fn),
+        "comment_flat_rows": len(all_comment_rows),
+        "detail_ware_csv_rows": len(detail_csv_rows),
+        "with_comment_list": bool(WITH_COMMENT_LIST),
+        "list_pages": (LIST_PAGES or "").strip(),
+    }
+    (run_dir / FILE_RUN_META_JSON).write_text(
+        json.dumps(meta, ensure_ascii=False, indent=2) + "\n",
+        encoding="utf-8",
+    )
+    if stop_pipeline:
+        print("[流水线] 已按请求终止（已写出当前进度）", file=sys.stderr)
+        raise PipelineCancelled(run_dir)
+    return run_dir
+
+
+if __name__ == "__main__":
+    main()

backend/crawler_copy/jd_pc_search/package-lock.json

@@ -0,0 +1,18 @@
+{
+  "name": "jd_pc_search",
+  "lockfileVersion": 3,
+  "requires": true,
+  "packages": {
+    "": {
+      "dependencies": {
+        "crypto-js": "^4.2.0"
+      }
+    },
+    "node_modules/crypto-js": {
+      "version": "4.2.0",
+      "resolved": "https://registry.npmjs.org/crypto-js/-/crypto-js-4.2.0.tgz",
+      "integrity": "sha512-KALDyEYgpY+Rlob/iriUtjV6d5Eq+Y191A5g4UqLAi8CyGP9N1+FdVbkc1SxKc2r4YAYqG8JzO2KGL+AizD70Q==",
+      "license": "MIT"
+    }
+  }
+}

backend/crawler_copy/jd_pc_search/package.json

@@ -0,0 +1,5 @@
+{
+  "dependencies": {
+    "crypto-js": "^4.2.0"
+  }
+}

backend/crawler_copy/jd_pc_search/scenario_filter.py

@@ -0,0 +1,152 @@
+# -*- coding: utf-8 -*-
+"""
+根据 ``brief_content.txt`` 4.1 / 4.2 应用场景，对京东搜索导出行做关键词保留筛选。
+
+- **保留**：标题/卖点/类目/规格中至少命中一类场景词（中式面点主食或烘焙）。
+- **剔除**：未命中任一类（如饮料、保健品、与面点/烘焙无关的零食等）。
+
+说明：基于展示文案的规则匹配，边界案例需人工复核；可随时扩充 ``_KW_41`` / ``_KW_42``。
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+# 与 CSV 导出列名一致（jd_h5_search_requests.CSV_FIELDS 子集）
+_SCENARIO_TEXT_FIELDS: tuple[str, ...] = (
+    "标题(wareName)",
+    "卖点(sellingPoint)",
+    "类目(leafCategory,cid3Name,catid)",
+    "规格属性(propertyList,color,catid,shortName)",
+)
+
+# 4.1 中式（米）面点及主食（含常见同义/细分）
+_KW_41: tuple[str, ...] = (
+    "包子",
+    "馒头",
+    "花卷",
+    "饺子",
+    "饺皮",
+    "饺子皮",
+    "水饺",
+    "蒸饺",
+    "锅贴",
+    "馄饨",
+    "云吞",
+    "抄手",
+    "面条",
+    "挂面",
+    "拉面",
+    "刀削面",
+    "凉面",
+    "冷面",
+    "热干面",
+    "意面",
+    "方便面",
+    "泡面",
+    "速食面",
+    "米粉",
+    "米线",
+    "河粉",
+    "粉丝",
+    "螺蛳粉",
+    "米糕",
+    "年糕",
+    "糍粑",
+    "发糕",
+    "重组米",
+    "重组大米",
+    "大米",
+    "米饭",
+    "杂粮饭",
+    "白米饭",
+    "自热米饭",
+    "煲仔饭",
+    "烧麦",
+    "烧卖",
+    "面皮",
+    "春卷",
+    "手抓饼",
+    "葱油饼",
+    "馅饼",
+    "烧饼",
+    "油条",
+    "窝头",
+    "窝窝头",
+    "荞麦面",
+    "青稞",
+    "全麦面条",
+)
+
+# 4.2 烘焙（含与 brief 一致的低温慢烤饼干等）
+_KW_42: tuple[str, ...] = (
+    "面包",
+    "吐司",
+    "列巴",
+    "欧包",
+    "贝果",
+    "可颂",
+    "牛角",
+    "蛋糕",
+    "糕点",
+    "饼干",
+    "曲奇",
+    "烘焙",
+    "酥饼",
+    "桃酥",
+    "威化",
+    "华夫",
+    "司康",
+    "蛋挞",
+    "月饼",
+    "酥性",
+    "苏打饼干",
+    "全麦面包",
+    "手撕面包",
+)
+
+
+def scenario_row_text(row: dict[str, Any]) -> str:
+    parts: list[str] = []
+    for k in _SCENARIO_TEXT_FIELDS:
+        parts.append(str(row.get(k) or ""))
+    return "\n".join(parts)
+
+
+def row_scenario_match(row: dict[str, Any]) -> tuple[bool, str]:
+    """
+    是否命中应用场景；第二个返回值为标签 ``4.1`` / ``4.2`` / ``4.1+4.2`` / ````。
+    """
+    text = scenario_row_text(row)
+    hit41 = any(kw in text for kw in _KW_41)
+    hit42 = any(kw in text for kw in _KW_42)
+    if hit41 and hit42:
+        return True, "4.1+4.2"
+    if hit41:
+        return True, "4.1"
+    if hit42:
+        return True, "4.2"
+    return False, ""
+
+
+def filter_rows_by_scenario(
+    rows: list[dict[str, Any]],
+) -> tuple[list[dict[str, Any]], dict[str, Any]]:
+    """
+    保持原顺序，仅保留命中 4.1 或 4.2 的行。
+    返回 (filtered_rows, stats)。
+    """
+    kept: list[dict[str, Any]] = []
+    tag_counts: dict[str, int] = {}
+    for r in rows:
+        ok, tag = row_scenario_match(r)
+        if ok:
+            kept.append(r)
+            tag_counts[tag] = tag_counts.get(tag, 0) + 1
+    stats: dict[str, Any] = {
+        "input_rows": len(rows),
+        "kept_rows": len(kept),
+        "dropped_rows": len(rows) - len(kept),
+        "tag_counts": tag_counts,
+    }
+    return kept, stats

backend/crawler_copy/jd_pc_search/search/collect_pc_search_items.py

@@ -0,0 +1,487 @@
+# -*- coding: utf-8 -*-
+"""
+pc_search 多逻辑页采集（供 ``jd_search_playwright`` 与 ``jd_keyword_pipeline`` 共用）。
+"""
+
+from __future__ import annotations
+
+import json
+import re
+import sys
+import time
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Any, Callable
+from urllib.parse import parse_qs, unquote, urlparse
+
+class SearchCollectionCancelled(Exception):
+    """工作台请求终止：携带已解析、尚未 jd_row_to_export 的累计行。"""
+
+    def __init__(self, partial_rows: list[dict[str, str]]) -> None:
+        self.partial_rows = partial_rows
+        super().__init__("search collection cancelled")
+
+
+from jd_h5_search_requests import (
+    JD_PC_SEARCH_CHUNKS_PER_LOGICAL_PAGE,
+    JD_PC_SEARCH_FALLBACK_S_STEP,
+    JD_PC_SEARCH_ITEMS_PER_PAGE,
+    _detect_blocked,
+    _jd_row_count_for_page,
+    export_pc_search_request_json,
+    jd_pc_api_body_page_first_pack,
+    jd_row_to_export,
+    parse_items_and_pc_search_s_step_from_response_body,
+    pc_search_response_is_empty_ware_list,
+    pc_search_should_retry_fetch,
+    pc_search_ware_list_slot_count_from_body,
+    sleep_pc_search_request_gap,
+)
+
+
+def _pc_request_record_from_url(url: str) -> dict[str, object]:
+    u = urlparse(url)
+    q = parse_qs(u.query, keep_blank_values=True)
+    flat: dict[str, object] = {}
+    for k, v in q.items():
+        flat[k] = v[0] if len(v) == 1 else v
+    body_raw = flat.get("body")
+    body_json: object = None
+    if isinstance(body_raw, str):
+        try:
+            body_json = json.loads(unquote(body_raw))
+        except json.JSONDecodeError:
+            body_json = body_raw
+    return {
+        "url_host": u.netloc,
+        "url_path": u.path,
+        "query_params": flat,
+        "body_param_json": body_json,
+    }
+
+
+def save_pc_request_record(
+    directory: Path,
+    seq: int,
+    *,
+    label: str,
+    keyword: str,
+    api_page: int,
+    api_s: int,
+    log_ctx: str,
+    url: str,
+    headers: dict[str, str],
+    http_status: int,
+    status_text: str,
+    content_type: str,
+) -> None:
+    directory.mkdir(parents=True, exist_ok=True)
+    safe_label = re.sub(r"[^\w.\-]+", "_", label).strip("_") or "req"
+    path = directory / f"pc_request_{seq:03d}_{safe_label}_p{api_page}_s{api_s}.json"
+    record: dict[str, object] = {
+        "seq": seq,
+        "keyword": keyword,
+        "log_ctx": log_ctx,
+        "api_body_page": api_page,
+        "api_body_s": api_s,
+        "http_status": http_status,
+        "http_status_text": status_text,
+        "response_content_type": content_type,
+        "request_url_full": url,
+        **_pc_request_record_from_url(url),
+        "request_headers": headers,
+    }
+    path.write_text(
+        json.dumps(record, ensure_ascii=False, indent=2) + "\n",
+        encoding="utf-8",
+    )
+    print(f"[京东] 已保存请求记录: {path}", file=sys.stderr)
+
+
+def save_pc_search_response_raw(
+    directory: Path,
+    seq: int,
+    body: str,
+    *,
+    label: str,
+    req_page: int,
+    req_s: int,
+    pretty: bool,
+) -> None:
+    directory.mkdir(parents=True, exist_ok=True)
+    ext = "json" if pretty else "js"
+    safe_label = re.sub(r"[^\w.\-]+", "_", label).strip("_") or "resp"
+    path = directory / f"pc_search_{seq:03d}_{safe_label}_req_p{req_page}_s{req_s}.{ext}"
+    if pretty:
+        try:
+            out = json.dumps(json.loads(body), ensure_ascii=False, indent=2) + "\n"
+        except json.JSONDecodeError:
+            out = body
+    else:
+        out = body
+    path.write_text(out, encoding="utf-8")
+    print(f"[京东] 已保存原始响应: {path}", file=sys.stderr)
+
+
+def collect_pc_search_export_rows(
+    context: Any,
+    args: SimpleNamespace,
+    *,
+    page_start: int,
+    pe: int,
+    req_delay_range: tuple[float, float] | None,
+    save_js_dir: Path | None,
+    record_req_dir: Path | None,
+    node_pvid: str | None,
+    cancel_check: Callable[[], bool] | None = None,
+    node_cookie_file: str | None = None,
+) -> list[dict[str, str]]:
+    """
+    执行与 ``jd_search_playwright`` 相同的多页 pc_search 逻辑，返回
+    ``jd_row_to_export`` 后的行（CSV 表头为键），不写 CSV 文件。
+    """
+    fetch_seq: list[int] = [0]
+    all_rows: list[dict[str, str]] = []
+    seen: set[str] = set()
+
+    api_page = 1
+    api_s = 1
+    run_aborted = False
+    n_api_requests = 0
+
+    def _fetch_pc_body_pw(
+        ap: int,
+        as_: int,
+        *,
+        log_ctx: str = "",
+        save_label: str = "fetch",
+        apply_request_gap: bool = True,
+    ) -> tuple[str, str, int]:
+        nonlocal n_api_requests
+        if cancel_check is not None and cancel_check():
+            raise SearchCollectionCancelled(list(all_rows))
+        if apply_request_gap and n_api_requests > 0:
+            sleep_pc_search_request_gap(req_delay_range)
+        if cancel_check is not None and cancel_check():
+            raise SearchCollectionCancelled(list(all_rows))
+        n_api_requests += 1
+        fetch_seq[0] += 1
+        seq_n = fetch_seq[0]
+        data = export_pc_search_request_json(
+            args.q,
+            ap,
+            s=as_,
+            pvid=node_pvid,
+            cookie_file=node_cookie_file,
+        )
+        if cancel_check is not None and cancel_check():
+            raise SearchCollectionCancelled(list(all_rows))
+        u = data["url"]
+        hdrs = {str(k): str(v) for k, v in data["headers"].items()}
+        r = context.request.get(u, headers=hdrs)
+        ctx = f" {log_ctx}" if log_ctx else ""
+        print(
+            f"[京东]{ctx} body.page={ap} body.s={as_} "
+            f"HTTP {r.status} {r.status_text}",
+            file=sys.stderr,
+        )
+        ct = r.headers.get("content-type", "") or ""
+        if record_req_dir is not None:
+            save_pc_request_record(
+                record_req_dir,
+                seq_n,
+                label=save_label,
+                keyword=args.q,
+                api_page=ap,
+                api_s=as_,
+                log_ctx=log_ctx,
+                url=u,
+                headers=hdrs,
+                http_status=r.status,
+                status_text=r.status_text or "",
+                content_type=ct,
+            )
+        return u, r.text(), seq_n
+
+    max_fetch_tries = max(1, int(args.fetch_retries) + 1)
+    retry_pause = max(0.0, float(args.fetch_retry_delay))
+    last_s_step = JD_PC_SEARCH_FALLBACK_S_STEP
+
+    for _skip_screen in range(max(0, page_start - 1)):
+        for _skip_chunk in range(JD_PC_SEARCH_CHUNKS_PER_LOGICAL_PAGE):
+            sl_base = (
+                f"skip_screen{_skip_screen + 1}_"
+                f"chunk{_skip_chunk + 1}"
+            )
+            url, body, seq_n = "", "", 0
+            _skip_rows: list[dict[str, str]] = []
+            s_step = 0
+            blocked: str | None = None
+            for rt in range(max_fetch_tries):
+                sl = sl_base if rt == 0 else f"{sl_base}_retry{rt}"
+                if rt > 0:
+                    if retry_pause > 0:
+                        time.sleep(retry_pause)
+                    print(
+                        f"[京东] 跳过前序屏 重试 {rt}/{max_fetch_tries - 1} "
+                        f"body.page={api_page} body.s={api_s}",
+                        file=sys.stderr,
+                    )
+                url, body, seq_n = _fetch_pc_body_pw(
+                    api_page,
+                    api_s,
+                    log_ctx="跳过前序屏",
+                    save_label=sl,
+                    apply_request_gap=(rt == 0),
+                )
+                if save_js_dir is not None:
+                    save_pc_search_response_raw(
+                        save_js_dir,
+                        seq_n,
+                        body,
+                        label=sl,
+                        req_page=api_page,
+                        req_s=api_s,
+                        pretty=args.pretty_raw_json,
+                    )
+                blocked = _detect_blocked(body)
+                if blocked:
+                    break
+                _skip_rows, s_step = (
+                    parse_items_and_pc_search_s_step_from_response_body(
+                        body,
+                        keyword=args.q,
+                        page=page_start,
+                        request_api_page=api_page,
+                        request_body_s=api_s,
+                    )
+                )
+                if _skip_rows or s_step > 0:
+                    break
+                if rt + 1 < max_fetch_tries and pc_search_should_retry_fetch(
+                    body, has_rows=bool(_skip_rows), s_step=s_step
+                ):
+                    continue
+                break
+            if blocked:
+                print(
+                    f"[京东] 跳过前序屏时 body.page={api_page} body.s={api_s}：{blocked}",
+                    file=sys.stderr,
+                )
+                print(f"  当前 URL: {url[:160]}…", file=sys.stderr)
+                run_aborted = True
+                break
+            if s_step <= 0 and not _skip_rows:
+                if pc_search_response_is_empty_ware_list(body):
+                    print(
+                        "[京东] 跳过前序屏：接口返回空 wareList，停止",
+                        file=sys.stderr,
+                    )
+                    run_aborted = True
+                    break
+                print(
+                    f"[京东] 跳过前序屏 本包仍无有效数据，按 Δs={last_s_step} 强推进游标并继续",
+                    file=sys.stderr,
+                )
+                api_page += 1
+                api_s += last_s_step
+                continue
+            last_s_step = max(last_s_step, s_step)
+            api_page += 1
+            api_s += s_step
+        if run_aborted:
+            break
+
+    if not run_aborted:
+        expect_after_skip = jd_pc_api_body_page_first_pack(page_start)
+        if api_page != expect_after_skip:
+            print(
+                f"[京东] 警告：跳过前序页后首包 body.page 应为 {expect_after_skip}，当前 {api_page}",
+                file=sys.stderr,
+            )
+
+    if not run_aborted:
+        for user_p in range(page_start, pe + 1):
+            page_aborted = False
+            expect_first = jd_pc_api_body_page_first_pack(user_p)
+            if api_page != expect_first:
+                print(
+                    f"[京东] 警告：逻辑第{user_p}页 首包 body.page 应为 {expect_first}，当前 {api_page}",
+                    file=sys.stderr,
+                )
+            for _attempt in range(JD_PC_SEARCH_CHUNKS_PER_LOGICAL_PAGE):
+                sl_base = (
+                    f"logic{user_p}_"
+                    f"chunk{_attempt + 1}of{JD_PC_SEARCH_CHUNKS_PER_LOGICAL_PAGE}"
+                )
+                url, body, seq_n = "", "", 0
+                rows: list[dict[str, str]] = []
+                s_step = 0
+                blocked: str | None = None
+                for rt in range(max_fetch_tries):
+                    sl = sl_base if rt == 0 else f"{sl_base}_retry{rt}"
+                    if rt > 0:
+                        if retry_pause > 0:
+                            time.sleep(retry_pause)
+                        print(
+                            f"[京东] 逻辑第{user_p}页 第{_attempt + 1}包 "
+                            f"重试 {rt}/{max_fetch_tries - 1} "
+                            f"body.page={api_page} body.s={api_s}",
+                            file=sys.stderr,
+                        )
+                    url, body, seq_n = _fetch_pc_body_pw(
+                        api_page,
+                        api_s,
+                        log_ctx=(
+                            f"逻辑第{user_p}页 "
+                            f"第{_attempt + 1}/{JD_PC_SEARCH_CHUNKS_PER_LOGICAL_PAGE}包"
+                        ),
+                        save_label=sl,
+                        apply_request_gap=(rt == 0),
+                    )
+                    if save_js_dir is not None:
+                        save_pc_search_response_raw(
+                            save_js_dir,
+                            seq_n,
+                            body,
+                            label=sl,
+                            req_page=api_page,
+                            req_s=api_s,
+                            pretty=args.pretty_raw_json,
+                        )
+                    blocked = _detect_blocked(body)
+                    if blocked:
+                        break
+                    rows, s_step = (
+                        parse_items_and_pc_search_s_step_from_response_body(
+                            body,
+                            keyword=args.q,
+                            page=user_p,
+                            request_api_page=api_page,
+                            request_body_s=api_s,
+                        )
+                    )
+                    if rows or s_step > 0:
+                        break
+                    if rt + 1 < max_fetch_tries and pc_search_should_retry_fetch(
+                        body, has_rows=bool(rows), s_step=s_step
+                    ):
+                        continue
+                    break
+                if blocked:
+                    print(
+                        f"[京东] CSV第{user_p}页 "
+                        f"body.page={api_page} body.s={api_s}：{blocked}",
+                        file=sys.stderr,
+                    )
+                    print(f"  当前 URL: {url[:160]}…", file=sys.stderr)
+                    page_aborted = True
+                    break
+
+                if not rows and s_step <= 0:
+                    if pc_search_response_is_empty_ware_list(body):
+                        print(
+                            f"[京东] CSV第{user_p}页 "
+                            f"body.page={api_page} body.s={api_s}："
+                            f"接口空 wareList，无更多商品，停止采集",
+                            file=sys.stderr,
+                        )
+                        page_aborted = True
+                        break
+                    print(
+                        f"[京东] CSV第{user_p}页 "
+                        f"body.page={api_page} body.s={api_s}："
+                        f"多次重试后仍无商品；按 Δs={last_s_step} 强推进并继续下一包",
+                        file=sys.stderr,
+                    )
+                    print(f"  当前 URL: {url[:160]}…", file=sys.stderr)
+                    out_dbg = getattr(args, "out", None)
+                    if out_dbg and args.csv:
+                        dbg = Path(out_dbg).with_suffix(
+                            ".debug.json"
+                            if body.lstrip().startswith("{")
+                            else ".debug.html"
+                        )
+                        dbg.write_text(body, encoding="utf-8")
+                        print(f"  已保存调试样本: {dbg}", file=sys.stderr)
+                    api_page += 1
+                    api_s += last_s_step
+                    continue
+
+                if not rows and s_step > 0:
+                    last_s_step = max(last_s_step, s_step)
+                    api_page += 1
+                    api_s += s_step
+                    continue
+
+                pack_skus = {
+                    (r.get("sku_id") or "").strip()
+                    for r in rows
+                    if (r.get("sku_id") or "").strip()
+                }
+                dup_vs_accumulated = len(pack_skus & seen)
+
+                n_added = 0
+                for r in rows:
+                    sku = (r.get("sku_id") or "").strip()
+                    if not sku or sku in seen:
+                        continue
+                    if (
+                        _jd_row_count_for_page(all_rows, user_p)
+                        >= JD_PC_SEARCH_ITEMS_PER_PAGE
+                    ):
+                        break
+                    seen.add(sku)
+                    all_rows.append(r)
+                    n_added += 1
+
+                slots = pc_search_ware_list_slot_count_from_body(body)
+                slot_s = str(slots) if slots is not None else "?"
+                slot_gap = ""
+                if slots is not None and len(rows) < slots:
+                    slot_gap = (
+                        f"，{slots - len(rows)} 个槽为无 SKU 占位（活动/对比卡等）未入库"
+                    )
+                dup_note = ""
+                if dup_vs_accumulated:
+                    dup_note = (
+                        f"；本包 {dup_vs_accumulated} 个 SKU 与此前已采重复"
+                        f"（去重后本包新增 {n_added}）"
+                    )
+                print(
+                    f"[京东] 逻辑第{user_p}页 第{_attempt + 1}包 "
+                    f"wareList 槽位={slot_s}，本包解析 {len(rows)} 行"
+                    f"{slot_gap}，新增 CSV {n_added}{dup_note}",
+                    file=sys.stderr,
+                )
+
+                last_s_step = max(last_s_step, s_step)
+                api_page += 1
+                api_s += s_step
+
+            if page_aborted:
+                run_aborted = True
+                break
+            if user_p < pe:
+                if cancel_check is not None and cancel_check():
+                    raise SearchCollectionCancelled(list(all_rows))
+                time.sleep(max(0.0, args.page_delay))
+
+    if not run_aborted:
+        per_page = [
+            _jd_row_count_for_page(all_rows, p)
+            for p in range(page_start, pe + 1)
+        ]
+        print(
+            f"[京东] 小结：逻辑页 {page_start}–{pe}，"
+            f"各页 CSV 行数（按 page 列）{per_page}，"
+            f"目标≤{JD_PC_SEARCH_ITEMS_PER_PAGE}/页；"
+            f"pc_search {n_api_requests} 次，全局去重合计 {len(all_rows)}。"
+            f" 说明：每包列表槽位数以响应 wareList 长度为准（会随场景变化）；"
+            f"一屏内多包槽位之和也不是固定值。"
+            f"CSV 为 SKU 全局去重行数，不必等于各包槽位之和；"
+            f"若「新增 CSV」远小于当包槽位，多为游标重叠（核对 body.s 与上文逐包日志）。",
+            file=sys.stderr,
+        )
+
+    return [jd_row_to_export(r) for r in all_rows]

backend/crawler_copy/jd_pc_search/search/dump_warelist_skus_to_csv.py

@@ -0,0 +1,85 @@
+# -*- coding: utf-8 -*-
+"""从 pc_search dump .js 中按 wareList 顺序导出 sku_id、shortName（每文件一屏槽位）。"""
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import sys
+from pathlib import Path
+
+# 同目录导入
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from jd_h5_search_requests import (  # noqa: E402
+    JD_SKU_KEYS,
+    _jd_flatten_ware,
+    _sval_jd,
+)
+
+
+def _short_name(d: dict) -> str:
+    d0 = _jd_flatten_ware(d)
+    sn = d0.get("shortName")
+    if sn is None:
+        return ""
+    return str(sn).strip()
+
+
+def _sku_id(d: dict) -> str:
+    return _sval_jd(_jd_flatten_ware(d), JD_SKU_KEYS).strip()
+
+
+def main() -> None:
+    p = argparse.ArgumentParser(description="wareList 槽位 → sku_id + shortName CSV")
+    p.add_argument(
+        "js_files",
+        nargs="+",
+        type=Path,
+        help="按顺序的 dump .js（JSON 一行）",
+    )
+    p.add_argument(
+        "-o",
+        "--out",
+        type=Path,
+        required=True,
+        help="输出 CSV 路径",
+    )
+    args = p.parse_args()
+
+    rows: list[dict[str, str]] = []
+    seq = 0
+    for fi, path in enumerate(args.js_files, start=1):
+        text = path.read_text(encoding="utf-8")
+        payload = json.loads(text)
+        wl = (payload.get("data") or {}).get("wareList")
+        if not isinstance(wl, list):
+            raise SystemExit(f"{path}: 无 data.wareList")
+        for slot, w in enumerate(wl):
+            seq += 1
+            if not isinstance(w, dict):
+                rows.append(
+                    {
+                        "seq": str(seq),
+                        "sku_id": "",
+                        "shortName": "",
+                    }
+                )
+                continue
+            rows.append(
+                {
+                    "seq": str(seq),
+                    "sku_id": _sku_id(w),
+                    "shortName": _short_name(w),
+                }
+            )
+
+    args.out.parent.mkdir(parents=True, exist_ok=True)
+    with args.out.open("w", encoding="utf-8-sig", newline="") as fp:
+        w = csv.DictWriter(fp, fieldnames=["seq", "sku_id", "shortName"])
+        w.writeheader()
+        w.writerows(rows)
+    print(f"Wrote {len(rows)} rows -> {args.out}", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

backend/crawler_copy/jd_pc_search/search/jd_export_search_request.js

@@ -0,0 +1,36 @@
+/**
+ * stdout 输出一行 JSON：{ url, headers }，供 jd_search_playwright.py 使用。h5st 来自 get_h5st。
+ *
+ *   cd crawler/jd_pc_search/search && node jd_export_search_request.js --q 低GI --page 1
+ *   # --page / --s 为 **API 请求体** body.page / body.s（非 Python 里的「逻辑页」L；逻辑页 L 的首包为 page=2L-1）
+ *   node jd_export_search_request.js --q 低GI --page 3 --s 45
+ *   set JD_PVID=搜索页 URL 里的 pvid   # 与浏览器 body.pvid、Referer 一致（可选）
+ */
+const { parseSearchCliArgs, loadJdSearchAuth } = require("../common/jd_search_common.js");
+const { get_h5st, build_pc_search_api_url } = require("./jd_h5st.js");
+const { buildJdPcSearchWareHeaders } = require("./jd_pc_api_headers.js");
+
+try {
+  const { q, page, s, pvid: pvidOpt, cookiePath } = parseSearchCliArgs();
+  const { cookie, uuid, xApiEidToken } = loadJdSearchAuth(cookiePath);
+  if (!cookie) throw new Error("Cookie 为空或文件不存在（--cookie-file 或 common/jd_cookie.txt）");
+  if (!uuid || !xApiEidToken) throw new Error("缺少 uuid 或 x-api-eid-token（Cookie）");
+
+  const pvid = pvidOpt && String(pvidOpt).trim();
+  const pack = get_h5st(pvid ? { page, s, pvid } : { page, s });
+  const url = build_pc_search_api_url(pack, {
+    keyword: q,
+    uuid,
+    xApiEidToken,
+    bodyMode: "json",
+  });
+  const headers = buildJdPcSearchWareHeaders({
+    cookie,
+    keyword: q,
+    pvid: pack.searchParams.pvid,
+  });
+  process.stdout.write(JSON.stringify({ url, headers }));
+} catch (e) {
+  console.error(e.message || String(e));
+  process.exit(1);
+}

backend/crawler_copy/jd_pc_search/search/jd_h5st.js

@@ -0,0 +1,136 @@
+/**
+ * 统一封装：h5st 只通过 get_h5st() 获取（内部 ParamsSign / code.js）。
+ * 流程：get_h5st(opt) → build_pc_search_api_url(pack, { keyword, uuid, xApiEidToken }) → 直连 GET。
+ *
+ * 商品评论签名见 ../comment/jd_h5st_item_comment.js（不修改本文件搜索列表链路）。
+ */
+require("../common/jd_browser_env.js");
+require("../common/code.js");
+const CryptoJS = require("crypto-js");
+
+const DEFAULT_PVID = "90ac040818aa42a389a880e3b119e375";
+const DEFAULT_AREA = "19_1601_50258_129167";
+
+let _psign = null;
+function _ensurePsign() {
+  if (!_psign) {
+    _psign = new window.ParamsSign({
+      appId: "f06cc",
+      preRequest: false,
+      onSign: () => {},
+      onRequestTokenRemotely: () => {},
+    });
+  }
+  return _psign;
+}
+
+/**
+ * 生成 h5st 及签名侧字段（与传入签名的 body 为 SHA256(hex) 一致）。
+ *
+ * @param {object} [opt]
+ * @param {number} [opt.page=1] 请求 body.page（懒加载时每次续拉后浏览器会 page+1，与 body.s 一起变）
+ * @param {string} [opt.pvid]
+ * @param {string} [opt.area]
+ * @param {number} [opt.s=1] 请求 body.s（续拉：上一包 s + max(1, 本包自然位−1)；例 s=1 且自然位 22 → 次包 s=22）
+ * @param {string|number} [opt.psort='3'] 排序：与 PC 搜索一致，'3' 为按销量（与京东前台「销量」选项对应）
+ * @param {number} [opt.t] 签名字段 t，默认 Date.now()
+ * @param {string} [opt.functionId] 默认 pc_search_searchWare
+ * @returns {{
+ *   h5st: string,
+ *   signed: object,
+ *   bodyJson: string,
+ *   bodySha256: string,
+ *   searchParams: object,
+ *   tQuerySecond: string
+ * }}
+ */
+function get_h5st(opt) {
+  const o = opt || {};
+  const page = Math.max(1, parseInt(String(o.page != null ? o.page : 1), 10) || 1);
+  const pvid = o.pvid != null ? String(o.pvid) : DEFAULT_PVID;
+  const area = o.area != null ? String(o.area) : DEFAULT_AREA;
+  const time = o.t != null ? Number(o.t) : Date.now();
+  const functionId = o.functionId || "pc_search_searchWare";
+
+  const psort = o.psort != null ? String(o.psort) : "3";
+
+  const searchParams = {
+    area,
+    concise: false,
+    enc: "utf-8",
+    hoverPictures: false,
+    mode: null,
+    newAdvRepeat: false,
+    new_interval: true,
+    page,
+    pvid,
+    s: o.s != null ? o.s : 1,
+    psort,
+  };
+  const bodyJson = JSON.stringify(searchParams);
+  const bodySha = CryptoJS.SHA256(bodyJson).toString();
+  const paramsH5sign = {
+    appid: "search-pc-java",
+    functionId,
+    client: "pc",
+    clientVersion: "1.0.0",
+    t: time,
+    body: bodySha,
+  };
+  const signed = _ensurePsign()._$sdnmd({ ...paramsH5sign });
+  const tQuerySecond = String(Date.now());
+
+  return {
+    h5st: signed.h5st,
+    signed,
+    bodyJson,
+    bodySha256: signed.body,
+    searchParams,
+    tQuerySecond,
+  };
+}
+
+/**
+ * 拼 https://api.m.jd.com/api?...（query 里两个 t，body 为 JSON 或 SHA256 与签名一致）
+ *
+ * @param {object} pack get_h5st() 返回值
+ * @param {object} opts
+ * @param {string} opts.keyword
+ * @param {string} opts.uuid
+ * @param {string} opts.xApiEidToken
+ * @param {'json'|'sha256'} [opts.bodyMode='json']
+ */
+function build_pc_search_api_url(pack, opts) {
+  const keyword = opts.keyword != null ? String(opts.keyword) : "";
+  const uuid = opts.uuid != null ? String(opts.uuid) : "";
+  const xApiEidToken = opts.xApiEidToken != null ? String(opts.xApiEidToken) : "";
+  const bodyMode = opts.bodyMode === "sha256" ? "sha256" : "json";
+  const signed = pack.signed;
+  const bodyValue = bodyMode === "sha256" ? pack.bodySha256 : pack.bodyJson;
+  const qParts = [
+    ["appid", signed.appid],
+    ["t", String(signed.t)],
+    ["client", signed.client],
+    ["clientVersion", signed.clientVersion],
+    ["cthr", "1"],
+    ["uuid", uuid],
+    ["loginType", "3"],
+    ["keyword", keyword],
+    ["functionId", signed.functionId],
+    ["body", bodyValue],
+    ["x-api-eid-token", xApiEidToken],
+    ["h5st", signed.h5st],
+    ["t", pack.tQuerySecond],
+  ];
+  const qs = qParts
+    .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`)
+    .join("&");
+  return `https://api.m.jd.com/api?${qs}`;
+}
+
+module.exports = {
+  get_h5st,
+  getH5st: get_h5st,
+  build_pc_search_api_url,
+};
+

backend/crawler_copy/jd_pc_search/search/jd_pc_api_headers.js

@@ -0,0 +1,45 @@
+/**
+ * 与 Chrome 访问 search.jd.com → api.m.jd.com 的 pc_search_searchWare 请求头对齐
+ *（含首包、懒加载第二包：body 里 page/s 会变，头字段形态与 DevTools 一致）。
+ * Accept-Encoding 与常见 Chrome 一致；../common/jd_https_fetch.js 会解压 gzip/deflate/br。
+ *
+ * spmTag 可用环境变量 JD_SPM_TAG 覆盖。
+ */
+function buildJdPcSearchWareHeaders(opts) {
+  const keyword = opts.keyword || "低GI";
+  const pvid = opts.pvid || "90ac040818aa42a389a880e3b119e375";
+  const spmTag =
+    opts.spmTag ||
+    process.env.JD_SPM_TAG ||
+    "YTAyMTkuYjAwMjM1Ni5jMDAwMDQ2ODkuc2VhcmNoX2NvbmZpcm1";
+  const encKw = encodeURIComponent(keyword);
+  const encPvid = encodeURIComponent(pvid);
+  const encSpm = encodeURIComponent(spmTag);
+  const referer = `https://search.jd.com/Search?keyword=${encKw}&enc=utf-8&wq=${encKw}&pvid=${encPvid}&spmTag=${encSpm}`;
+
+  const h = {
+    Accept: "application/json, text/plain, */*",
+    "Accept-Encoding": "gzip, deflate, br",
+    "Accept-Language": "zh-CN,zh;q=0.9",
+    "Cache-Control": "no-cache",
+    Pragma: "no-cache",
+    Priority: "u=1, i",
+    "User-Agent":
+      "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36",
+    Referer: referer,
+    Origin: "https://search.jd.com",
+    "sec-ch-ua":
+      '"Chromium";v="146", "Not-A.Brand";v="24", "Google Chrome";v="146"',
+    "sec-ch-ua-mobile": "?0",
+    "sec-ch-ua-platform": '"Windows"',
+    "sec-fetch-dest": "empty",
+    "sec-fetch-mode": "cors",
+    "sec-fetch-site": "same-site",
+    "x-referer-page": "https://search.jd.com/Search",
+    "x-rp-client": "h5_1.0.0",
+  };
+  if (opts.cookie) h.Cookie = opts.cookie;
+  return h;
+}
+
+module.exports = { buildJdPcSearchWareHeaders };

backend/crawler_copy/jd_pc_search/search/jd_search_playwright.py

@@ -0,0 +1,233 @@
+# -*- coding: utf-8 -*-
+"""
+get_h5st（Node）拼 URL + Header，Playwright/Chromium 发 GET（浏览器 TLS，避免 Node 直连 jfe 403）。
+
+响应解析与 ``jd_h5_search_requests.py`` 一致：``_detect_blocked``、
+``parse_items_and_pc_search_s_step_from_response_body``（优先读响应里的下一跳 s，否则启发式）、
+多页合并去重、CSV / 文件输出。
+
+依赖: pip install playwright && playwright install chromium
+
+用法: 修改下方「运行配置」后执行 ``python jd_search_playwright.py``（无命令行参数）。
+
+多页采集逻辑与落盘辅助函数在 ``collect_pc_search_items.py``，供本脚本与上级 ``jd_keyword_pipeline.py`` 共用。
+"""
+
+from __future__ import annotations
+
+import csv
+import json
+import sys
+import time
+from io import StringIO
+from pathlib import Path
+from types import SimpleNamespace
+
+from playwright.sync_api import sync_playwright
+
+from collect_pc_search_items import (
+    collect_pc_search_export_rows,
+    save_pc_request_record,
+    save_pc_search_response_raw,
+)
+from jd_h5_search_requests import CSV_FIELDS, export_pc_search_request_json, parse_request_delay_range
+
+_JD_PC_SEARCH = Path(__file__).resolve().parents[1]
+if str(_JD_PC_SEARCH) not in sys.path:
+    sys.path.insert(0, str(_JD_PC_SEARCH))
+from _low_gi_root import low_gi_project_root  # noqa: E402
+
+# ---------------------------------------------------------------------------
+# 运行配置（按需改这里）
+# ---------------------------------------------------------------------------
+# 路径：副本通过 LOW_GI_PROJECT_ROOT 指向「Low GI」根目录
+_PROJECT_ROOT = low_gi_project_root()
+_PROJECT_DATA = _PROJECT_ROOT / "data" / "JD"
+
+# QUERY：搜索关键词，写入 pc_search 请求与 Referer
+QUERY = "低GI"
+# PVID：与 search.jd.com 结果页 URL 中 pvid 一致时填入；空则用 Node 内默认值
+PVID = ""
+# PAGE_START：起始逻辑页 L（从 1 起）；每逻辑页固定 2 次 pc_search（body.page 为 2L−1、2L）
+PAGE_START = 1
+# PAGE_TO：结束逻辑页（含）；None 表示只采 PAGE_START 这一逻辑页
+PAGE_TO = 10
+# PAGE_DELAY_SEC：相邻两个逻辑页之间的休眠秒数（与 REQUEST_DELAY 可叠加）
+PAGE_DELAY_SEC = 1.2
+# REQUEST_DELAY：每次 pc_search 完成后再发起下一次前的随机等待，如 "30-60"；None 关闭
+REQUEST_DELAY = "30-60"
+# HEADED：True 有头浏览器，便于调试
+HEADED = False
+# FORMAT："items" 解析商品列表；"raw" 仅打一包原始 JSON（勿与多页组合）
+FORMAT = "items"
+# RAW_SINGLE：True 等同只采一包原始响应（与 FORMAT=raw 类同，勿设 PAGE_TO 多页）
+RAW_SINGLE = False
+# CSV_OUTPUT：True 输出 CSV 列（需 FORMAT=items）；False 输出 JSON 数组
+CSV_OUTPUT = True
+# OUT_PATH：结果文件；空则 CSV/JSON 打到 stdout
+OUT_PATH = str(_PROJECT_DATA / "jd_p1_10_2.csv")
+# SAVE_PC_SEARCH_JS_DIR：非空则每次 pc_search 后把响应全文落盘到此目录（对照 Network）
+SAVE_PC_SEARCH_JS_DIR = str(_PROJECT_DATA / "pc_raw_p1_10_2")
+# PRETTY_RAW_JSON：与 SAVE 目录合用 True 时保存为缩进 .json，否则单行 .js
+PRETTY_RAW_JSON = True
+# RECORD_REQUESTS_DIR：非空则每次请求写入 URL、query、body、请求头、HTTP 状态等 JSON
+RECORD_REQUESTS_DIR = str(_PROJECT_DATA / "pc_requests_p1_10_2")
+# FETCH_RETRIES：同一 body.page/s 遇空包或零解析时，除首次外最多再试次数
+FETCH_RETRIES = 3
+# FETCH_RETRY_DELAY_SEC：上述重试间隔（秒），不走 REQUEST_DELAY
+FETCH_RETRY_DELAY_SEC = 3.0
+# ---------------------------------------------------------------------------
+
+
+def _dump_out(text: str, out_path: str | None) -> None:
+    if out_path:
+        Path(out_path).parent.mkdir(parents=True, exist_ok=True)
+        Path(out_path).write_text(text, encoding="utf-8")
+    else:
+        sys.stdout.write(text)
+
+
+def main() -> None:
+    fmt = (FORMAT or "items").strip().lower()
+    if fmt not in ("raw", "items"):
+        print('FORMAT 须为 "items" 或 "raw"', file=sys.stderr)
+        sys.exit(2)
+    args = SimpleNamespace(
+        q=QUERY,
+        pvid=(PVID or "").strip(),
+        page=int(PAGE_START),
+        page_to=PAGE_TO,
+        page_delay=float(PAGE_DELAY_SEC),
+        request_delay=REQUEST_DELAY,
+        headed=bool(HEADED),
+        format=fmt,
+        raw=bool(RAW_SINGLE),
+        csv=bool(CSV_OUTPUT),
+        out=(OUT_PATH or "").strip() or None,
+        save_pc_search_js=(SAVE_PC_SEARCH_JS_DIR or "").strip() or None,
+        pretty_raw_json=bool(PRETTY_RAW_JSON),
+        record_requests=(RECORD_REQUESTS_DIR or "").strip() or None,
+        fetch_retries=int(FETCH_RETRIES),
+        fetch_retry_delay=float(FETCH_RETRY_DELAY_SEC),
+    )
+
+    if args.page_to is not None and args.page_to < args.page:
+        print("PAGE_TO 必须大于等于 PAGE_START", file=sys.stderr)
+        sys.exit(2)
+    if args.raw and args.page_to is not None:
+        print("RAW_SINGLE=True 时不要设置多页 PAGE_TO", file=sys.stderr)
+        sys.exit(2)
+    if args.csv and (args.format == "raw" or args.raw):
+        print("CSV_OUTPUT 需与 FORMAT=items 同时使用", file=sys.stderr)
+        sys.exit(2)
+
+    req_delay_range: tuple[float, float] | None = None
+    if args.request_delay:
+        try:
+            req_delay_range = parse_request_delay_range(str(args.request_delay).strip())
+        except ValueError as e:
+            print(f"[京东] REQUEST_DELAY 无效: {e}", file=sys.stderr)
+            sys.exit(2)
+    if args.fetch_retries < 0:
+        print("[京东] FETCH_RETRIES 不能为负", file=sys.stderr)
+        sys.exit(2)
+
+    page_start = max(1, args.page)
+    pe = args.page_to if args.page_to is not None else page_start
+    pe = max(page_start, pe)
+
+    want_raw = args.format == "raw" or args.raw
+    save_js_dir = (
+        Path(args.save_pc_search_js).resolve()
+        if args.save_pc_search_js
+        else None
+    )
+    record_req_dir = (
+        Path(args.record_requests).resolve()
+        if args.record_requests
+        else None
+    )
+    node_pvid = (args.pvid or "").strip() or None
+
+    with sync_playwright() as pw:
+        browser = pw.chromium.launch(headless=not args.headed)
+        context = browser.new_context()
+        try:
+            if want_raw:
+                data = export_pc_search_request_json(
+                    args.q, 1, s=1, pvid=node_pvid
+                )
+                url = data["url"]
+                headers = {str(k): str(v) for k, v in data["headers"].items()}
+                resp = context.request.get(url, headers=headers)
+                print("HTTP", resp.status, resp.status_text, file=sys.stderr)
+                ct = resp.headers.get("content-type", "")
+                if ct:
+                    print("content-type:", ct, file=sys.stderr)
+                body = resp.text()
+                raw_seq = 1
+                if record_req_dir is not None:
+                    save_pc_request_record(
+                        record_req_dir,
+                        raw_seq,
+                        label="raw_single",
+                        keyword=args.q,
+                        api_page=1,
+                        api_s=1,
+                        log_ctx="--format raw",
+                        url=url,
+                        headers=headers,
+                        http_status=resp.status,
+                        status_text=resp.status_text or "",
+                        content_type=ct,
+                    )
+                if save_js_dir is not None:
+                    save_pc_search_response_raw(
+                        save_js_dir,
+                        raw_seq,
+                        body,
+                        label="raw_single",
+                        req_page=1,
+                        req_s=1,
+                        pretty=args.pretty_raw_json,
+                    )
+                try:
+                    pretty = json.dumps(json.loads(body), ensure_ascii=False, indent=2)
+                    _dump_out(pretty + ("\n" if not pretty.endswith("\n") else ""), args.out)
+                except json.JSONDecodeError:
+                    text = (body[:8000] if body else "(空)") + "\n"
+                    _dump_out(text, args.out)
+                return
+
+            export_rows = collect_pc_search_export_rows(
+                context,
+                args,
+                page_start=page_start,
+                pe=pe,
+                req_delay_range=req_delay_range,
+                save_js_dir=save_js_dir,
+                record_req_dir=record_req_dir,
+                node_pvid=node_pvid,
+            )
+            if args.csv:
+                buf = StringIO()
+                w = csv.DictWriter(
+                    buf, fieldnames=list(CSV_FIELDS), extrasaction="ignore"
+                )
+                w.writeheader()
+                w.writerows(export_rows)
+                csv_text = buf.getvalue()
+                if args.out:
+                    Path(args.out).parent.mkdir(parents=True, exist_ok=True)
+                    Path(args.out).write_text("\ufeff" + csv_text, encoding="utf-8")
+                else:
+                    sys.stdout.write(csv_text)
+            else:
+                txt = json.dumps(export_rows, ensure_ascii=False, indent=2)
+                _dump_out(txt + "\n", args.out)
+        finally:
+            browser.close()
+
+
+if __name__ == "__main__":
+    main()

backend/manage.py

@@ -0,0 +1,22 @@
+#!/usr/bin/env python
+"""Django's command-line utility for administrative tasks."""
+import os
+import sys
+
+
+def main():
+    """Run administrative tasks."""
+    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
+    try:
+        from django.core.management import execute_from_command_line
+    except ImportError as exc:
+        raise ImportError(
+            "Couldn't import Django. Are you sure it's installed and "
+            "available on your PYTHONPATH environment variable? Did you "
+            "forget to activate a virtual environment?"
+        ) from exc
+    execute_from_command_line(sys.argv)
+
+
+if __name__ == '__main__':
+    main()

backend/pipeline/admin.py

@@ -0,0 +1,74 @@
+from django.contrib import admin
+
+from .models import (
+    JdJobCommentRow,
+    JdJobDetailRow,
+    JdJobMergedRow,
+    JdJobSearchRow,
+    JdProduct,
+    JdProductSnapshot,
+    PipelineJob,
+)
+
+
+@admin.register(PipelineJob)
+class PipelineJobAdmin(admin.ModelAdmin):
+    list_display = ("id", "platform", "keyword", "status", "created_at")
+    list_filter = ("status", "platform")
+    search_fields = ("keyword", "run_dir")
+
+
+@admin.register(JdProduct)
+class JdProductAdmin(admin.ModelAdmin):
+    list_display = (
+        "id",
+        "platform",
+        "sku_id",
+        "title",
+        "detail_brand",
+        "last_captured_at",
+        "last_job",
+    )
+    list_filter = ("platform",)
+    search_fields = ("sku_id", "title", "detail_brand", "ware_id")
+    raw_id_fields = ("last_job",)
+
+
+@admin.register(JdProductSnapshot)
+class JdProductSnapshotAdmin(admin.ModelAdmin):
+    list_display = ("id", "product", "job", "captured_at", "run_dir")
+    list_filter = ("captured_at",)
+    search_fields = ("run_dir", "product__sku_id")
+    raw_id_fields = ("product", "job")
+
+
+@admin.register(JdJobSearchRow)
+class JdJobSearchRowAdmin(admin.ModelAdmin):
+    list_display = ("id", "job", "row_index", "sku_id")
+    list_filter = ("job",)
+    search_fields = ("sku_id",)
+    raw_id_fields = ("job",)
+
+
+@admin.register(JdJobDetailRow)
+class JdJobDetailRowAdmin(admin.ModelAdmin):
+    list_display = ("id", "job", "row_index", "sku_id")
+    list_filter = ("job",)
+    search_fields = ("sku_id",)
+    raw_id_fields = ("job",)
+
+
+@admin.register(JdJobCommentRow)
+class JdJobCommentRowAdmin(admin.ModelAdmin):
+    list_display = ("id", "job", "row_index", "sku_id")
+    list_filter = ("job",)
+    search_fields = ("sku_id",)
+    raw_id_fields = ("job",)
+
+
+@admin.register(JdJobMergedRow)
+class JdJobMergedRowAdmin(admin.ModelAdmin):
+    list_display = ("id", "job", "row_index", "sku_id")
+    list_filter = ("job",)
+    search_fields = ("sku_id", "pipeline_keyword")
+    raw_id_fields = ("job",)

backend/pipeline/brief_compact.py

@@ -0,0 +1,103 @@
+"""压缩 competitor-brief 供大模型输入，控制 token 体积（无 Django 依赖）。"""
+from __future__ import annotations
+
+import copy
+import json
+from typing import Any
+
+
+def matrix_overview_for_llm(brief: dict[str, Any]) -> list[dict[str, Any]]:
+    """
+    从**完整** brief 提取矩阵分组摘要，供大模型在 matrix 正文被裁剪后仍能写「按细分类目分组」章节。
+    """
+    mg = brief.get("matrix_by_group")
+    if not isinstance(mg, list) or not mg:
+        return []
+    out: list[dict[str, Any]] = []
+    for g in mg:
+        if not isinstance(g, dict):
+            continue
+        name = g.get("group") or "—"
+        skus = g.get("skus") if isinstance(g.get("skus"), list) else []
+        brands: list[str] = []
+        for s in skus[:100]:
+            if isinstance(s, dict):
+                bb = (s.get("brand") or "").strip()
+                if bb and bb not in brands:
+                    brands.append(bb)
+        out.append(
+            {
+                "group": name,
+                "sku_count": int(g.get("sku_count") or len(skus)),
+                "distinct_brands_sample": brands[:15],
+            }
+        )
+    return out
+
+
+def _trim_matrix(b: dict[str, Any], *, per_group: int, max_groups: int) -> None:
+    mg = b.get("matrix_by_group")
+    if not isinstance(mg, list):
+        return
+    trimmed: list[dict[str, Any]] = []
+    for g in mg[:max_groups]:
+        if not isinstance(g, dict):
+            continue
+        g2 = dict(g)
+        skus = g2.get("skus")
+        if isinstance(skus, list):
+            if per_group <= 0:
+                g2["skus"] = []
+            else:
+                g2["skus"] = skus[:per_group]
+        trimmed.append(g2)
+    b["matrix_by_group"] = trimmed
+
+
+def _trim_feedback(b: dict[str, Any], max_groups: int) -> None:
+    cf = b.get("consumer_feedback_by_matrix_group")
+    if isinstance(cf, list):
+        b["consumer_feedback_by_matrix_group"] = cf[:max_groups]
+
+
+def _json_len(b: dict[str, Any]) -> int:
+    return len(json.dumps(b, ensure_ascii=False))
+
+
+def compact_brief_for_llm(
+    brief: dict[str, Any],
+    *,
+    max_chars: int = 350_000,
+) -> dict[str, Any]:
+    """
+    深拷贝后裁剪矩阵 SKU 列表、反馈组数；仍超长则逐步收紧直至省略大块。
+    始终附带 ``matrix_overview_for_llm``（来自裁剪前完整 brief），避免大模型漏写竞品矩阵章节。
+    """
+    matrix_ov = matrix_overview_for_llm(brief)
+    b = copy.deepcopy(brief)
+    if isinstance(b.get("run_dir"), str):
+        b["run_dir"] = "(已省略)"
+
+    def _finalize() -> dict[str, Any]:
+        b["matrix_overview_for_llm"] = matrix_ov
+        return b
+
+    caps = [(120, 24), (80, 20), (40, 16), (30, 12), (18, 10), (10, 8), (5, 6), (0, 6)]
+    for per_g, max_gr in caps:
+        _trim_matrix(b, per_group=per_g, max_groups=max_gr)
+        _trim_feedback(b, max_gr)
+        if _json_len(b) <= max_chars:
+            return _finalize()
+
+    b.pop("matrix_by_group", None)
+    b["matrix_by_group_omitted"] = True
+    _trim_feedback(b, 6)
+    if _json_len(b) <= max_chars:
+        return _finalize()
+
+    b.pop("consumer_feedback_by_matrix_group", None)
+    b["consumer_feedback_by_matrix_group_omitted"] = True
+    lv = b.get("list_visibility_proxy")
+    if isinstance(lv, dict) and _json_len(b) > max_chars:
+        b["list_visibility_proxy"] = {"_omitted": True, "keys": list(lv.keys())[:20]}
+    return _finalize()

backend/pipeline/brief_pack.py

@@ -0,0 +1,198 @@
+"""一键简报包：ZIP 内含完整 Markdown 报告、结构化 JSON、要点摘录。"""
+from __future__ import annotations
+
+import io
+import json
+import zipfile
+from pathlib import Path
+from typing import Any
+
+
+def _pct(x: Any) -> str:
+    if x is None:
+        return "—"
+    try:
+        return f"{100 * float(x):.1f}%"
+    except (TypeError, ValueError):
+        return str(x)
+
+
+def _num(x: Any) -> str:
+    if x is None:
+        return "—"
+    if isinstance(x, (int, float)):
+        if isinstance(x, float) and x != int(x):
+            return f"{x:.2f}"
+        return str(int(x)) if isinstance(x, float) and x == int(x) else str(x)
+    return str(x)
+
+
+def markdown_summary_from_brief(brief: dict[str, Any]) -> str:
+    """由 ``competitor-brief`` JSON 生成便于扫读的 Markdown（非 LLM）。"""
+    lines: list[str] = [
+        "# 竞品要点摘录（机器整理）",
+        "",
+        "> 与同批 **完整报告**、**结构化 JSON** 同源；规则汇总，定稿前请人工核对。",
+        "",
+    ]
+    kw = brief.get("keyword") or "—"
+    batch = brief.get("batch_label") or "—"
+    lines.extend(
+        [
+            "## 基本信息",
+            "",
+            f"- **监测词**：{kw}",
+            f"- **批次**：{batch}",
+            "",
+        ]
+    )
+
+    scope = brief.get("scope") or {}
+    if scope:
+        lines.extend(
+            [
+                "## 样本范围",
+                "",
+                f"- **深入 SKU 数**：{_num(scope.get('merged_sku_count'))}",
+                f"- **评价条数（扁平）**：{_num(scope.get('comment_flat_rows'))}",
+                f"- **结构分析用列表行数**：{_num(scope.get('structure_source_rows'))}",
+                f"- **是否含 PC 搜索全量导出**：{'是' if scope.get('uses_pc_search_list_export') else '否'}",
+                "",
+            ]
+        )
+
+    raw = brief.get("pc_search_raw") or {}
+    if raw.get("result_count_consensus") is not None:
+        lines.extend(
+            [
+                "## 列表侧检索规模（接口申报）",
+                "",
+                f"- **resultCount 共识值**：{_num(raw.get('result_count_consensus'))}",
+                "",
+            ]
+        )
+
+    conc = brief.get("concentration") or {}
+    shops = conc.get("shops_from_list") or {}
+    if shops.get("cr1") is not None or shops.get("top_label"):
+        lines.extend(
+            [
+                "## 店铺集中度（列表）",
+                "",
+                f"- **第一大店铺份额**：{_pct(shops.get('cr1'))}（第一店铺：{shops.get('top_label') or '—'}）",
+                f"- **前三店铺合计份额**：{_pct(shops.get('cr3'))}",
+                "",
+            ]
+        )
+    dbrand = conc.get("detail_brand_among_merged") or {}
+    if dbrand.get("cr1") is not None or dbrand.get("top_label"):
+        lines.extend(
+            [
+                "## 品牌（深入样本）",
+                "",
+                f"- **第一大品牌份额（深入样本）**：{_pct(dbrand.get('cr1'))}（头部：{dbrand.get('top_label') or '—'}）",
+                f"- **前三品牌合计份额**：{_pct(dbrand.get('cr3'))}",
+                "",
+            ]
+        )
+
+    pst = brief.get("price_stats") or {}
+    if pst.get("n"):
+        src = brief.get("price_stats_source") or "—"
+        lines.extend(
+            [
+                "## 价格（展示价统计）",
+                "",
+                f"- **样本量 n**：{_num(pst.get('n'))}；**统计口径**：{src}",
+                f"- **区间**：{_num(pst.get('min'))} ～ {_num(pst.get('max'))}；**中位数**：{_num(pst.get('median'))}",
+                "",
+            ]
+        )
+
+    mix = brief.get("category_mix_top") or []
+    if mix:
+        lines.extend(["## 类目结构（Top）", ""])
+        for item in mix[:8]:
+            if isinstance(item, dict):
+                lines.append(
+                    f"- {item.get('label') or '—'}：{_num(item.get('count'))}"
+                )
+        lines.append("")
+
+    ckw = brief.get("comment_focus_keywords") or []
+    if ckw:
+        lines.extend(["## 评价关注词（Top）", ""])
+        for item in ckw[:10]:
+            if isinstance(item, dict):
+                lines.append(
+                    f"- **{item.get('word') or '—'}**：{_num(item.get('count'))} 次"
+                )
+        lines.append("")
+
+    usc = brief.get("usage_scenarios") or []
+    if usc:
+        lines.extend(["## 用途/场景（预设词组，Top）", ""])
+        for item in usc[:8]:
+            if isinstance(item, dict):
+                lines.append(
+                    f"- **{item.get('scenario') or '—'}**：{_num(item.get('count'))} 条（约 {_pct(item.get('share_of_text_units'))} 文本单元）"
+                )
+        lines.append("")
+
+    hints = brief.get("strategy_hints") or []
+    if hints:
+        lines.extend(["## 策略提示（规则）", ""])
+        for h in hints:
+            lines.append(f"- {h}")
+        lines.append("")
+
+    lines.extend(
+        [
+            "---",
+            "",
+            "*更细的矩阵与消费者反馈见简报包内完整分析报告；结构化字段见同包内摘要数据文件。*",
+            "",
+        ]
+    )
+    return "\n".join(lines)
+
+
+README_TXT = """竞品「一键简报包」说明（Market-Assistant）
+============================================
+
+本 ZIP 由「报告查看」页一键导出，内含：
+
+  说明文件              — 本文件
+  完整分析报告          — 与任务批次中的主报告文稿一致
+  统计图 PNG            — report_assets 目录（与报告内插图同源）
+  结构化摘要数据        — 与「结构化摘要」接口同源，可供其它工具读取
+  要点摘录              — 由摘要自动整理的速读稿，便于邮件/转发前浏览
+
+使用建议：对外发送前请核对要点摘录与完整报告中的结论；数据边界见报告第一章。
+"""
+
+
+def build_brief_pack_zip_bytes(run_dir: Path, brief: dict[str, Any]) -> bytes:
+    """
+    生成 ZIP 字节流。``run_dir`` 下须存在 ``competitor_analysis.md``。
+    """
+    run_dir = Path(run_dir).resolve()
+    report_path = run_dir / "competitor_analysis.md"
+    if not report_path.is_file():
+        raise FileNotFoundError("缺少已生成的分析报告文件，请先在「报告生成」中生成报告")
+
+    buf = io.BytesIO()
+    with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
+        zf.writestr("01_竞品分析报告.md", report_path.read_text(encoding="utf-8"))
+        zf.writestr(
+            "02_结构化摘要.json",
+            json.dumps(brief, ensure_ascii=False, indent=2),
+        )
+        zf.writestr("03_要点摘录.md", markdown_summary_from_brief(brief))
+        zf.writestr("00_说明.txt", README_TXT)
+        assets = run_dir / "report_assets"
+        if assets.is_dir():
+            for fp in sorted(assets.iterdir()):
+                if fp.is_file():
+                    zf.write(fp, f"report_assets/{fp.name}")
+    return buf.getvalue()

backend/pipeline/cookie_paste.py

@@ -0,0 +1,16 @@
+# -*- coding: utf-8 -*-
+"""工作台粘贴的 Cookie 规范化（与 jd_cookie.txt 单行内容一致）。"""
+
+
+def normalize_browser_cookie_paste(raw: str) -> str:
+    """
+    - 去掉首尾空白
+    - 若整段以 ``Cookie:`` 开头（不区分大小写，常见于 DevTools 复制请求头），去掉该前缀
+    """
+    s = (raw or "").strip()
+    if not s:
+        return ""
+    prefix = "cookie:"
+    if s.lower().startswith(prefix):
+        return s[len(prefix) :].strip()
+    return s

backend/pipeline/csv_schema.py

@@ -0,0 +1,181 @@
+"""
+与 ``jd_pc_search`` 导出 CSV 列对齐的字段名映射（入库 / API / 导出共用）。
+内部键与爬虫侧 ``JD_ITEM_CSV_FIELDS`` / ``WARE_PARSED_CSV_FIELDNAMES`` 一致，便于对照源码。
+"""
+from __future__ import annotations
+
+# --- 搜索导出 pc_search_export.csv（列名为中文，与 jd_h5_search_requests.JD_EXPORT_COLUMN_HEADERS 一致）---
+JD_SEARCH_INTERNAL_KEYS: tuple[str, ...] = (
+    "item_id",
+    "sku_id",
+    "title",
+    "price",
+    "coupon_price",
+    "original_price",
+    "selling_point",
+    "comment_sales_floor",
+    "hot_list_rank",
+    "comment_count",
+    "shop_name",
+    "shop_url",
+    "shop_info_url",
+    "location",
+    "detail_url",
+    "image",
+    "seckill_info",
+    "attributes",
+    "leaf_category",
+    "platform",
+    "keyword",
+    "page",
+)
+
+JD_SEARCH_CSV_HEADERS: dict[str, str] = {
+    "item_id": "主商品ID(wareId)",
+    "sku_id": "SKU(skuId)",
+    "title": "标题(wareName)",
+    "price": "标价(jdPrice,jdPriceText,realPrice)",
+    "coupon_price": "券后到手价(couponPrice,subsidyPrice,finalPrice.estimatedPrice,priceShow)",
+    "original_price": "原价(oriPrice,originalPrice,marketPrice)",
+    "selling_point": "卖点(sellingPoint)",
+    "comment_sales_floor": "销量楼层(commentSalesFloor)",
+    "hot_list_rank": "榜单类文案(标签/腰带/标题数组中的榜、TOP 等)",
+    "comment_count": "评价量(commentFuzzy)",
+    "shop_name": "店铺名(shopName)",
+    "shop_url": "店铺链接(shopUrl,shopId)",
+    "shop_info_url": "店铺信息链接(shopInfoUrl,brandUrl)",
+    "location": "地域(deliveryAddress,area,procity)",
+    "detail_url": "商品链接(toUrl,clickUrl,item.m.jd.com)",
+    "image": "主图(imageurl,imageUrl)",
+    "seckill_info": "秒杀(seckillInfo,secKill)",
+    "attributes": "规格属性(propertyList,color,catid,shortName)",
+    "leaf_category": "类目(leafCategory,cid3Name,catid)",
+    "platform": "平台(platform)",
+    "keyword": "搜索词(keyword)",
+    "page": "页码(page)",
+}
+
+# CSV 表头 -> 模型属性名
+SEARCH_CSV_HEADER_TO_FIELD: dict[str, str] = {
+    h: k for k, h in JD_SEARCH_CSV_HEADERS.items()
+}
+
+# lean 商详子集：合并宽表商详块、detail_ware_export（lean）、JdJobDetailRow 共用（CSV 列名与 ORM 一致）
+LEAN_DETAIL_EXPORT_FIELDNAMES: tuple[str, ...] = (
+    "detail_brand",
+    "detail_price_final",
+    "detail_shop_name",
+    "detail_category_path",
+    "detail_product_attributes",
+    "detail_body_ingredients",
+)
+
+# --- 商详 detail_ware_export.csv（lean：skuId + 上列；full 模式爬虫仍可能多列，入库只认 DETAIL_CSV_COLUMNS）---
+JD_DETAIL_MERGE_KEYS: tuple[str, ...] = LEAN_DETAIL_EXPORT_FIELDNAMES
+
+DETAIL_CSV_COLUMNS: tuple[str, ...] = ("skuId", *JD_DETAIL_MERGE_KEYS)
+
+DETAIL_CSV_TO_FIELD: dict[str, str] = {
+    "skuId": "sku_id",
+    **{k: k for k in JD_DETAIL_MERGE_KEYS},
+}
+
+# --- 评价 comments_flat.csv ---
+COMMENT_CSV_COLUMNS: tuple[str, ...] = (
+    "sku",
+    "commentId",
+    "userNickName",
+    "tagCommentContent",
+    "commentDate",
+    "buyCountText",
+    "largePicURLs",
+    "commentScore",
+)
+
+COMMENT_CSV_TO_FIELD: dict[str, str] = {
+    "sku": "sku_id",
+    "commentId": "comment_id",
+    "userNickName": "user_nick_name",
+    "tagCommentContent": "tag_comment_content",
+    "commentDate": "comment_date",
+    "buyCountText": "buy_count_text",
+    "largePicURLs": "large_pic_urls",
+    "commentScore": "comment_score",
+}
+
+# --- 合并宽表 keyword_pipeline_merged.csv（lean = 搜索块 + 商详块 + 评论块；改列请改对应块，勿在尾部堆列）---
+
+MERGED_SEARCH_CSV_COLUMNS: tuple[str, ...] = (
+    "pipeline_keyword",
+    "SKU(skuId)",
+    "主商品ID(wareId)",
+    "标题(wareName)",
+    "标价(jdPrice,jdPriceText,realPrice)",
+    "券后到手价(couponPrice,subsidyPrice,finalPrice.estimatedPrice,priceShow)",
+    "原价(oriPrice,originalPrice,marketPrice)",
+    "卖点(sellingPoint)",
+    "榜单类文案(标签/腰带/标题数组中的榜、TOP 等)",
+    "评价量(commentFuzzy)",
+    "销量楼层(commentSalesFloor)",
+    "店铺名(shopName)",
+    "商品链接(toUrl,clickUrl,item.m.jd.com)",
+    "主图(imageurl,imageUrl)",
+    "规格属性(propertyList,color,catid,shortName)",
+    "类目(leafCategory,cid3Name,catid)",
+    "搜索词(keyword)",
+    "页码(page)",
+)
+
+MERGED_SEARCH_INTERNAL_KEYS: tuple[str, ...] = (
+    "pipeline_keyword",
+    "sku_id",
+    "ware_id",
+    "title",
+    "price",
+    "coupon_price",
+    "original_price",
+    "selling_point",
+    "hot_list_rank",
+    "comment_fuzzy",
+    "comment_sales_floor",
+    "shop_name",
+    "detail_url",
+    "image",
+    "attributes",
+    "leaf_category",
+    "keyword",
+    "page",
+)
+
+# 商详块：列名与 ORM 属性同名；与 LEAN_DETAIL_EXPORT_FIELDNAMES / 流水线 lean 一致
+MERGED_LEAN_DETAIL_KEYS: tuple[str, ...] = LEAN_DETAIL_EXPORT_FIELDNAMES
+
+MERGED_COMMENT_CSV_COLUMNS: tuple[str, ...] = (
+    "comment_count",
+    "comment_preview",
+)
+
+MERGED_COMMENT_INTERNAL_KEYS: tuple[str, ...] = (
+    "pipeline_comment_count",
+    "comment_preview",
+)
+
+MERGED_CSV_COLUMNS: tuple[str, ...] = (
+    *MERGED_SEARCH_CSV_COLUMNS,
+    *MERGED_LEAN_DETAIL_KEYS,
+    *MERGED_COMMENT_CSV_COLUMNS,
+)
+
+MERGED_INTERNAL_KEYS: tuple[str, ...] = (
+    *MERGED_SEARCH_INTERNAL_KEYS,
+    *MERGED_LEAN_DETAIL_KEYS,
+    *MERGED_COMMENT_INTERNAL_KEYS,
+)
+
+assert len(MERGED_CSV_COLUMNS) == len(MERGED_INTERNAL_KEYS)
+
+MERGED_CSV_TO_FIELD: dict[str, str] = dict(zip(MERGED_CSV_COLUMNS, MERGED_INTERNAL_KEYS))
+
+MERGED_FIELD_TO_CSV_HEADER: dict[str, str] = {
+    internal: csv_h for csv_h, internal in MERGED_CSV_TO_FIELD.items()
+}

backend/pipeline/dataset_nonempty.py

@@ -0,0 +1,122 @@
+"""按任务扫描库内行，得到「全表至少一格非空」的列，供摘要 / 浏览 / 导出一致裁剪。"""
+from __future__ import annotations
+
+from .csv_schema import (
+    COMMENT_CSV_COLUMNS,
+    COMMENT_CSV_TO_FIELD,
+    DETAIL_CSV_COLUMNS,
+    DETAIL_CSV_TO_FIELD,
+    JD_SEARCH_CSV_HEADERS,
+    JD_SEARCH_INTERNAL_KEYS,
+    MERGED_FIELD_TO_CSV_HEADER,
+    MERGED_INTERNAL_KEYS,
+)
+from .models import JdJobCommentRow, JdJobDetailRow, JdJobMergedRow, JdJobSearchRow, PipelineJob
+from .row_serialize import COMMENT_FIELDS_ORDER, DETAIL_FIELDS_ORDER
+
+
+def _is_nonempty(val) -> bool:
+    if val is None:
+        return False
+    return str(val).strip() != ""
+
+
+def nonempty_search_keys_for_job(job: PipelineJob) -> list[str]:
+    qs = JdJobSearchRow.objects.filter(job=job)
+    present = {k: False for k in JD_SEARCH_INTERNAL_KEYS}
+    for row in qs.iterator(chunk_size=400):
+        for k in JD_SEARCH_INTERNAL_KEYS:
+            if not present[k] and _is_nonempty(getattr(row, k, None)):
+                present[k] = True
+    return [k for k in JD_SEARCH_INTERNAL_KEYS if present[k]]
+
+
+def nonempty_detail_fields_for_job(job: PipelineJob) -> list[str]:
+    qs = JdJobDetailRow.objects.filter(job=job)
+    present = {k: False for k in DETAIL_FIELDS_ORDER}
+    for row in qs.iterator(chunk_size=400):
+        for k in DETAIL_FIELDS_ORDER:
+            if not present[k] and _is_nonempty(getattr(row, k, None)):
+                present[k] = True
+    return [k for k in DETAIL_FIELDS_ORDER if present[k]]
+
+
+def nonempty_comment_fields_for_job(job: PipelineJob) -> list[str]:
+    qs = JdJobCommentRow.objects.filter(job=job)
+    present = {k: False for k in COMMENT_FIELDS_ORDER}
+    for row in qs.iterator(chunk_size=400):
+        for k in COMMENT_FIELDS_ORDER:
+            if not present[k] and _is_nonempty(getattr(row, k, None)):
+                present[k] = True
+    return [k for k in COMMENT_FIELDS_ORDER if present[k]]
+
+
+def nonempty_merged_fields_for_job(job: PipelineJob) -> list[str]:
+    qs = JdJobMergedRow.objects.filter(job=job)
+    present = {k: False for k in MERGED_INTERNAL_KEYS}
+    for row in qs.iterator(chunk_size=400):
+        for k in MERGED_INTERNAL_KEYS:
+            if not present[k] and _is_nonempty(getattr(row, k, None)):
+                present[k] = True
+    return [k for k in MERGED_INTERNAL_KEYS if present[k]]
+
+
+def search_columns_for_api(job: PipelineJob) -> list[dict[str, str]]:
+    return [{"key": k, "label": JD_SEARCH_CSV_HEADERS[k]} for k in nonempty_search_keys_for_job(job)]
+
+
+def _detail_field_to_csv_col(field: str) -> str:
+    for col, fn in DETAIL_CSV_TO_FIELD.items():
+        if fn == field:
+            return col
+    return field
+
+
+def detail_columns_for_api(job: PipelineJob) -> list[dict[str, str]]:
+    return [
+        {"key": f, "label": _detail_field_to_csv_col(f)}
+        for f in nonempty_detail_fields_for_job(job)
+    ]
+
+
+def _comment_field_to_csv_col(field: str) -> str:
+    for col, fn in COMMENT_CSV_TO_FIELD.items():
+        if fn == field:
+            return col
+    return field
+
+
+def comment_columns_for_api(job: PipelineJob) -> list[dict[str, str]]:
+    return [
+        {"key": f, "label": _comment_field_to_csv_col(f)}
+        for f in nonempty_comment_fields_for_job(job)
+    ]
+
+
+def merged_columns_for_api(job: PipelineJob) -> list[dict[str, str]]:
+    return [
+        {"key": k, "label": MERGED_FIELD_TO_CSV_HEADER[k]}
+        for k in nonempty_merged_fields_for_job(job)
+    ]
+
+
+def search_export_headers(job: PipelineJob) -> list[str]:
+    keys = nonempty_search_keys_for_job(job)
+    return ["id", "row_index"] + [JD_SEARCH_CSV_HEADERS[k] for k in keys]
+
+
+def detail_export_headers(job: PipelineJob) -> list[str]:
+    fields = set(nonempty_detail_fields_for_job(job))
+    cols = [c for c in DETAIL_CSV_COLUMNS if DETAIL_CSV_TO_FIELD[c] in fields]
+    return ["id", "row_index"] + cols
+
+
+def comment_export_headers(job: PipelineJob) -> list[str]:
+    fields = set(nonempty_comment_fields_for_job(job))
+    cols = [c for c in COMMENT_CSV_COLUMNS if COMMENT_CSV_TO_FIELD[c] in fields]
+    return ["id", "row_index"] + cols
+
+
+def merged_export_headers(job: PipelineJob) -> list[str]:
+    keys = nonempty_merged_fields_for_job(job)
+    return ["id", "row_index"] + [MERGED_FIELD_TO_CSV_HEADER[k] for k in keys]

backend/pipeline/detail_ware_regen.py

@@ -0,0 +1,74 @@
+"""从 ``run_dir/detail/ware_*_response.json`` 与 ``keyword_pipeline_merged.csv`` 重写 ``detail_ware_export.csv``（lean 列集，不重新抓接口）。"""
+from __future__ import annotations
+
+import csv
+import sys
+from pathlib import Path
+
+from .ingest import FILE_DETAIL_WARE_CSV, FILE_MERGED_CSV, SKU_FIELD_MERGED
+
+
+def _ensure_crawler_copy_path() -> None:
+    root = Path(__file__).resolve().parent.parent / "crawler_copy" / "jd_pc_search"
+    for sub in ("detail", ""):
+        p = root / sub if sub else root
+        s = str(p.resolve())
+        if s not in sys.path:
+            sys.path.insert(0, s)
+
+
+def regenerate_detail_ware_rows(run_dir: Path) -> list[dict[str, str]]:
+    _ensure_crawler_copy_path()
+    from jd_detail_ware_business_requests import (  # noqa: WPS433
+        DETAIL_WARE_LEAN_CSV_FIELDNAMES,
+        detail_ware_lean_csv_row,
+    )
+
+    run_dir = run_dir.expanduser().resolve()
+    merged_path = run_dir / FILE_MERGED_CSV
+    detail_dir = run_dir / "detail"
+    if not merged_path.is_file():
+        raise FileNotFoundError(f"缺少合并表: {merged_path}")
+    if not detail_dir.is_dir():
+        raise FileNotFoundError(f"缺少 detail 目录: {detail_dir}")
+
+    rows_out: list[dict[str, str]] = []
+    with merged_path.open(encoding="utf-8-sig", newline="") as f:
+        reader = csv.DictReader(f)
+        for row in reader:
+            sku = (row.get(SKU_FIELD_MERGED) or "").strip()
+            if not sku:
+                continue
+            jp = detail_dir / f"ware_{sku}_response.json"
+            if not jp.is_file():
+                continue
+            text = jp.read_text(encoding="utf-8")
+            ing = (row.get("detail_body_ingredients") or "").strip()
+            rows_out.append(
+                detail_ware_lean_csv_row(
+                    sku,
+                    200,
+                    text,
+                    detail_body_ingredients=ing,
+                    detail_body_ingredients_source_url="",
+                )
+            )
+    return rows_out
+
+
+def write_detail_ware_export_csv(run_dir: Path) -> tuple[int, Path]:
+    rows = regenerate_detail_ware_rows(run_dir)
+    _ensure_crawler_copy_path()
+    from jd_detail_ware_business_requests import DETAIL_WARE_LEAN_CSV_FIELDNAMES  # noqa: WPS433
+
+    out = run_dir.expanduser().resolve() / FILE_DETAIL_WARE_CSV
+    out.parent.mkdir(parents=True, exist_ok=True)
+    with out.open("w", encoding="utf-8-sig", newline="") as f:
+        w = csv.DictWriter(
+            f,
+            fieldnames=list(DETAIL_WARE_LEAN_CSV_FIELDNAMES),
+            extrasaction="ignore",
+        )
+        w.writeheader()
+        w.writerows(rows)
+    return len(rows), out

backend/pipeline/export_job.py

@@ -0,0 +1,344 @@
+"""任务维度数据集导出：JSON / CSV（UTF-8 BOM）/ xlsx。列与源 CSV 对齐。"""
+from __future__ import annotations
+
+import csv
+import json
+from io import BytesIO, StringIO
+from typing import Any
+
+from django.db.models import QuerySet
+from openpyxl import Workbook
+
+from .csv_schema import (
+    COMMENT_CSV_TO_FIELD,
+    DETAIL_CSV_TO_FIELD,
+    JD_SEARCH_CSV_HEADERS,
+    MERGED_FIELD_TO_CSV_HEADER,
+)
+from .dataset_nonempty import (
+    comment_export_headers,
+    detail_export_headers,
+    merged_export_headers,
+    nonempty_comment_fields_for_job,
+    nonempty_detail_fields_for_job,
+    nonempty_merged_fields_for_job,
+    nonempty_search_keys_for_job,
+    search_export_headers,
+)
+from .models import JdJobCommentRow, JdJobDetailRow, JdJobMergedRow, JdJobSearchRow, PipelineJob
+from .row_serialize import (
+    comment_row_to_dict,
+    detail_row_to_dict,
+    merged_row_to_dict,
+    search_row_to_dict,
+)
+
+
+def _search_row_csv_dict(r: JdJobSearchRow, internal_keys: list[str]) -> dict[str, Any]:
+    d = search_row_to_dict(r)
+    out: dict[str, Any] = {"id": d["id"], "row_index": d["row_index"]}
+    for k in internal_keys:
+        out[JD_SEARCH_CSV_HEADERS[k]] = d.get(k, "")
+    return out
+
+
+def _detail_row_csv_dict(r: JdJobDetailRow, csv_cols: list[str]) -> dict[str, Any]:
+    d = detail_row_to_dict(r)
+    out: dict[str, Any] = {"id": d["id"], "row_index": d["row_index"]}
+    for col in csv_cols:
+        fn = DETAIL_CSV_TO_FIELD[col]
+        out[col] = d.get(fn, "")
+    return out
+
+
+def _comment_row_csv_dict(r: JdJobCommentRow, csv_cols: list[str]) -> dict[str, Any]:
+    d = comment_row_to_dict(r)
+    out: dict[str, Any] = {"id": d["id"], "row_index": d["row_index"]}
+    for col in csv_cols:
+        fn = COMMENT_CSV_TO_FIELD[col]
+        out[col] = d.get(fn, "")
+    return out
+
+
+def _merged_row_csv_dict(r: JdJobMergedRow, internal_keys: list[str]) -> dict[str, Any]:
+    d = merged_row_to_dict(r)
+    out: dict[str, Any] = {"id": d["id"], "row_index": d["row_index"]}
+    for k in internal_keys:
+        out[MERGED_FIELD_TO_CSV_HEADER[k]] = d.get(k, "")
+    return out
+
+
+def _prune_search_dict(d: dict[str, Any], internal_keys: list[str]) -> dict[str, Any]:
+    out = {"id": d["id"], "row_index": d["row_index"]}
+    for k in internal_keys:
+        out[k] = d.get(k, "")
+    return out
+
+
+def _prune_detail_dict(d: dict[str, Any], fields: list[str]) -> dict[str, Any]:
+    out = {"id": d["id"], "row_index": d["row_index"]}
+    for k in fields:
+        out[k] = d.get(k, "")
+    return out
+
+
+def _prune_comment_dict(d: dict[str, Any], fields: list[str]) -> dict[str, Any]:
+    out = {"id": d["id"], "row_index": d["row_index"]}
+    for k in fields:
+        out[k] = d.get(k, "")
+    return out
+
+
+def _prune_merged_dict(d: dict[str, Any], fields: list[str]) -> dict[str, Any]:
+    out = {"id": d["id"], "row_index": d["row_index"]}
+    for k in fields:
+        out[k] = d.get(k, "")
+    return out
+
+
+def _rows_as_list_search(job: PipelineJob) -> list[dict[str, Any]]:
+    keys = nonempty_search_keys_for_job(job)
+    qs = JdJobSearchRow.objects.filter(job=job)
+    return [
+        _prune_search_dict(search_row_to_dict(obj), keys)
+        for obj in qs.order_by("row_index").iterator(chunk_size=400)
+    ]
+
+
+def _rows_as_list_detail(job: PipelineJob) -> list[dict[str, Any]]:
+    fields = nonempty_detail_fields_for_job(job)
+    qs = JdJobDetailRow.objects.filter(job=job)
+    return [
+        _prune_detail_dict(detail_row_to_dict(obj), fields)
+        for obj in qs.order_by("row_index").iterator(chunk_size=400)
+    ]
+
+
+def _rows_as_list_comment(job: PipelineJob) -> list[dict[str, Any]]:
+    fields = nonempty_comment_fields_for_job(job)
+    qs = JdJobCommentRow.objects.filter(job=job)
+    return [
+        _prune_comment_dict(comment_row_to_dict(obj), fields)
+        for obj in qs.order_by("row_index").iterator(chunk_size=400)
+    ]
+
+
+def _rows_as_list_merged(job: PipelineJob) -> list[dict[str, Any]]:
+    fields = nonempty_merged_fields_for_job(job)
+    qs = JdJobMergedRow.objects.filter(job=job)
+    return [
+        _prune_merged_dict(merged_row_to_dict(obj), fields)
+        for obj in qs.order_by("row_index").iterator(chunk_size=400)
+    ]
+
+
+def build_json_bytes(*, job: PipelineJob, kind: str) -> tuple[bytes, str]:
+    if kind == "search":
+        data = _rows_as_list_search(job)
+        name = f"job_{job.id}_search.json"
+    elif kind == "detail":
+        data = _rows_as_list_detail(job)
+        name = f"job_{job.id}_detail.json"
+    elif kind == "comments":
+        data = _rows_as_list_comment(job)
+        name = f"job_{job.id}_comments.json"
+    elif kind == "all":
+        data = {
+            "job_id": job.id,
+            "keyword": job.keyword,
+            "search": _rows_as_list_search(job),
+            "detail": _rows_as_list_detail(job),
+            "comments": _rows_as_list_comment(job),
+            "merged": _rows_as_list_merged(job),
+        }
+        name = f"job_{job.id}_all.json"
+    elif kind == "merged":
+        data = _rows_as_list_merged(job)
+        name = f"job_{job.id}_merged.json"
+    else:
+        raise ValueError(f"unknown kind: {kind}")
+    raw = json.dumps(data, ensure_ascii=False, indent=2)
+    return raw.encode("utf-8"), name
+
+
+def _write_csv_from_qs(
+    *,
+    qs: QuerySet,
+    headers: list[str],
+    row_fn: Any,
+) -> str:
+    buf = StringIO()
+    w = csv.DictWriter(buf, fieldnames=headers, extrasaction="ignore")
+    w.writeheader()
+    for obj in qs.order_by("row_index").iterator(chunk_size=400):
+        w.writerow(row_fn(obj))
+    return buf.getvalue()
+
+
+def build_csv_bytes(*, job: PipelineJob, kind: str) -> tuple[bytes, str]:
+    if kind == "search":
+        sk = nonempty_search_keys_for_job(job)
+        text = _write_csv_from_qs(
+            qs=JdJobSearchRow.objects.filter(job=job),
+            headers=search_export_headers(job),
+            row_fn=lambda o, _sk=sk: _search_row_csv_dict(o, _sk),
+        )
+        name = f"job_{job.id}_search.csv"
+    elif kind == "detail":
+        dcols = [c for c in detail_export_headers(job) if c not in ("id", "row_index")]
+        text = _write_csv_from_qs(
+            qs=JdJobDetailRow.objects.filter(job=job),
+            headers=detail_export_headers(job),
+            row_fn=lambda o, _dc=dcols: _detail_row_csv_dict(o, _dc),
+        )
+        name = f"job_{job.id}_detail.csv"
+    elif kind == "comments":
+        ccols = [c for c in comment_export_headers(job) if c not in ("id", "row_index")]
+        text = _write_csv_from_qs(
+            qs=JdJobCommentRow.objects.filter(job=job),
+            headers=comment_export_headers(job),
+            row_fn=lambda o, _cc=ccols: _comment_row_csv_dict(o, _cc),
+        )
+        name = f"job_{job.id}_comments.csv"
+    elif kind == "all":
+        sk = nonempty_search_keys_for_job(job)
+        dcols = [c for c in detail_export_headers(job) if c not in ("id", "row_index")]
+        ccols = [c for c in comment_export_headers(job) if c not in ("id", "row_index")]
+        mk = nonempty_merged_fields_for_job(job)
+        parts = [
+            "# search",
+            _write_csv_from_qs(
+                qs=JdJobSearchRow.objects.filter(job=job),
+                headers=search_export_headers(job),
+                row_fn=lambda o, _sk=sk: _search_row_csv_dict(o, _sk),
+            ),
+            "",
+            "# detail",
+            _write_csv_from_qs(
+                qs=JdJobDetailRow.objects.filter(job=job),
+                headers=detail_export_headers(job),
+                row_fn=lambda o, _dc=dcols: _detail_row_csv_dict(o, _dc),
+            ),
+            "",
+            "# comments",
+            _write_csv_from_qs(
+                qs=JdJobCommentRow.objects.filter(job=job),
+                headers=comment_export_headers(job),
+                row_fn=lambda o, _cc=ccols: _comment_row_csv_dict(o, _cc),
+            ),
+            "",
+            "# merged",
+            _write_csv_from_qs(
+                qs=JdJobMergedRow.objects.filter(job=job),
+                headers=merged_export_headers(job),
+                row_fn=lambda o, _mk=mk: _merged_row_csv_dict(o, _mk),
+            ),
+        ]
+        text = "\n".join(parts)
+        name = f"job_{job.id}_all.csv"
+    elif kind == "merged":
+        mk = nonempty_merged_fields_for_job(job)
+        text = _write_csv_from_qs(
+            qs=JdJobMergedRow.objects.filter(job=job),
+            headers=merged_export_headers(job),
+            row_fn=lambda o, _mk=mk: _merged_row_csv_dict(o, _mk),
+        )
+        name = f"job_{job.id}_merged.csv"
+    else:
+        raise ValueError(f"unknown kind: {kind}")
+    return ("\ufeff" + text).encode("utf-8"), name
+
+
+def _append_sheet(ws, headers: list[str], qs: QuerySet, row_fn: Any) -> None:
+    ws.append(headers)
+    for obj in qs.order_by("row_index").iterator(chunk_size=400):
+        rowd = row_fn(obj)
+        ws.append([rowd.get(h, "") for h in headers])
+
+
+def build_xlsx_bytes(*, job: PipelineJob, kind: str) -> tuple[bytes, str]:
+    wb = Workbook()
+    if kind == "search":
+        ws = wb.active
+        ws.title = "search"[:31]
+        sk = nonempty_search_keys_for_job(job)
+        _append_sheet(
+            ws,
+            search_export_headers(job),
+            JdJobSearchRow.objects.filter(job=job),
+            lambda o, _sk=sk: _search_row_csv_dict(o, _sk),
+        )
+        name = f"job_{job.id}_search.xlsx"
+    elif kind == "detail":
+        ws = wb.active
+        ws.title = "detail"[:31]
+        dcols = [c for c in detail_export_headers(job) if c not in ("id", "row_index")]
+        _append_sheet(
+            ws,
+            detail_export_headers(job),
+            JdJobDetailRow.objects.filter(job=job),
+            lambda o, _dc=dcols: _detail_row_csv_dict(o, _dc),
+        )
+        name = f"job_{job.id}_detail.xlsx"
+    elif kind == "comments":
+        ws = wb.active
+        ws.title = "comments"[:31]
+        ccols = [c for c in comment_export_headers(job) if c not in ("id", "row_index")]
+        _append_sheet(
+            ws,
+            comment_export_headers(job),
+            JdJobCommentRow.objects.filter(job=job),
+            lambda o, _cc=ccols: _comment_row_csv_dict(o, _cc),
+        )
+        name = f"job_{job.id}_comments.xlsx"
+    elif kind == "all":
+        sk = nonempty_search_keys_for_job(job)
+        dcols = [c for c in detail_export_headers(job) if c not in ("id", "row_index")]
+        ccols = [c for c in comment_export_headers(job) if c not in ("id", "row_index")]
+        mk = nonempty_merged_fields_for_job(job)
+        ws1 = wb.active
+        ws1.title = "search"[:31]
+        _append_sheet(
+            ws1,
+            search_export_headers(job),
+            JdJobSearchRow.objects.filter(job=job),
+            lambda o, _sk=sk: _search_row_csv_dict(o, _sk),
+        )
+        ws2 = wb.create_sheet("detail"[:31])
+        _append_sheet(
+            ws2,
+            detail_export_headers(job),
+            JdJobDetailRow.objects.filter(job=job),
+            lambda o, _dc=dcols: _detail_row_csv_dict(o, _dc),
+        )
+        ws3 = wb.create_sheet("comments"[:31])
+        _append_sheet(
+            ws3,
+            comment_export_headers(job),
+            JdJobCommentRow.objects.filter(job=job),
+            lambda o, _cc=ccols: _comment_row_csv_dict(o, _cc),
+        )
+        ws4 = wb.create_sheet("merged"[:31])
+        _append_sheet(
+            ws4,
+            merged_export_headers(job),
+            JdJobMergedRow.objects.filter(job=job),
+            lambda o, _mk=mk: _merged_row_csv_dict(o, _mk),
+        )
+        name = f"job_{job.id}_all.xlsx"
+    elif kind == "merged":
+        mk = nonempty_merged_fields_for_job(job)
+        ws = wb.active
+        ws.title = "merged"[:31]
+        _append_sheet(
+            ws,
+            merged_export_headers(job),
+            JdJobMergedRow.objects.filter(job=job),
+            lambda o, _mk=mk: _merged_row_csv_dict(o, _mk),
+        )
+        name = f"job_{job.id}_merged.xlsx"
+    else:
+        raise ValueError(f"unknown kind: {kind}")
+    bio = BytesIO()
+    wb.save(bio)
+    return bio.getvalue(), name

backend/pipeline/ingest.py

@@ -0,0 +1,278 @@
+"""
+任务成功后入库：
+- 搜索导出 / 商详导出 / 评价扁平：按任务分表存储，便于分页与导出；
+- 合并表：更新全局 ``JdProduct`` + 任务维度 ``JdProductSnapshot``。
+"""
+from __future__ import annotations
+
+import csv
+import logging
+from pathlib import Path
+from typing import Any
+
+from django.db import transaction
+from django.utils import timezone
+
+from .csv_schema import (
+    COMMENT_CSV_COLUMNS,
+    COMMENT_CSV_TO_FIELD,
+    DETAIL_CSV_COLUMNS,
+    DETAIL_CSV_TO_FIELD,
+    JD_SEARCH_CSV_HEADERS,
+    JD_SEARCH_INTERNAL_KEYS,
+    MERGED_CSV_COLUMNS,
+    MERGED_CSV_TO_FIELD,
+    SEARCH_CSV_HEADER_TO_FIELD,
+)
+from .models import (
+    JdJobCommentRow,
+    JdJobDetailRow,
+    JdJobMergedRow,
+    JdJobSearchRow,
+    JdProduct,
+    JdProductSnapshot,
+    PipelineJob,
+)
+
+logger = logging.getLogger(__name__)
+
+FILE_MERGED_CSV = "keyword_pipeline_merged.csv"
+FILE_PC_SEARCH_CSV = "pc_search_export.csv"
+FILE_DETAIL_WARE_CSV = "detail_ware_export.csv"
+FILE_COMMENTS_FLAT_CSV = "comments_flat.csv"
+
+SKU_FIELD_MERGED = "SKU(skuId)"
+WARE_FIELD = "主商品ID(wareId)"
+TITLE_FIELD = "标题(wareName)"
+
+BULK_CHUNK = 400
+
+
+def _read_csv_rows(path: Path) -> list[dict[str, str]]:
+    if not path.is_file():
+        return []
+    raw = path.read_text(encoding="utf-8-sig")
+    lines = raw.splitlines()
+    if not lines:
+        return []
+    return list(csv.DictReader(lines))
+
+
+def _payload_as_json(row: dict[str, str]) -> dict[str, str]:
+    return {str(k): str(v) if v is not None else "" for k, v in row.items()}
+
+
+def _search_row_kwargs(row: dict[str, str]) -> dict[str, str]:
+    vals = {k: "" for k in JD_SEARCH_INTERNAL_KEYS}
+    for csv_header, cell in row.items():
+        h = (csv_header or "").strip()
+        fn = SEARCH_CSV_HEADER_TO_FIELD.get(h)
+        if fn:
+            vals[fn] = str(cell or "").strip()
+    return vals
+
+
+def _detail_row_kwargs(row: dict[str, str]) -> dict[str, str]:
+    return {
+        DETAIL_CSV_TO_FIELD[col]: str(row.get(col) or "").strip() for col in DETAIL_CSV_COLUMNS
+    }
+
+
+def _comment_row_kwargs(row: dict[str, str]) -> dict[str, str]:
+    return {
+        COMMENT_CSV_TO_FIELD[col]: str(row.get(col) or "").strip() for col in COMMENT_CSV_COLUMNS
+    }
+
+
+def _merged_row_kwargs(row: dict[str, str]) -> dict[str, str]:
+    return {
+        MERGED_CSV_TO_FIELD[col]: str(row.get(col) or "").strip() for col in MERGED_CSV_COLUMNS
+    }
+
+
+def _bulk_create_in_chunks(model, objects: list[Any]) -> None:
+    for i in range(0, len(objects), BULK_CHUNK):
+        model.objects.bulk_create(objects[i : i + BULK_CHUNK])
+
+
+def _run_dir(job: PipelineJob) -> Path:
+    return Path(job.run_dir or "").expanduser().resolve()
+
+
+def ingest_job_dataset_rows(job: PipelineJob) -> dict[str, Any]:
+    """
+    删除该任务旧数据后，将 ``pc_search_export`` / ``detail_ware_export`` / ``comments_flat`` 全量写入数据库。
+    """
+    if not (job.run_dir or "").strip():
+        raise FileNotFoundError("任务无 run_dir")
+
+    run_dir = _run_dir(job)
+    stats: dict[str, Any] = {
+        "search_rows": 0,
+        "detail_rows": 0,
+        "comment_rows": 0,
+        "merged_table_rows": 0,
+    }
+
+    JdJobSearchRow.objects.filter(job=job).delete()
+    JdJobDetailRow.objects.filter(job=job).delete()
+    JdJobCommentRow.objects.filter(job=job).delete()
+    JdJobMergedRow.objects.filter(job=job).delete()
+
+    search_path = run_dir / FILE_PC_SEARCH_CSV
+    search_rows = _read_csv_rows(search_path)
+    if not search_rows and search_path.is_file() is False:
+        pass
+    s_objs: list[JdJobSearchRow] = []
+    for i, row in enumerate(search_rows):
+        kw = _search_row_kwargs(row)
+        s_objs.append(JdJobSearchRow(job=job, row_index=i, **kw))
+    _bulk_create_in_chunks(JdJobSearchRow, s_objs)
+    stats["search_rows"] = len(s_objs)
+
+    detail_path = run_dir / FILE_DETAIL_WARE_CSV
+    detail_rows = _read_csv_rows(detail_path)
+    d_objs: list[JdJobDetailRow] = []
+    for i, row in enumerate(detail_rows):
+        kw = _detail_row_kwargs(row)
+        d_objs.append(JdJobDetailRow(job=job, row_index=i, **kw))
+    _bulk_create_in_chunks(JdJobDetailRow, d_objs)
+    stats["detail_rows"] = len(d_objs)
+
+    comment_path = run_dir / FILE_COMMENTS_FLAT_CSV
+    comment_rows = _read_csv_rows(comment_path)
+    c_objs: list[JdJobCommentRow] = []
+    for i, row in enumerate(comment_rows):
+        kw = _comment_row_kwargs(row)
+        c_objs.append(JdJobCommentRow(job=job, row_index=i, **kw))
+    _bulk_create_in_chunks(JdJobCommentRow, c_objs)
+    stats["comment_rows"] = len(c_objs)
+
+    merged_path = run_dir / FILE_MERGED_CSV
+    merged_rows = _read_csv_rows(merged_path) if merged_path.is_file() else []
+    m_objs: list[JdJobMergedRow] = []
+    for i, row in enumerate(merged_rows):
+        kw = _merged_row_kwargs(row)
+        m_objs.append(JdJobMergedRow(job=job, row_index=i, **kw))
+    _bulk_create_in_chunks(JdJobMergedRow, m_objs)
+    stats["merged_table_rows"] = len(m_objs)
+
+    return stats
+
+
+def ingest_job_merged_csv(job: PipelineJob) -> dict[str, Any]:
+    """
+    读取合并表，upsert ``JdProduct``，并按 (商品, 任务) 写入 ``JdProductSnapshot``。
+    """
+    run_dir = _run_dir(job)
+    path = run_dir / FILE_MERGED_CSV
+    if not path.is_file():
+        raise FileNotFoundError(f"合并表不存在: {path}")
+
+    rows = _read_csv_rows(path)
+    captured_at = job.updated_at or timezone.now()
+    stats = {
+        "merged_file": str(path),
+        "rows_in_csv": len(rows),
+        "rows_ingested": 0,
+        "products_created": 0,
+        "snapshots_upserted": 0,
+    }
+
+    platform = (job.platform or "jd").strip() or "jd"
+
+    for row in rows:
+        sku = (row.get(SKU_FIELD_MERGED) or "").strip()
+        if not sku:
+            continue
+        payload = _payload_as_json(row)
+        title = (row.get(TITLE_FIELD) or "")[:2000]
+        ware = (row.get(WARE_FIELD) or "").strip()[:64]
+        brand = (row.get("detail_brand") or "").strip()[:512]
+        price = (
+            (row.get("detail_price_final") or "").strip()
+            or (row.get(JD_SEARCH_CSV_HEADERS["coupon_price"]) or "").strip()
+            or (row.get(JD_SEARCH_CSV_HEADERS["price"]) or "").strip()
+        )[:128]
+        cat = (
+            (row.get("detail_category_path") or "").strip()
+            or (row.get(JD_SEARCH_CSV_HEADERS["leaf_category"]) or "").strip()
+        )[:2000]
+
+        product, created = JdProduct.objects.get_or_create(
+            platform=platform,
+            sku_id=sku,
+            defaults={
+                "ware_id": ware,
+                "title": title,
+                "detail_brand": brand,
+                "detail_price_final": price,
+                "detail_category_path": cat,
+                "current_payload": payload,
+                "last_job": job,
+                "last_captured_at": captured_at,
+            },
+        )
+        if created:
+            stats["products_created"] += 1
+        else:
+            product.ware_id = ware or product.ware_id
+            product.title = title or product.title
+            product.detail_brand = brand
+            product.detail_price_final = price
+            product.detail_category_path = cat
+            product.current_payload = payload
+            product.last_job = job
+            product.last_captured_at = captured_at
+            product.save(
+                update_fields=[
+                    "ware_id",
+                    "title",
+                    "detail_brand",
+                    "detail_price_final",
+                    "detail_category_path",
+                    "current_payload",
+                    "last_job",
+                    "last_captured_at",
+                    "updated_at",
+                ]
+            )
+
+        JdProductSnapshot.objects.update_or_create(
+            product=product,
+            job=job,
+            defaults={
+                "run_dir": job.run_dir or "",
+                "captured_at": captured_at,
+                "payload": payload,
+            },
+        )
+        stats["snapshots_upserted"] += 1
+        stats["rows_ingested"] += 1
+
+    return stats
+
+
+def ingest_job_full(job: PipelineJob) -> dict[str, Any]:
+    """
+    先提交搜索/详情/评论（与 CSV 行一一对应），再单独提交合并表主档与快照。
+    合并表缺失时仍保留前三类数据，便于仅用列表/评价做回顾。
+    """
+    out: dict[str, Any] = {}
+    with transaction.atomic():
+        out["dataset"] = ingest_job_dataset_rows(job)
+    try:
+        with transaction.atomic():
+            out["merged"] = ingest_job_merged_csv(job)
+    except FileNotFoundError as e:
+        logger.warning("ingest merged skipped job=%s: %s", job.id, e)
+        out["merged"] = {"error": str(e), "rows_ingested": 0, "snapshots_upserted": 0}
+    return out
+
+
+def try_ingest_job_full(job: PipelineJob) -> None:
+    try:
+        stats = ingest_job_full(job)
+        logger.info("ingest_job_full job=%s %s", job.id, stats)
+    except Exception:
+        logger.exception("ingest_job_full failed job=%s", job.id)

backend/pipeline/jd_runner.py

@@ -0,0 +1,490 @@
+"""
+使用 ``crawler_copy/jd_pc_search`` 中的副本脚本执行流水线并生成竞品 Markdown。
+依赖环境变量 ``LOW_GI_PROJECT_ROOT``（由 Django settings 从 ``market_assistant/.env`` 注入）。
+"""
+from __future__ import annotations
+
+import json
+import os
+import re
+import sys
+from pathlib import Path
+from typing import Any
+
+from django.conf import settings
+
+from .models import PipelineJob
+
+
+def merge_llm_supplement_with_rules_report(llm_md: str, rules_md: str) -> str:
+    """
+    **以规则引擎全文为正文**（含 §5 完整竞品矩阵、各章内嵌统计图与表格）。
+
+    大模型稿仅作为开篇「速读/策略补充」插入在「## 一、」之前，**不得**再用纯 LLM 稿
+    覆盖规则正文（否则会丢失矩阵与章节结构）。
+    """
+    body = (rules_md or "").strip()
+    sup = (llm_md or "").strip()
+    if not sup:
+        return body
+    if not body:
+        return sup
+    block = (
+        "---\n\n"
+        "## 大模型速读与策略要点（补充）\n\n"
+        "> **说明**：以下由大模型依据结构化摘要生成，便于速览；**完整竞品对比矩阵、全部表格、"
+        "统计图与定量口径以正文各章（尤其 §5）为准**，请勿仅依据本段理解 SKU 明细。\n\n"
+        f"{sup}\n"
+    )
+    marker = "\n---\n\n## 一、研究范围、数据来源与局限"
+    if marker in body:
+        return body.replace(marker, "\n" + block + marker, 1)
+    return block + "\n---\n\n" + body
+
+
+def merge_llm_report_with_rules_charts(llm_md: str, rules_md: str) -> str:
+    """兼容旧名：等价于 ``merge_llm_supplement_with_rules_report``。"""
+    return merge_llm_supplement_with_rules_report(llm_md, rules_md)
+
+
+def _flat_comment_texts(comment_rows: list[dict[str, str]]) -> list[str]:
+    """全部非空评价正文（与报告统计同源）。"""
+    out: list[str] = []
+    for row in comment_rows:
+        t = (row.get("tagCommentContent") or "").strip()
+        if t:
+            out.append(t)
+    return out
+
+
+def _safe_dir_segment_for_job(s: str, max_len: int = 48) -> str:
+    """与 ``jd_keyword_pipeline._safe_dir_segment`` 一致，避免多线程下改模块全局。"""
+    bad = '<>:"/\\|?*\n\r\t'
+    t = "".join("_" if c in bad else c for c in (s or "").strip())[:max_len]
+    t = t.strip(" .") or "run"
+    return t
+
+
+def resolve_pipeline_run_directory_for_job(job: PipelineJob) -> Path:
+    """
+    在拉起子进程前固定本次 ``run_dir``（与 ``jd_keyword_pipeline._resolve_pipeline_run_dir`` 同口径）。
+    调用方负责 ``mkdir``。
+    """
+    root = (settings.LOW_GI_PROJECT_ROOT or "").strip()
+    if not root:
+        raise RuntimeError("LOW_GI_PROJECT_ROOT 未配置")
+    project_data = Path(root).resolve() / "data" / "JD"
+    prd = (job.pipeline_run_dir or "").strip()
+    kw = (job.keyword or "").strip()
+    if prd:
+        p = Path(prd).expanduser()
+        if not p.is_absolute():
+            p = project_data / p
+        return p.resolve()
+    import time
+
+    stamp = time.strftime("%Y%m%d_%H%M%S")
+    seg = _safe_dir_segment_for_job(kw)
+    return (project_data / "pipeline_runs" / f"{stamp}_{seg}").resolve()
+
+
+def try_write_competitor_report_if_merged_exists(
+    run_dir: Path,
+    keyword: str,
+    *,
+    report_config: dict[str, Any] | None = None,
+) -> None:
+    """若已有合并表则补写竞品 Markdown（用于子进程被 terminate 后的部分产物）。"""
+    _, kpl = _jd_crawler_modules()
+    base = Path(run_dir).resolve()
+    merged = base / kpl.FILE_MERGED_CSV
+    if not merged.is_file():
+        return
+    try:
+        write_competitor_analysis_for_run_dir(
+            base, keyword, report_config=report_config
+        )
+    except Exception:
+        pass
+
+
+def _jd_crawler_modules():
+    root = Path(settings.CRAWLER_JD_ROOT)
+    if not root.is_dir():
+        raise FileNotFoundError(f"爬虫副本目录不存在: {root}")
+    root_s = str(root.resolve())
+    if root_s not in sys.path:
+        sys.path.insert(0, root_s)
+    import jd_competitor_report as jcr  # noqa: WPS433
+    import jd_keyword_pipeline as kpl  # noqa: WPS433
+
+    return jcr, kpl
+
+
+def get_default_report_config() -> dict[str, Any]:
+    """与 ``jd_competitor_report`` 模块常量一致的默认报告调参（供前端回填）。"""
+    jcr, _ = _jd_crawler_modules()
+    return {
+        "llm_comment_sentiment": False,
+        "comment_focus_words": list(jcr.COMMENT_FOCUS_WORDS),
+        "comment_scenario_groups": [
+            {"label": lbl, "triggers": list(trs)}
+            for lbl, trs in jcr.COMMENT_SCENARIO_GROUPS
+        ],
+        "external_market_table_rows": [
+            {"indicator": a, "value_and_scope": b, "source": c, "year": d}
+            for a, b, c, d in jcr.EXTERNAL_MARKET_TABLE_ROWS
+        ],
+    }
+
+
+def write_competitor_analysis_for_run_dir(
+    run_dir: Path,
+    keyword: str,
+    *,
+    report_config: dict[str, Any] | None = None,
+) -> Path:
+    """
+    在已有流水线目录上读取 CSV / meta，写入 ``competitor_analysis.md``（不重新爬取）。
+    """
+    jcr, kpl = _jd_crawler_modules()
+    kw = (keyword or "").strip()
+    if not kw:
+        raise ValueError("keyword 不能为空")
+
+    run_dir = Path(run_dir).resolve()
+    merged_path = run_dir / kpl.FILE_MERGED_CSV
+    if not merged_path.is_file():
+        raise FileNotFoundError(f"缺少合并表，无法生成报告: {merged_path.name}")
+
+    _, merged_rows = jcr._read_csv_rows(merged_path)
+    _, search_export_rows = jcr._read_csv_rows(run_dir / kpl.FILE_PC_SEARCH_CSV)
+    _, comment_rows = jcr._read_csv_rows(run_dir / kpl.FILE_COMMENTS_FLAT_CSV)
+
+    meta_path = run_dir / kpl.FILE_RUN_META_JSON
+    meta: dict[str, Any] | None = None
+    if meta_path.is_file():
+        try:
+            meta = json.loads(meta_path.read_text(encoding="utf-8"))
+        except json.JSONDecodeError:
+            meta = None
+
+    eff_rc: dict[str, Any] = (
+        dict(report_config) if isinstance(report_config, dict) else {}
+    )
+    all_tx = _flat_comment_texts(comment_rows)
+    suggest_path = run_dir / "keyword_suggest_llm.json"
+    suggest_record: dict[str, Any] = {
+        "schema_version": 2,
+        "total_comment_texts": len(all_tx),
+    }
+    skip_kw = os.environ.get("MA_SKIP_LLM_KEYWORD_SUGGEST", "").strip().lower() in (
+        "1",
+        "true",
+        "yes",
+    )
+    if not skip_kw:
+        try:
+            from .llm_keyword_suggest import suggest_focus_keywords_from_all_comments
+
+            brief_pre = jcr.build_competitor_brief(
+                run_dir=run_dir,
+                keyword=kw,
+                merged_rows=merged_rows,
+                search_export_rows=search_export_rows,
+                comment_rows=comment_rows,
+                meta=meta,
+                report_config=eff_rc,
+            )
+            brief_slice = {
+                "keyword": brief_pre.get("keyword"),
+                "comment_focus_keywords": (
+                    brief_pre.get("comment_focus_keywords") or []
+                )[:20],
+                "usage_scenarios": (brief_pre.get("usage_scenarios") or [])[:8],
+                "category_mix_top": (brief_pre.get("category_mix_top") or [])[:6],
+                "scope": brief_pre.get("scope"),
+            }
+            sug = suggest_focus_keywords_from_all_comments(
+                keyword=kw,
+                brief_slice=brief_slice,
+                all_comment_texts=all_tx,
+            )
+            suggest_record.update(sug)
+            base_words = list(eff_rc.get("comment_focus_words") or [])
+            for w in sug.get("suggested_focus_keywords") or []:
+                if isinstance(w, str):
+                    t = w.strip()
+                    if t and t not in base_words:
+                        base_words.append(t)
+            eff_rc["comment_focus_words"] = base_words[:80]
+        except Exception as e:
+            suggest_record["error"] = str(e)
+            suggest_record["suggested_focus_keywords"] = []
+    else:
+        suggest_record["skipped"] = True
+        suggest_record["suggested_focus_keywords"] = []
+
+    suggest_path.write_text(
+        json.dumps(suggest_record, ensure_ascii=False, indent=2),
+        encoding="utf-8",
+    )
+    (run_dir / "effective_report_config.json").write_text(
+        json.dumps(eff_rc, ensure_ascii=False, indent=2),
+        encoding="utf-8",
+    )
+
+    brief_final = jcr.build_competitor_brief(
+        run_dir=run_dir,
+        keyword=kw,
+        merged_rows=merged_rows,
+        search_export_rows=search_export_rows,
+        comment_rows=comment_rows,
+        meta=meta,
+        report_config=eff_rc,
+    )
+    from .report_charts import generate_report_charts
+
+    generate_report_charts(run_dir, brief_final)
+
+    llm_sentiment_md = ""
+    sentiment_llm_record: dict[str, Any] = {
+        "schema_version": 1,
+        "attempted": False,
+    }
+    skip_sent = os.environ.get(
+        "MA_SKIP_LLM_COMMENT_SENTIMENT", ""
+    ).strip().lower() in ("1", "true", "yes")
+    env_on = os.environ.get("MA_ENABLE_LLM_COMMENT_SENTIMENT", "").strip().lower() in (
+        "1",
+        "true",
+        "yes",
+    )
+    want_sent = bool(eff_rc.get("llm_comment_sentiment")) or env_on
+    if want_sent and not skip_sent:
+        comment_units = jcr._iter_comment_text_units(comment_rows, merged_rows)
+        if len(comment_units) >= 2:
+            sentiment_llm_record["attempted"] = True
+            try:
+                from .llm_generate import generate_comment_sentiment_analysis_llm
+
+                pl = jcr.build_comment_sentiment_llm_payload(comment_units)
+                pl["keyword"] = kw
+                llm_sentiment_md = generate_comment_sentiment_analysis_llm(pl)
+                sentiment_llm_record["ok"] = True
+                sentiment_llm_record["chars"] = len(llm_sentiment_md)
+            except Exception as e:
+                sentiment_llm_record["ok"] = False
+                sentiment_llm_record["error"] = str(e)
+        else:
+            sentiment_llm_record["skipped"] = "insufficient_comment_texts"
+    elif skip_sent:
+        sentiment_llm_record["skipped"] = "MA_SKIP_LLM_COMMENT_SENTIMENT"
+    elif not want_sent:
+        sentiment_llm_record["skipped"] = "not_enabled"
+
+    (run_dir / "comment_sentiment_llm.json").write_text(
+        json.dumps(sentiment_llm_record, ensure_ascii=False, indent=2),
+        encoding="utf-8",
+    )
+
+    md = jcr.build_competitor_markdown(
+        run_dir=run_dir,
+        keyword=kw,
+        merged_rows=merged_rows,
+        search_export_rows=search_export_rows,
+        comment_rows=comment_rows,
+        meta=meta,
+        report_config=eff_rc,
+        llm_sentiment_section_md=llm_sentiment_md or None,
+    )
+    out_md = run_dir / "competitor_analysis.md"
+    out_md.write_text(md, encoding="utf-8")
+    return run_dir
+
+
+def regenerate_competitor_report(
+    run_dir_str: str,
+    keyword: str,
+    *,
+    report_config: dict[str, Any] | None = None,
+) -> Path:
+    """校验 ``run_dir`` 位于 ``LOW_GI_PROJECT_ROOT/data/JD`` 下后，重写竞品 Markdown。"""
+    low_root = (settings.LOW_GI_PROJECT_ROOT or "").strip()
+    if not low_root:
+        raise RuntimeError("LOW_GI_PROJECT_ROOT 未配置")
+    base = Path(run_dir_str).expanduser().resolve()
+    jd_root = (Path(low_root) / "data" / "JD").resolve()
+    try:
+        base.relative_to(jd_root)
+    except ValueError as e:
+        raise ValueError("run_dir 不在京东数据目录下") from e
+    return write_competitor_analysis_for_run_dir(
+        base, keyword, report_config=report_config
+    )
+
+
+def write_competitor_analysis_markdown(run_dir_str: str, markdown: str) -> Path:
+    """将已生成的 Markdown 正文写入 ``run_dir/competitor_analysis.md``（与规则重生成同路径）。"""
+    low_root = (settings.LOW_GI_PROJECT_ROOT or "").strip()
+    if not low_root:
+        raise RuntimeError("LOW_GI_PROJECT_ROOT 未配置")
+    base = Path(run_dir_str).expanduser().resolve()
+    jd_root = (Path(low_root) / "data" / "JD").resolve()
+    try:
+        base.relative_to(jd_root)
+    except ValueError as e:
+        raise ValueError("run_dir 不在京东数据目录下") from e
+    out = base / "competitor_analysis.md"
+    out.write_text(markdown or "", encoding="utf-8")
+    return out
+
+
+def build_competitor_brief_for_job(
+    run_dir_str: str,
+    keyword: str,
+    *,
+    report_config: dict[str, Any] | None = None,
+) -> dict[str, Any]:
+    """
+    读取 ``run_dir`` 下合并表 / 搜索导出 / 评价 / meta，返回与 Markdown 报告同口径的 **JSON 结构化摘要**（规则驱动）。
+    ``run_dir`` 须位于 ``LOW_GI_PROJECT_ROOT/data/JD`` 下。
+    """
+    low_root = (settings.LOW_GI_PROJECT_ROOT or "").strip()
+    if not low_root:
+        raise RuntimeError("LOW_GI_PROJECT_ROOT 未配置")
+    base = Path(run_dir_str).expanduser().resolve()
+    jd_root = (Path(low_root) / "data" / "JD").resolve()
+    try:
+        base.relative_to(jd_root)
+    except ValueError as e:
+        raise ValueError("run_dir 不在京东数据目录下") from e
+
+    jcr, kpl = _jd_crawler_modules()
+    kw = (keyword or "").strip()
+    if not kw:
+        raise ValueError("keyword 不能为空")
+
+    merged_path = base / kpl.FILE_MERGED_CSV
+    if not merged_path.is_file():
+        raise FileNotFoundError(f"缺少合并表，无法生成摘要: {merged_path.name}")
+
+    _, merged_rows = jcr._read_csv_rows(merged_path)
+    _, search_export_rows = jcr._read_csv_rows(base / kpl.FILE_PC_SEARCH_CSV)
+    _, comment_rows = jcr._read_csv_rows(base / kpl.FILE_COMMENTS_FLAT_CSV)
+
+    meta_path = base / kpl.FILE_RUN_META_JSON
+    meta: dict[str, Any] | None = None
+    if meta_path.is_file():
+        try:
+            meta = json.loads(meta_path.read_text(encoding="utf-8"))
+        except json.JSONDecodeError:
+            meta = None
+
+    eff: dict[str, Any] | None = None
+    if isinstance(report_config, dict):
+        eff = dict(report_config)
+    eff_path = base / "effective_report_config.json"
+    if eff_path.is_file():
+        try:
+            loaded = json.loads(eff_path.read_text(encoding="utf-8"))
+            if isinstance(loaded, dict) and loaded:
+                eff = loaded
+        except json.JSONDecodeError:
+            pass
+
+    return jcr.build_competitor_brief(
+        run_dir=base,
+        keyword=kw,
+        merged_rows=merged_rows,
+        search_export_rows=search_export_rows,
+        comment_rows=comment_rows,
+        meta=meta,
+        report_config=eff,
+    )
+
+
+def run_jd_keyword_and_report(
+    keyword: str,
+    *,
+    max_skus: int | None = None,
+    page_start: int | None = None,
+    page_to: int | None = None,
+    pipeline_run_dir: str | None = None,
+    cookie_file_path: str | None = None,
+    pvid: str | None = None,
+    request_delay: str | None = None,
+    list_pages: str | None = None,
+    scenario_filter_enabled: bool | None = None,
+    report_config: dict[str, Any] | None = None,
+    cancel_check: Any | None = None,
+) -> Path:
+    _, kpl = _jd_crawler_modules()
+
+    kw = (keyword or "").strip()
+    if not kw:
+        raise ValueError("keyword 不能为空")
+
+    backup: dict[str, Any] = {}
+    if cancel_check is not None:
+        backup["PIPELINE_CANCEL_CHECK"] = getattr(kpl, "PIPELINE_CANCEL_CHECK", None)
+        kpl.PIPELINE_CANCEL_CHECK = cancel_check
+    try:
+        if max_skus is not None:
+            backup["MAX_SKUS"] = kpl.MAX_SKUS
+            kpl.MAX_SKUS = max(1, int(max_skus))
+        if page_start is not None:
+            backup["PAGE_START"] = kpl.PAGE_START
+            kpl.PAGE_START = max(1, int(page_start))
+        if page_to is not None:
+            backup["PAGE_TO"] = kpl.PAGE_TO
+            kpl.PAGE_TO = max(1, int(page_to))
+
+        prd = (pipeline_run_dir or "").strip()
+        if prd:
+            backup["PIPELINE_RUN_DIR"] = kpl.PIPELINE_RUN_DIR
+            kpl.PIPELINE_RUN_DIR = prd
+
+        cf = (cookie_file_path or "").strip()
+        if cf:
+            backup["PIPELINE_COOKIE_FILE"] = kpl.PIPELINE_COOKIE_FILE
+            kpl.PIPELINE_COOKIE_FILE = cf
+
+        pv = (pvid or "").strip()
+        if pv:
+            backup["PVID"] = kpl.PVID
+            kpl.PVID = pv
+
+        rd = (request_delay or "").strip()
+        if rd:
+            backup["REQUEST_DELAY"] = kpl.REQUEST_DELAY
+            kpl.REQUEST_DELAY = rd
+
+        lp = (list_pages or "").strip()
+        if lp:
+            backup["LIST_PAGES"] = kpl.LIST_PAGES
+            kpl.LIST_PAGES = lp
+
+        if scenario_filter_enabled is not None:
+            backup["SCENARIO_FILTER_ENABLED"] = kpl.SCENARIO_FILTER_ENABLED
+            kpl.SCENARIO_FILTER_ENABLED = bool(scenario_filter_enabled)
+
+        run_dir = kpl.main(keyword=kw)
+    except kpl.PipelineCancelled as e:
+        run_dir_path = Path(e.run_dir).resolve()
+        merged = run_dir_path / kpl.FILE_MERGED_CSV
+        if merged.is_file():
+            try:
+                write_competitor_analysis_for_run_dir(
+                    run_dir_path, kw, report_config=report_config
+                )
+            except Exception:
+                pass
+        raise
+    finally:
+        for name, val in backup.items():
+            setattr(kpl, name, val)
+
+    return write_competitor_analysis_for_run_dir(
+        Path(run_dir).resolve(), kw, report_config=report_config
+    )

backend/pipeline/llm_generate.py

@@ -0,0 +1,150 @@
+"""
+竞品报告 / 策略稿的**大模型生成**：通过 ``crawler_copy/jd_pc_search/AI_crawler`` 的
+``chat_completion_text`` 调用，与配料识别共用网关与密钥。
+"""
+from __future__ import annotations
+
+import json
+import sys
+from pathlib import Path
+from typing import Any
+
+from django.conf import settings
+
+from .brief_compact import compact_brief_for_llm
+from .strategy_draft import build_strategy_draft_markdown
+
+
+def _ensure_ai_crawler_path() -> None:
+    root = Path(settings.CRAWLER_JD_ROOT).resolve()
+    if not root.is_dir():
+        raise FileNotFoundError(f"爬虫副本目录不存在: {root}")
+    rs = str(root)
+    if rs not in sys.path:
+        sys.path.insert(0, rs)
+
+
+def _call_llm(system_prompt: str, user_prompt: str) -> str:
+    _ensure_ai_crawler_path()
+    import AI_crawler as ac  # noqa: WPS433
+
+    raw = ac.chat_completion_text(
+        system_prompt=system_prompt,
+        user_prompt=user_prompt,
+    )
+    return ac.strip_outer_markdown_fence(raw)
+
+
+REPORT_SYSTEM = """你撰写一段**短小的「速读与策略补充」**，插在完整规则报告**之前**供读者扫读。读者为业务与产品。
+
+**输入**：JSON 含 `keyword`、`competitor_brief`（可能经裁剪）、`matrix_overview_for_llm`（按细分类目的 SKU 数与品牌样本）。
+所有数字、占比、条数、品牌名、价格区间等**必须严格来自输入 JSON**，禁止编造未在输入中出现的定量结论。
+
+**硬性禁止**：
+- **不要**输出完整报告目录或重复「研究范围与方法」等长章结构；
+- **不要**撰写 Markdown 表格版「竞品对比矩阵」或罗列 SKU 明细——**正文报告已含完整矩阵**，此处仅可概括分组级结论（细类名、SKU 数、主要品牌来自 `matrix_overview_for_llm` / brief）；
+- **不要**写「matrix_by_group 已省略」「仅保留代表性品牌」等免责声明，也不要引导读者认为明细缺失；
+- **不要使用** CR1、CR3 等英文缩写；集中度请用「第一大品牌份额」「前三品牌合计份额」。
+
+**请输出**（仅输出正文，不要前言后语）：
+- 使用 **Markdown**，控制在约 **800～1500 字**；
+- 建议小节标题（二级）：**执行摘要要点**、**竞争与价盘速读**、**用户声量与关注点**、**策略提示与数据边界**；
+- 若有 `comment_sentiment_lexicon`，概括正/负向粗判与局限（非深度学习）；
+- 语气专业、中文；缺失项写「本摘要未提供该项」而非猜测。"""
+
+REPORT_USER_PREFIX = """请根据以下 JSON 撰写完整竞品分析报告（Markdown 正文）。\n\n"""
+
+
+def generate_competitor_report_markdown_llm(brief: dict[str, Any], keyword: str) -> str:
+    compact = compact_brief_for_llm(brief)
+    payload = {
+        "keyword": keyword,
+        "competitor_brief": compact,
+        "matrix_overview_for_llm": compact.get("matrix_overview_for_llm") or [],
+    }
+    user = REPORT_USER_PREFIX + json.dumps(payload, ensure_ascii=False)
+    return _call_llm(REPORT_SYSTEM, user)
+
+
+SENTIMENT_LLM_SYSTEM = """你是电商/食品类用户研究助手。输入 JSON 含：
+- ``comment_sentiment_lexicon``：关键词规则下的条数与短语命中（粗判，非深度学习）；
+- ``sample_reviews_*``：按同一规则从评价中抽样的短文（已截断），**仅可依据这些原文与 lexicon 数字归纳**。
+
+**硬性要求**：
+- **仅输出 Markdown 正文**（不要用 ``` 围栏包裹全文）；
+- **不要编造**样本中未出现的具体事实、品牌、价格、医学功效；
+- 条数、占比等**定量表述须与** ``comment_sentiment_lexicon`` **一致**，勿与样本矛盾。
+
+**建议结构**（使用四级标题 ``####``）：
+1. ``#### 正向要点归纳``：3～6 条要点，概括满意点（口感、甜度、包装、物流、性价比等）；
+2. ``#### 负向与风险点归纳``：3～6 条要点；
+3. ``#### 使用注意``：1～2 句说明样本量、抽样局限、与关键词规则可能不一致之处。
+
+总字数约 **400～900 字**，简体中文，语气客观。"""
+
+
+def generate_comment_sentiment_analysis_llm(payload: dict[str, Any]) -> str:
+    """基于规则分桶抽样评价 + lexicon 统计，生成 §8.2 大模型解读段落（Markdown）。"""
+    p = dict(payload)
+    raw = json.dumps(p, ensure_ascii=False)
+    if len(raw) > 88_000:
+        for k in (
+            "sample_reviews_positive_biased",
+            "sample_reviews_negative_biased",
+            "sample_reviews_mixed_tone",
+        ):
+            lst = p.get(k)
+            if isinstance(lst, list):
+                p[k] = [str(x)[:140] for x in lst[:8]]
+        raw = json.dumps(p, ensure_ascii=False)
+    if len(raw) > 88_000:
+        raw = raw[:82_000] + "\n\n…（输入过长已截断，请勿编造截断外内容）\n"
+    user = "请根据以下 JSON 按系统说明输出 Markdown：\n\n" + raw
+    return _call_llm(SENTIMENT_LLM_SYSTEM, user)
+
+
+STRATEGY_SYSTEM = """你是市场策略顾问，根据**结构化监测摘要**与业务侧填写的**决策字段**，把「规则底稿」润色为可读的策略 Markdown。
+
+**规则**：
+- 输入 JSON 含 `rules_draft_markdown`（规则引擎生成的底稿，与同任务数据一致）、`structured_brief`（摘要子集）、`strategy_decisions`、`business_notes` 等；
+- **不得编造**输入中不存在的销量、占比、价格数字；若底稿与摘要中有数字，须保持一致；表述集中度时用「第一大品牌份额」等中文，**不要用** CR1、CR3 缩写；
+- 若 `structured_brief` 含 `matrix_overview_for_llm` 或矩阵相关字段，策略中应**呼应**细分类目分组与竞品矩阵结论，不得无故删光矩阵相关建议；
+- 可调整段落衔接、标题层级、列表与表格呈现，使更易读；可补充「建议」「待业务确认」类表述，但不虚构竞品名称或数据；
+- **仅输出** Markdown 正文（不要 ``` 围栏包裹全文）。"""
+
+STRATEGY_USER_PREFIX = """请基于以下 JSON 输出最终策略稿（Markdown）。\n\n"""
+
+
+def generate_strategy_draft_markdown_llm(
+    *,
+    job_id: int,
+    keyword: str,
+    brief: dict[str, Any],
+    business_notes: str,
+    generated_at_iso: str,
+    strategy_decisions: dict[str, Any],
+) -> str:
+    rules_md = build_strategy_draft_markdown(
+        job_id=job_id,
+        keyword=keyword,
+        brief=brief,
+        business_notes=business_notes,
+        generated_at_iso=generated_at_iso,
+        strategy_decisions=strategy_decisions,
+    )
+    compact = compact_brief_for_llm(brief, max_chars=80_000)
+    payload = {
+        "job_id": job_id,
+        "keyword": keyword,
+        "generated_at_iso": generated_at_iso,
+        "strategy_decisions": strategy_decisions,
+        "business_notes": business_notes,
+        "structured_brief": compact,
+        "rules_draft_markdown": rules_md,
+    }
+    raw = json.dumps(payload, ensure_ascii=False)
+    if len(raw) > 500_000:
+        payload["rules_draft_markdown"] = rules_md[:200_000] + "\n\n…（底稿过长已截断，请勿编造截断后内容）\n"
+        raw = json.dumps(payload, ensure_ascii=False)
+    user = STRATEGY_USER_PREFIX + raw
+    return _call_llm(STRATEGY_SYSTEM, user)

backend/pipeline/llm_keyword_suggest.py

@@ -0,0 +1,164 @@
+"""在报告生成前：基于**全量**评价文本分块调用大模型，联想补充关注词（参与后续统计与报告）。"""
+from __future__ import annotations
+
+import json
+import re
+import sys
+from pathlib import Path
+from typing import Any
+
+from django.conf import settings
+
+MAX_CHUNK_CHARS = 24_000
+MAX_CHUNKS = 12
+
+_CHUNK_SYSTEM = """你是电商评价挖掘助手。输入 JSON 含 keyword、excerpt_index、excerpts（一段用户评价正文合集）。
+任务：从 excerpts 中抽取值得纳入「关注词/卖点监测」的**中文短语**（2～12 字为主，可为词组）。
+
+硬性规则：
+- 仅输出一段 JSON：{"phrases": ["短语1", ...]}，短语共 6～20 条。
+- 不要医疗功效、治愈、降血糖承诺；不要完整复制整句评价。
+- 不要输出与常见停用词无信息量的单字。
+- 不要输出 JSON 以外的文字。"""
+
+
+def _ensure_ai_crawler_path() -> None:
+    root = Path(settings.CRAWLER_JD_ROOT).resolve()
+    if not root.is_dir():
+        raise FileNotFoundError(f"爬虫副本目录不存在: {root}")
+    rs = str(root)
+    if rs not in sys.path:
+        sys.path.insert(0, rs)
+
+
+def _call_llm(system_prompt: str, user_prompt: str) -> str:
+    _ensure_ai_crawler_path()
+    import AI_crawler as ac  # noqa: WPS433
+
+    return ac.chat_completion_text(
+        system_prompt=system_prompt,
+        user_prompt=user_prompt,
+    )
+
+
+def _chunk_comment_texts(texts: list[str]) -> list[str]:
+    """将全量评价划为若干段，控制单段字符量与最大段数。"""
+    parts: list[str] = []
+    cur: list[str] = []
+    cur_len = 0
+    for t in texts:
+        s = (t or "").strip()
+        if not s:
+            continue
+        extra = len(s) + 1
+        if cur and cur_len + extra > MAX_CHUNK_CHARS:
+            parts.append("\n".join(cur))
+            cur = []
+            cur_len = 0
+        cur.append(s)
+        cur_len += extra
+    if cur:
+        parts.append("\n".join(cur))
+    if not parts:
+        return []
+    if len(parts) <= MAX_CHUNKS:
+        return parts
+    idxs = sorted(
+        {min(int(i * len(parts) / MAX_CHUNKS), len(parts) - 1) for i in range(MAX_CHUNKS)}
+    )
+    return [parts[i] for i in idxs]
+
+
+def _parse_phrases_object(raw: str) -> list[str]:
+    t = raw.strip()
+    t = re.sub(r"^```(?:json)?\s*", "", t)
+    t = re.sub(r"\s*```$", "", t)
+    try:
+        obj = json.loads(t)
+        if isinstance(obj, dict) and isinstance(obj.get("phrases"), list):
+            return [str(x).strip() for x in obj["phrases"] if str(x).strip()]
+    except json.JSONDecodeError:
+        pass
+    m = re.search(r"\{[\s\S]*\}", t)
+    if m:
+        try:
+            obj = json.loads(m.group(0))
+            if isinstance(obj, dict) and isinstance(obj.get("phrases"), list):
+                return [str(x).strip() for x in obj["phrases"] if str(x).strip()]
+        except json.JSONDecodeError:
+            pass
+    return []
+
+
+def suggest_focus_keywords_from_all_comments(
+    *,
+    keyword: str,
+    brief_slice: dict[str, Any],
+    all_comment_texts: list[str],
+) -> dict[str, Any]:
+    """
+    读取全量评价（在服务端分块），每块调用模型抽取短语，合并去重后返回 ``suggested_focus_keywords``。
+    """
+    if not all_comment_texts:
+        return {
+            "suggested_focus_keywords": [],
+            "suggested_scenario_hints": [],
+            "rationale": "无评价正文可分析。",
+            "chunks_processed": 0,
+            "total_comment_texts": 0,
+        }
+
+    existing: set[str] = set()
+    for x in brief_slice.get("comment_focus_keywords") or []:
+        if isinstance(x, dict):
+            w = str(x.get("word") or "").strip()
+            if w:
+                existing.add(w)
+
+    chunks = _chunk_comment_texts(all_comment_texts)
+    collected: list[str] = []
+    for i, ch in enumerate(chunks):
+        payload = {
+            "keyword": keyword,
+            "excerpt_index": i + 1,
+            "excerpts": ch,
+        }
+        raw = _call_llm(_CHUNK_SYSTEM, json.dumps(payload, ensure_ascii=False))
+        collected.extend(_parse_phrases_object(raw))
+
+    seen: set[str] = set()
+    merged: list[str] = []
+    for p in collected:
+        t = p.strip()
+        if len(t) < 2 or len(t) > 24:
+            continue
+        if t in seen or t in existing:
+            continue
+        seen.add(t)
+        merged.append(t)
+
+    out_kw = merged[:22]
+    return {
+        "suggested_focus_keywords": out_kw,
+        "suggested_scenario_hints": [],
+        "rationale": (
+            f"基于全量 {len(all_comment_texts)} 条评价文本，分 {len(chunks)} 段调用模型抽取短语并去重；"
+            f"已排除与当前关注词统计表完全相同的词。"
+        ),
+        "chunks_processed": len(chunks),
+        "total_comment_texts": len(all_comment_texts),
+    }
+
+
+# 兼容旧接口名（若仍有调用）
+def suggest_comment_keywords_llm(
+    *,
+    keyword: str,
+    brief_slice: dict[str, Any],
+    comment_samples: list[str],
+) -> dict[str, Any]:
+    return suggest_focus_keywords_from_all_comments(
+        keyword=keyword,
+        brief_slice=brief_slice,
+        all_comment_texts=list(comment_samples),
+    )

backend/pipeline/management/commands/run_pipeline_job.py

@@ -0,0 +1,59 @@
+# -*- coding: utf-8 -*-
+"""由 ``execute_job`` 在子进程中调用，勿直接用于交互调试。"""
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+from django.conf import settings
+from django.core.management.base import BaseCommand, CommandError
+
+from pipeline.cookie_paste import normalize_browser_cookie_paste
+from pipeline.jd_runner import run_jd_keyword_and_report
+from pipeline.models import PipelineJob
+
+
+class Command(BaseCommand):
+    help = "内部命令：在独立进程中执行京东流水线（便于父进程 terminate 模拟 Ctrl+C）。"
+
+    def add_arguments(self, parser) -> None:
+        parser.add_argument("job_id", type=int)
+
+    def handle(self, *args, **options) -> None:
+        job_id = int(options["job_id"])
+        run_dir = (os.environ.get("PIPELINE_JOB_RUN_DIR") or "").strip()
+        if not run_dir:
+            raise CommandError("缺少环境变量 PIPELINE_JOB_RUN_DIR")
+
+        job = PipelineJob.objects.filter(pk=job_id).first()
+        if not job:
+            raise CommandError(f"任务不存在: {job_id}")
+
+        cookie_path = (os.environ.get("PIPELINE_JOB_COOKIE_PATH") or "").strip() or None
+        if cookie_path and not Path(cookie_path).is_file():
+            cookie_path = None
+        # 与父进程 env 双保险：粘贴 Cookie 仍以 DB 为准再落盘（避免 Windows 等环境下 env 未传到子进程）
+        _from_db = normalize_browser_cookie_paste(job.cookie_text or "")
+        if not cookie_path and _from_db:
+            runtime_dir = Path(settings.BASE_DIR) / "runtime_cookies"
+            runtime_dir.mkdir(parents=True, exist_ok=True)
+            worker_cookie = (runtime_dir / f"job_{job_id}_cookie_worker.txt").resolve()
+            worker_cookie.write_text(_from_db, encoding="utf-8")
+            cookie_path = str(worker_cookie)
+
+        rc = job.report_config if isinstance(job.report_config, dict) else {}
+
+        run_jd_keyword_and_report(
+            job.keyword,
+            max_skus=job.max_skus,
+            page_start=job.page_start,
+            page_to=job.page_to,
+            pipeline_run_dir=run_dir,
+            cookie_file_path=cookie_path,
+            pvid=(job.pvid or "").strip() or None,
+            request_delay=(job.request_delay or "").strip() or None,
+            list_pages=(job.list_pages or "").strip() or None,
+            scenario_filter_enabled=job.scenario_filter_enabled,
+            report_config=rc or None,
+            cancel_check=None,
+        )

backend/pipeline/md_document_export.py

@@ -0,0 +1,250 @@
+"""Markdown → Word（.docx）/ 简易 PDF；供任务报告与策略稿导出。"""
+from __future__ import annotations
+
+import os
+import re
+from io import BytesIO
+from pathlib import Path
+from typing import Any
+from xml.sax.saxutils import escape as xml_escape
+
+
+def _strip_inline_md(s: str) -> str:
+    s = re.sub(r"\*\*(.+?)\*\*", r"\1", s)
+    s = re.sub(r"`([^`]+)`", r"\1", s)
+    return s
+
+
+def _is_table_sep(line: str) -> bool:
+    t = line.strip()
+    if not t.startswith("|"):
+        return False
+    inner = t.strip("|").replace(" ", "")
+    return bool(inner) and all(p in ("", "---", ":---", "---:", ":---:") for p in t.split("|"))
+
+
+_img_line = re.compile(r"^!\[([^\]]*)\]\(([^)]+)\)\s*$")
+
+
+def markdown_to_docx_bytes(md: str, *, asset_root: Path | None = None) -> bytes:
+    from docx import Document
+    from docx.enum.text import WD_PARAGRAPH_ALIGNMENT
+    from docx.shared import Inches, Pt
+
+    doc = Document()
+    try:
+        style = doc.styles["Normal"]
+        style.font.name = "Microsoft YaHei"
+        style.font.size = Pt(10.5)
+    except Exception:
+        pass
+
+    lines = (md or "").replace("\r\n", "\n").split("\n")
+    i = 0
+    in_fence = False
+    while i < len(lines):
+        raw = lines[i]
+        if raw.strip().startswith("```"):
+            in_fence = not in_fence
+            i += 1
+            continue
+        if in_fence:
+            p = doc.add_paragraph(xml_escape(raw) or " ")
+            p.style = doc.styles["Normal"]
+            for run in p.runs:
+                run.font.name = "Consolas"
+                run.font.size = Pt(9)
+            i += 1
+            continue
+
+        line = raw.rstrip()
+        if not line.strip():
+            doc.add_paragraph("")
+            i += 1
+            continue
+        if line.startswith("# "):
+            doc.add_heading(_strip_inline_md(line[2:].strip()), level=0)
+            i += 1
+            continue
+        if line.startswith("## "):
+            doc.add_heading(_strip_inline_md(line[3:].strip()), level=1)
+            i += 1
+            continue
+        if line.startswith("### "):
+            doc.add_heading(_strip_inline_md(line[4:].strip()), level=2)
+            i += 1
+            continue
+        if line.startswith("#### "):
+            doc.add_heading(_strip_inline_md(line[5:].strip()), level=3)
+            i += 1
+            continue
+        mimg = _img_line.match(line.strip())
+        if mimg and asset_root is not None:
+            rel = mimg.group(2).strip()
+            if not (rel.startswith("http://") or rel.startswith("https://")):
+                img_path = (asset_root / rel).resolve()
+                try:
+                    img_path.relative_to(asset_root.resolve())
+                except ValueError:
+                    i += 1
+                    continue
+                if img_path.is_file():
+                    doc.add_picture(str(img_path), width=Inches(5.9))
+            i += 1
+            continue
+        if line.strip().startswith("|"):
+            rows: list[list[str]] = []
+            while i < len(lines) and lines[i].strip().startswith("|"):
+                row_line = lines[i].strip()
+                if _is_table_sep(row_line):
+                    i += 1
+                    continue
+                cells = [c.strip() for c in row_line.strip("|").split("|")]
+                rows.append([_strip_inline_md(c) for c in cells])
+                i += 1
+            if rows:
+                max_cols = max(len(r) for r in rows)
+                pad_rows = [r + [""] * (max_cols - len(r)) for r in rows]
+                tbl = doc.add_table(rows=len(pad_rows), cols=max_cols)
+                tbl.style = "Table Grid"
+                for ri, row in enumerate(pad_rows):
+                    for ci, cell in enumerate(row):
+                        tbl.rows[ri].cells[ci].text = cell
+            continue
+
+        p = doc.add_paragraph()
+        p.alignment = WD_PARAGRAPH_ALIGNMENT.LEFT
+        text = _strip_inline_md(line)
+        p.add_run(text)
+
+    bio = BytesIO()
+    doc.save(bio)
+    return bio.getvalue()
+
+
+def _pdf_font_candidates() -> list[Path]:
+    raw = (os.environ.get("MA_PDF_FONT") or "").strip()
+    out: list[Path] = []
+    if raw:
+        out.append(Path(raw))
+    windir = os.environ.get("WINDIR", r"C:\Windows")
+    out.extend(
+        [
+            Path(windir) / "Fonts" / "simhei.ttf",
+            Path(windir) / "Fonts" / "simsun.ttc",
+            Path(windir) / "Fonts" / "msyh.ttf",
+        ]
+    )
+    return out
+
+
+def markdown_to_pdf_bytes(md: str, *, asset_root: Path | None = None) -> bytes:
+    """简易纯文本流式 PDF；需本机 .ttf 中文字体或环境变量 MA_PDF_FONT。"""
+    from reportlab.lib.pagesizes import A4
+    from reportlab.lib.styles import ParagraphStyle, getSampleStyleSheet
+    from reportlab.lib.units import cm
+    from reportlab.pdfbase import pdfmetrics
+    from reportlab.pdfbase.ttfonts import TTFont
+    from reportlab.platypus import Image as RLImage
+    from reportlab.platypus import Paragraph, SimpleDocTemplate, Spacer
+
+    font_name = "MaExportCJK"
+    registered = False
+    for p in _pdf_font_candidates():
+        if not p.is_file():
+            continue
+        try:
+            if p.suffix.lower() == ".ttc":
+                try:
+                    pdfmetrics.registerFont(
+                        TTFont(font_name, str(p), subfontIndex=0)
+                    )
+                except TypeError:
+                    pdfmetrics.registerFont(TTFont(font_name, str(p)))
+            else:
+                pdfmetrics.registerFont(TTFont(font_name, str(p)))
+            registered = True
+            break
+        except Exception:
+            continue
+    if not registered:
+        raise ValueError(
+            "未找到可用的中文字体文件。请在 Windows 上安装黑体/宋体，"
+            "或设置环境变量 MA_PDF_FONT 指向 .ttf 文件路径。"
+        )
+
+    styles = getSampleStyleSheet()
+    body = ParagraphStyle(
+        name="BodyCJK",
+        parent=styles["Normal"],
+        fontName=font_name,
+        fontSize=10,
+        leading=14,
+    )
+    h1s = ParagraphStyle(
+        name="H1CJK",
+        parent=body,
+        fontSize=16,
+        leading=20,
+        spaceAfter=8,
+    )
+    h2s = ParagraphStyle(
+        name="H2CJK",
+        parent=body,
+        fontSize=13,
+        leading=17,
+        spaceAfter=6,
+    )
+
+    story: list[Any] = []
+    lines = (md or "").replace("\r\n", "\n").split("\n")
+    in_fence = False
+    for raw in lines:
+        if raw.strip().startswith("```"):
+            in_fence = not in_fence
+            continue
+        s = raw.rstrip()
+        if in_fence:
+            story.append(Paragraph(xml_escape(s or " "), body))
+            story.append(Spacer(1, 0.1 * cm))
+            continue
+        if not s.strip():
+            story.append(Spacer(1, 0.15 * cm))
+            continue
+        mimg = _img_line.match(s.strip())
+        if mimg and asset_root is not None:
+            rel = mimg.group(2).strip()
+            if not (rel.startswith("http://") or rel.startswith("https://")):
+                img_path = (asset_root / rel).resolve()
+                try:
+                    img_path.relative_to(asset_root.resolve())
+                except ValueError:
+                    continue
+                if img_path.is_file():
+                    story.append(RLImage(str(img_path), width=13 * cm))
+                    story.append(Spacer(1, 0.2 * cm))
+            continue
+        plain = _strip_inline_md(s)
+        text = xml_escape(plain)
+        if s.startswith("# "):
+            story.append(Paragraph(xml_escape(plain[2:]), h1s))
+        elif s.startswith("## "):
+            story.append(Paragraph(xml_escape(plain[3:]), h2s))
+        elif s.startswith("### "):
+            story.append(Paragraph(xml_escape(plain[4:]), body))
+        elif s.strip().startswith("|"):
+            story.append(Paragraph(text.replace("|", " │ "), body))
+        else:
+            story.append(Paragraph(text, body))
+
+    buf = BytesIO()
+    doc = SimpleDocTemplate(
+        buf,
+        pagesize=A4,
+        leftMargin=2 * cm,
+        rightMargin=2 * cm,
+        topMargin=2 * cm,
+        bottomMargin=2 * cm,
+    )
+    doc.build(story)
+    return buf.getvalue()

backend/pipeline/migrations/0001_initial.py

@@ -0,0 +1,33 @@
+# Generated by Django 5.2.13 on 2026-04-08 08:34
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='PipelineJob',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('platform', models.CharField(db_index=True, default='jd', max_length=32)),
+                ('keyword', models.CharField(max_length=256)),
+                ('max_skus', models.PositiveIntegerField(blank=True, null=True)),
+                ('page_start', models.PositiveIntegerField(blank=True, null=True)),
+                ('page_to', models.PositiveIntegerField(blank=True, null=True)),
+                ('status', models.CharField(choices=[('pending', '待执行'), ('running', '执行中'), ('success', '成功'), ('failed', '失败')], db_index=True, default='pending', max_length=16)),
+                ('run_dir', models.TextField(blank=True, default='')),
+                ('error_message', models.TextField(blank=True, default='')),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+                ('updated_at', models.DateTimeField(auto_now=True)),
+            ],
+            options={
+                'ordering': ['-created_at'],
+            },
+        ),
+    ]

backend/pipeline/migrations/0002_job_options.py

@@ -0,0 +1,48 @@
+# Generated by Django 5.2.13 on 2026-04-08 08:43
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='cookie_file_path',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='cookie_text',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='list_pages',
+            field=models.CharField(blank=True, default='', help_text='评论分页，如 "1-2"；空则沿用副本默认', max_length=64),
+        ),
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='pipeline_run_dir',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='pvid',
+            field=models.CharField(blank=True, default='', max_length=128),
+        ),
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='request_delay',
+            field=models.CharField(blank=True, default='', help_text='如 "30-60"；空则沿用副本默认', max_length=64),
+        ),
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='scenario_filter_enabled',
+            field=models.BooleanField(blank=True, null=True),
+        ),
+    ]

backend/pipeline/migrations/0003_jd_job_dataset_and_products.py

@@ -0,0 +1,102 @@
+# Generated by Django 5.2.13 on 2026-04-09 02:24
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0002_job_options'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='JdProduct',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('platform', models.CharField(db_index=True, default='jd', max_length=16)),
+                ('sku_id', models.CharField(db_index=True, max_length=64)),
+                ('ware_id', models.CharField(blank=True, default='', max_length=64)),
+                ('title', models.TextField(blank=True, default='')),
+                ('detail_brand', models.CharField(blank=True, default='', max_length=512)),
+                ('detail_price_final', models.CharField(blank=True, default='', max_length=128)),
+                ('detail_category_path', models.TextField(blank=True, default='')),
+                ('current_payload', models.JSONField(blank=True, default=dict)),
+                ('last_captured_at', models.DateTimeField(blank=True, db_index=True, null=True)),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+                ('updated_at', models.DateTimeField(auto_now=True)),
+                ('last_job', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='touched_products', to='pipeline.pipelinejob')),
+            ],
+            options={
+                'ordering': ['-updated_at'],
+            },
+        ),
+        migrations.CreateModel(
+            name='JdProductSnapshot',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('run_dir', models.TextField(blank=True, default='')),
+                ('captured_at', models.DateTimeField(db_index=True)),
+                ('payload', models.JSONField(blank=True, default=dict)),
+                ('job', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='product_snapshots', to='pipeline.pipelinejob')),
+                ('product', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='snapshots', to='pipeline.jdproduct')),
+            ],
+            options={
+                'ordering': ['-captured_at'],
+            },
+        ),
+        migrations.CreateModel(
+            name='JdJobCommentRow',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('row_index', models.PositiveIntegerField()),
+                ('sku_id', models.CharField(blank=True, db_index=True, default='', max_length=64)),
+                ('payload', models.JSONField(blank=True, default=dict)),
+                ('job', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='job_comment_rows', to='pipeline.pipelinejob')),
+            ],
+            options={
+                'ordering': ['row_index'],
+                'indexes': [models.Index(fields=['job', 'sku_id'], name='pipeline_jd_job_id_57eaf8_idx')],
+                'constraints': [models.UniqueConstraint(fields=('job', 'row_index'), name='uniq_pipeline_jdjobcommentrow_job_idx')],
+            },
+        ),
+        migrations.CreateModel(
+            name='JdJobDetailRow',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('row_index', models.PositiveIntegerField()),
+                ('sku_id', models.CharField(blank=True, db_index=True, default='', max_length=64)),
+                ('payload', models.JSONField(blank=True, default=dict)),
+                ('job', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='job_detail_rows', to='pipeline.pipelinejob')),
+            ],
+            options={
+                'ordering': ['row_index'],
+                'indexes': [models.Index(fields=['job', 'sku_id'], name='pipeline_jd_job_id_d9112a_idx')],
+                'constraints': [models.UniqueConstraint(fields=('job', 'row_index'), name='uniq_pipeline_jdjobdetailrow_job_idx')],
+            },
+        ),
+        migrations.CreateModel(
+            name='JdJobSearchRow',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('row_index', models.PositiveIntegerField()),
+                ('sku_id', models.CharField(blank=True, db_index=True, default='', max_length=64)),
+                ('payload', models.JSONField(blank=True, default=dict)),
+                ('job', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='job_search_rows', to='pipeline.pipelinejob')),
+            ],
+            options={
+                'ordering': ['row_index'],
+                'indexes': [models.Index(fields=['job', 'sku_id'], name='pipeline_jd_job_id_28447c_idx')],
+                'constraints': [models.UniqueConstraint(fields=('job', 'row_index'), name='uniq_pipeline_jdjobsearchrow_job_idx')],
+            },
+        ),
+        migrations.AddConstraint(
+            model_name='jdproduct',
+            constraint=models.UniqueConstraint(fields=('platform', 'sku_id'), name='uniq_pipeline_jdproduct_platform_sku'),
+        ),
+        migrations.AddConstraint(
+            model_name='jdproductsnapshot',
+            constraint=models.UniqueConstraint(fields=('product', 'job'), name='uniq_pipeline_jdproductsnapshot_product_job'),
+        ),
+    ]

backend/pipeline/migrations/0004_job_rows_split_csv_fields.py

@@ -0,0 +1,315 @@
+# Generated by Django 5.2.13 on 2026-04-09 02:40
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0003_jd_job_dataset_and_products'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='jdjobcommentrow',
+            name='payload',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='payload',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobsearchrow',
+            name='payload',
+        ),
+        migrations.AddField(
+            model_name='jdjobcommentrow',
+            name='buy_count_text',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobcommentrow',
+            name='comment_date',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobcommentrow',
+            name='comment_id',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobcommentrow',
+            name='comment_score',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobcommentrow',
+            name='large_pic_urls',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobcommentrow',
+            name='tag_comment_content',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobcommentrow',
+            name='user_nick_name',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_belt_banner',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_body_ingredients',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_body_ingredients_source_url',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_brand',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_category_path',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_csfh_text',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_delivery_promise',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_main_image',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_main_sku_id',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_page_sku_id',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_price_final',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_price_original',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_product_attributes',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_product_id',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_purchase_price',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_shop_id',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_shop_name',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_shop_url',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_sku_name',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_sku_title',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_stock_text',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_vender_id',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='http_status',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='attributes',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='comment_count',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='comment_sales_floor',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='coupon_price',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='detail_url',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='hot_list_rank',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='image',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='item_id',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='keyword',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='leaf_category',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='location',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='original_price',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='page',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='platform',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='price',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='seckill_info',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='selling_point',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='shop_info_title',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='shop_info_url',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='shop_logo',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='shop_name',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='shop_url',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='title',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='uniqpid',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobsearchrow',
+            name='video_url',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AlterField(
+            model_name='jdjobcommentrow',
+            name='sku_id',
+            field=models.TextField(blank=True, db_index=True, default=''),
+        ),
+        migrations.AlterField(
+            model_name='jdjobdetailrow',
+            name='sku_id',
+            field=models.TextField(blank=True, db_index=True, default=''),
+        ),
+        migrations.AlterField(
+            model_name='jdjobsearchrow',
+            name='sku_id',
+            field=models.TextField(blank=True, db_index=True, default=''),
+        ),
+    ]

backend/pipeline/migrations/0005_jd_job_merged_row.py

@@ -0,0 +1,61 @@
+# Generated by Django 5.2.13 on 2026-04-09 06:00
+
+import django.db.models.deletion
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0004_job_rows_split_csv_fields'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='JdJobMergedRow',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('row_index', models.PositiveIntegerField()),
+                ('pipeline_keyword', models.TextField(blank=True, default='')),
+                ('sku_id', models.TextField(blank=True, db_index=True, default='')),
+                ('ware_id', models.TextField(blank=True, default='')),
+                ('title', models.TextField(blank=True, default='')),
+                ('price', models.TextField(blank=True, default='')),
+                ('coupon_price', models.TextField(blank=True, default='')),
+                ('original_price', models.TextField(blank=True, default='')),
+                ('selling_point', models.TextField(blank=True, default='')),
+                ('hot_list_rank', models.TextField(blank=True, default='')),
+                ('comment_fuzzy', models.TextField(blank=True, default='')),
+                ('comment_sales_floor', models.TextField(blank=True, default='')),
+                ('shop_name', models.TextField(blank=True, default='')),
+                ('detail_url', models.TextField(blank=True, default='')),
+                ('image', models.TextField(blank=True, default='')),
+                ('attributes', models.TextField(blank=True, default='')),
+                ('leaf_category', models.TextField(blank=True, default='')),
+                ('keyword', models.TextField(blank=True, default='')),
+                ('page', models.TextField(blank=True, default='')),
+                ('detail_http_status', models.TextField(blank=True, default='')),
+                ('detail_brand', models.TextField(blank=True, default='')),
+                ('detail_sku_title', models.TextField(blank=True, default='')),
+                ('detail_price_final', models.TextField(blank=True, default='')),
+                ('detail_price_original', models.TextField(blank=True, default='')),
+                ('detail_shop_name', models.TextField(blank=True, default='')),
+                ('detail_category_path', models.TextField(blank=True, default='')),
+                ('detail_product_attributes', models.TextField(blank=True, default='')),
+                ('detail_main_image', models.TextField(blank=True, default='')),
+                ('detail_body_ingredients', models.TextField(blank=True, default='')),
+                ('detail_body_ingredients_source_url', models.TextField(blank=True, default='')),
+                ('detail_stock_text', models.TextField(blank=True, default='')),
+                ('detail_delivery_promise', models.TextField(blank=True, default='')),
+                ('comment_http_status', models.TextField(blank=True, default='')),
+                ('pipeline_comment_count', models.TextField(blank=True, default='')),
+                ('comment_preview', models.TextField(blank=True, default='')),
+                ('job', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='job_merged_rows', to='pipeline.pipelinejob')),
+            ],
+            options={
+                'ordering': ['row_index'],
+                'indexes': [models.Index(fields=['job', 'sku_id'], name='pipeline_jd_job_id_830248_idx')],
+                'constraints': [models.UniqueConstraint(fields=('job', 'row_index'), name='uniq_pipeline_jdjobmergedrow_job_idx')],
+            },
+        ),
+    ]

backend/pipeline/migrations/0006_trim_search_detail_merged_fields.py

@@ -0,0 +1,142 @@
+# Generated by Django 5.2.13 on 2026-04-09 06:13
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0005_jd_job_merged_row'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_belt_banner',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_body_ingredients_source_url',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_brand',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_category_path',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_csfh_text',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_delivery_promise',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_main_image',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_main_sku_id',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_page_sku_id',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_price_final',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_price_original',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_product_attributes',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_product_id',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_purchase_price',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_shop_id',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_shop_name',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_shop_url',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_sku_name',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_sku_title',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_stock_text',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='detail_vender_id',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobdetailrow',
+            name='http_status',
+        ),
+        # JdJobMergedRow：裁剪商详列，保留与 lean 合并表一致的子集（非「先删再在后续迁移加回」）
+        migrations.RemoveField(
+            model_name='jdjobmergedrow',
+            name='detail_body_ingredients_source_url',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobmergedrow',
+            name='detail_delivery_promise',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobmergedrow',
+            name='detail_main_image',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobmergedrow',
+            name='detail_price_original',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobmergedrow',
+            name='detail_sku_title',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobmergedrow',
+            name='detail_stock_text',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobsearchrow',
+            name='shop_info_title',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobsearchrow',
+            name='shop_logo',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobsearchrow',
+            name='uniqpid',
+        ),
+        migrations.RemoveField(
+            model_name='jdjobsearchrow',
+            name='video_url',
+        ),
+    ]

backend/pipeline/migrations/0007_drop_merged_http_status_columns.py

@@ -0,0 +1,46 @@
+# Generated by Django 5.2.13 on 2026-04-09 07:37
+
+from django.db import migrations
+
+
+def _drop_if_column_exists(apps, schema_editor) -> None:
+    """列已不存在时跳过（例如旧版 0006 已删过），避免数据库报错。"""
+    JdJobMergedRow = apps.get_model("pipeline", "JdJobMergedRow")
+    table = JdJobMergedRow._meta.db_table
+    conn = schema_editor.connection
+    for fname in ("detail_http_status", "comment_http_status"):
+        with conn.cursor() as cursor:
+            desc = conn.introspection.get_table_description(cursor, table)
+        col_names = {d.name for d in desc}
+        if fname not in col_names:
+            continue
+        field = JdJobMergedRow._meta.get_field(fname)
+        schema_editor.remove_field(JdJobMergedRow, field)
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("pipeline", "0006_trim_search_detail_merged_fields"),
+    ]
+
+    operations = [
+        migrations.SeparateDatabaseAndState(
+            state_operations=[
+                migrations.RemoveField(
+                    model_name="jdjobmergedrow",
+                    name="comment_http_status",
+                ),
+                migrations.RemoveField(
+                    model_name="jdjobmergedrow",
+                    name="detail_http_status",
+                ),
+            ],
+            database_operations=[
+                migrations.RunPython(
+                    _drop_if_column_exists,
+                    migrations.RunPython.noop,
+                ),
+            ],
+        ),
+    ]

backend/pipeline/migrations/0008_repair_jdjobmergedrow_detail_columns.py

@@ -0,0 +1,39 @@
+# SQLite 上曾出现「迁移已记录但表缺少 lean 商详列」的不一致；补全缺失列以便与模型一致。
+
+from django.db import migrations
+
+
+def _repair(apps, schema_editor) -> None:
+    conn = schema_editor.connection
+    if conn.vendor != "sqlite":
+        return
+    JdJobMergedRow = apps.get_model("pipeline", "JdJobMergedRow")
+    table = JdJobMergedRow._meta.db_table
+    # 与 models.JdJobMergedRow 商详块一致
+    additions: dict[str, str] = {
+        "detail_brand": "TEXT NOT NULL DEFAULT ''",
+        "detail_price_final": "TEXT NOT NULL DEFAULT ''",
+        "detail_shop_name": "TEXT NOT NULL DEFAULT ''",
+        "detail_category_path": "TEXT NOT NULL DEFAULT ''",
+        "detail_product_attributes": "TEXT NOT NULL DEFAULT ''",
+    }
+    qn = conn.ops.quote_name
+    t = qn(table)
+    with conn.cursor() as cursor:
+        desc = conn.introspection.get_table_description(cursor, table)
+        have = {d.name for d in desc}
+        for col, ddl in additions.items():
+            if col in have:
+                continue
+            cursor.execute(f"ALTER TABLE {t} ADD COLUMN {qn(col)} {ddl}")
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("pipeline", "0007_drop_merged_http_status_columns"),
+    ]
+
+    operations = [
+        migrations.RunPython(_repair, migrations.RunPython.noop),
+    ]

backend/pipeline/migrations/0009_detail_row_lean_fields.py

@@ -0,0 +1,38 @@
+# Generated by Django 5.2.13 on 2026-04-09 08:05
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0008_repair_jdjobmergedrow_detail_columns'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_brand',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_category_path',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_price_final',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_product_attributes',
+            field=models.TextField(blank=True, default=''),
+        ),
+        migrations.AddField(
+            model_name='jdjobdetailrow',
+            name='detail_shop_name',
+            field=models.TextField(blank=True, default=''),
+        ),
+    ]

backend/pipeline/migrations/0010_pipelinejob_report_config.py

@@ -0,0 +1,18 @@
+# Generated by Django 5.2.13 on 2026-04-09 08:48
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0009_detail_row_lean_fields'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='report_config',
+            field=models.JSONField(blank=True, default=dict),
+        ),
+    ]

backend/pipeline/migrations/0011_job_cancel_and_status.py

@@ -0,0 +1,23 @@
+# Generated by Django 5.2.13 on 2026-04-10 04:16
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('pipeline', '0010_pipelinejob_report_config'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='pipelinejob',
+            name='cancellation_requested',
+            field=models.BooleanField(db_index=True, default=False),
+        ),
+        migrations.AlterField(
+            model_name='pipelinejob',
+            name='status',
+            field=models.CharField(choices=[('pending', '待执行'), ('running', '执行中'), ('success', '成功'), ('failed', '失败'), ('cancelled', '已终止')], db_index=True, default='pending', max_length=16),
+        ),
+    ]

backend/pipeline/models.py

@@ -0,0 +1,287 @@
+from django.db import models
+
+
+class JobStatus(models.TextChoices):
+    PENDING = "pending", "待执行"
+    RUNNING = "running", "执行中"
+    SUCCESS = "success", "成功"
+    FAILED = "failed", "失败"
+    CANCELLED = "cancelled", "已终止"
+
+
+class PipelineJob(models.Model):
+    platform = models.CharField(max_length=32, default="jd", db_index=True)
+    keyword = models.CharField(max_length=256)
+    max_skus = models.PositiveIntegerField(null=True, blank=True)
+    page_start = models.PositiveIntegerField(null=True, blank=True)
+    page_to = models.PositiveIntegerField(null=True, blank=True)
+    # 相对 data/JD 的子路径，或绝对路径（须在 Low GI/data/JD 下）；空则时间戳_关键词
+    pipeline_run_dir = models.TextField(blank=True, default="")
+    # Cookie：二选一优先 cookie_text（任务内写入临时文件再跑流水线）
+    cookie_file_path = models.TextField(blank=True, default="")
+    cookie_text = models.TextField(blank=True, default="")
+    pvid = models.CharField(max_length=128, blank=True, default="")
+    request_delay = models.CharField(
+        max_length=64,
+        blank=True,
+        default="",
+        help_text='如 "30-60"；空则沿用副本默认',
+    )
+    list_pages = models.CharField(
+        max_length=64,
+        blank=True,
+        default="",
+        help_text='评论分页，如 "1-2"；空则沿用副本默认',
+    )
+    scenario_filter_enabled = models.BooleanField(null=True, blank=True)
+    # 竞品报告 / competitor-brief：关注词、场景词组、外部市场表等（JSON，空对象=用爬虫脚本默认）
+    report_config = models.JSONField(default=dict, blank=True)
+    status = models.CharField(
+        max_length=16,
+        choices=JobStatus.choices,
+        default=JobStatus.PENDING,
+        db_index=True,
+    )
+    cancellation_requested = models.BooleanField(default=False, db_index=True)
+    run_dir = models.TextField(blank=True, default="")
+    error_message = models.TextField(blank=True, default="")
+    created_at = models.DateTimeField(auto_now_add=True)
+    updated_at = models.DateTimeField(auto_now=True)
+
+    class Meta:
+        ordering = ["-created_at"]
+
+    def __str__(self) -> str:
+        return f"[{self.platform}] {self.keyword} ({self.status})"
+
+
+class JdProduct(models.Model):
+    """京东 SKU 主档：同一 ``platform`` + ``sku_id`` 唯一，多次抓取时覆盖为最新一行合并表数据。"""
+
+    platform = models.CharField(max_length=16, default="jd", db_index=True)
+    sku_id = models.CharField(max_length=64, db_index=True)
+    ware_id = models.CharField(max_length=64, blank=True, default="")
+    title = models.TextField(blank=True, default="")
+    detail_brand = models.CharField(max_length=512, blank=True, default="")
+    detail_price_final = models.CharField(max_length=128, blank=True, default="")
+    detail_category_path = models.TextField(blank=True, default="")
+    current_payload = models.JSONField(default=dict, blank=True)
+    last_job = models.ForeignKey(
+        PipelineJob,
+        null=True,
+        blank=True,
+        on_delete=models.SET_NULL,
+        related_name="touched_products",
+    )
+    last_captured_at = models.DateTimeField(null=True, blank=True, db_index=True)
+    created_at = models.DateTimeField(auto_now_add=True)
+    updated_at = models.DateTimeField(auto_now=True)
+
+    class Meta:
+        ordering = ["-updated_at"]
+        constraints = [
+            models.UniqueConstraint(
+                fields=["platform", "sku_id"],
+                name="uniq_pipeline_jdproduct_platform_sku",
+            ),
+        ]
+
+    def __str__(self) -> str:
+        return f"{self.platform}:{self.sku_id}"
+
+
+class JdProductSnapshot(models.Model):
+    """某次流水线任务下该 SKU 的整行快照，用于历史对比与回放。"""
+
+    product = models.ForeignKey(
+        JdProduct,
+        on_delete=models.CASCADE,
+        related_name="snapshots",
+    )
+    job = models.ForeignKey(
+        PipelineJob,
+        on_delete=models.CASCADE,
+        related_name="product_snapshots",
+    )
+    run_dir = models.TextField(blank=True, default="")
+    captured_at = models.DateTimeField(db_index=True)
+    payload = models.JSONField(default=dict, blank=True)
+
+    class Meta:
+        ordering = ["-captured_at"]
+        constraints = [
+            models.UniqueConstraint(
+                fields=["product", "job"],
+                name="uniq_pipeline_jdproductsnapshot_product_job",
+            ),
+        ]
+
+    def __str__(self) -> str:
+        return f"{self.product_id} @ job {self.job_id}"
+
+
+class JdJobSearchRow(models.Model):
+    """单次任务下 PC 搜索导出表一行，字段与 ``pc_search_export.csv`` 列一一对应（内部英文属性名）。"""
+
+    job = models.ForeignKey(
+        PipelineJob,
+        on_delete=models.CASCADE,
+        related_name="job_search_rows",
+    )
+    row_index = models.PositiveIntegerField()
+    item_id = models.TextField(blank=True, default="")
+    sku_id = models.TextField(blank=True, default="", db_index=True)
+    title = models.TextField(blank=True, default="")
+    price = models.TextField(blank=True, default="")
+    coupon_price = models.TextField(blank=True, default="")
+    original_price = models.TextField(blank=True, default="")
+    selling_point = models.TextField(blank=True, default="")
+    comment_sales_floor = models.TextField(blank=True, default="")
+    hot_list_rank = models.TextField(blank=True, default="")
+    comment_count = models.TextField(blank=True, default="")
+    shop_name = models.TextField(blank=True, default="")
+    shop_url = models.TextField(blank=True, default="")
+    shop_info_url = models.TextField(blank=True, default="")
+    location = models.TextField(blank=True, default="")
+    detail_url = models.TextField(blank=True, default="")
+    image = models.TextField(blank=True, default="")
+    seckill_info = models.TextField(blank=True, default="")
+    attributes = models.TextField(blank=True, default="")
+    leaf_category = models.TextField(blank=True, default="")
+    platform = models.TextField(blank=True, default="")
+    keyword = models.TextField(blank=True, default="")
+    page = models.TextField(blank=True, default="")
+
+    class Meta:
+        ordering = ["row_index"]
+        constraints = [
+            models.UniqueConstraint(
+                fields=["job", "row_index"],
+                name="uniq_pipeline_jdjobsearchrow_job_idx",
+            ),
+        ]
+        indexes = [
+            models.Index(fields=["job", "sku_id"]),
+        ]
+
+    def __str__(self) -> str:
+        return f"job{self.job_id} search#{self.row_index}"
+
+
+class JdJobDetailRow(models.Model):
+    """单次任务下 ``detail_ware_export.csv`` 一行（lean：skuId + 与合并表一致的商详子集）。"""
+
+    job = models.ForeignKey(
+        PipelineJob,
+        on_delete=models.CASCADE,
+        related_name="job_detail_rows",
+    )
+    row_index = models.PositiveIntegerField()
+    sku_id = models.TextField(blank=True, default="", db_index=True)
+    detail_brand = models.TextField(blank=True, default="")
+    detail_price_final = models.TextField(blank=True, default="")
+    detail_shop_name = models.TextField(blank=True, default="")
+    detail_category_path = models.TextField(blank=True, default="")
+    detail_product_attributes = models.TextField(blank=True, default="")
+    detail_body_ingredients = models.TextField(blank=True, default="")
+
+    class Meta:
+        ordering = ["row_index"]
+        constraints = [
+            models.UniqueConstraint(
+                fields=["job", "row_index"],
+                name="uniq_pipeline_jdjobdetailrow_job_idx",
+            ),
+        ]
+        indexes = [
+            models.Index(fields=["job", "sku_id"]),
+        ]
+
+    def __str__(self) -> str:
+        return f"job{self.job_id} detail#{self.row_index}"
+
+
+class JdJobCommentRow(models.Model):
+    """单次任务下 ``comments_flat.csv`` 一行。"""
+
+    job = models.ForeignKey(
+        PipelineJob,
+        on_delete=models.CASCADE,
+        related_name="job_comment_rows",
+    )
+    row_index = models.PositiveIntegerField()
+    sku_id = models.TextField(blank=True, default="", db_index=True)
+    comment_id = models.TextField(blank=True, default="")
+    user_nick_name = models.TextField(blank=True, default="")
+    tag_comment_content = models.TextField(blank=True, default="")
+    comment_date = models.TextField(blank=True, default="")
+    buy_count_text = models.TextField(blank=True, default="")
+    large_pic_urls = models.TextField(blank=True, default="")
+    comment_score = models.TextField(blank=True, default="")
+
+    class Meta:
+        ordering = ["row_index"]
+        constraints = [
+            models.UniqueConstraint(
+                fields=["job", "row_index"],
+                name="uniq_pipeline_jdjobcommentrow_job_idx",
+            ),
+        ]
+        indexes = [
+            models.Index(fields=["job", "sku_id"]),
+        ]
+
+    def __str__(self) -> str:
+        return f"job{self.job_id} cmt#{self.row_index}"
+
+
+class JdJobMergedRow(models.Model):
+    """单次任务下合并宽表一行（lean：搜索列 + 商详子集 + 评论摘要），与 ``keyword_pipeline_merged.csv`` 列一一对应。"""
+
+    job = models.ForeignKey(
+        PipelineJob,
+        on_delete=models.CASCADE,
+        related_name="job_merged_rows",
+    )
+    row_index = models.PositiveIntegerField()
+    pipeline_keyword = models.TextField(blank=True, default="")
+    sku_id = models.TextField(blank=True, default="", db_index=True)
+    ware_id = models.TextField(blank=True, default="")
+    title = models.TextField(blank=True, default="")
+    price = models.TextField(blank=True, default="")
+    coupon_price = models.TextField(blank=True, default="")
+    original_price = models.TextField(blank=True, default="")
+    selling_point = models.TextField(blank=True, default="")
+    hot_list_rank = models.TextField(blank=True, default="")
+    comment_fuzzy = models.TextField(blank=True, default="")
+    comment_sales_floor = models.TextField(blank=True, default="")
+    shop_name = models.TextField(blank=True, default="")
+    detail_url = models.TextField(blank=True, default="")
+    image = models.TextField(blank=True, default="")
+    attributes = models.TextField(blank=True, default="")
+    leaf_category = models.TextField(blank=True, default="")
+    keyword = models.TextField(blank=True, default="")
+    page = models.TextField(blank=True, default="")
+    detail_brand = models.TextField(blank=True, default="")
+    detail_price_final = models.TextField(blank=True, default="")
+    detail_shop_name = models.TextField(blank=True, default="")
+    detail_category_path = models.TextField(blank=True, default="")
+    detail_product_attributes = models.TextField(blank=True, default="")
+    detail_body_ingredients = models.TextField(blank=True, default="")
+    pipeline_comment_count = models.TextField(blank=True, default="")
+    comment_preview = models.TextField(blank=True, default="")
+
+    class Meta:
+        ordering = ["row_index"]
+        constraints = [
+            models.UniqueConstraint(
+                fields=["job", "row_index"],
+                name="uniq_pipeline_jdjobmergedrow_job_idx",
+            ),
+        ]
+        indexes = [
+            models.Index(fields=["job", "sku_id"]),
+        ]
+
+    def __str__(self) -> str:
+        return f"job{self.job_id} merged#{self.row_index}"

backend/pipeline/report_charts.py

@@ -0,0 +1,378 @@
+"""根据结构化 brief 生成报告用 PNG 统计图（matplotlib），写入 ``run_dir/report_assets/``。"""
+
+from __future__ import annotations
+
+import os
+import re
+from pathlib import Path
+from typing import Any
+
+
+def _setup_matplotlib_cjk() -> None:
+    import matplotlib
+
+    matplotlib.use("Agg")
+    import matplotlib.pyplot as plt
+    from matplotlib import font_manager
+
+    windir = os.environ.get("WINDIR", r"C:\Windows")
+    for name in ("simhei.ttf", "msyh.ttc", "simsun.ttc"):
+        fp = Path(windir) / "Fonts" / name
+        if fp.is_file():
+            try:
+                font_manager.fontManager.addfont(str(fp))
+                fam = font_manager.FontProperties(fname=str(fp)).get_name()
+                plt.rcParams["font.family"] = [fam]
+                break
+            except Exception:
+                continue
+    plt.rcParams["axes.unicode_minus"] = False
+
+
+def _label_count_pairs(
+    items: Any,
+    *,
+    key_label: str = "label",
+    key_count: str = "count",
+    cap: int = 40,
+) -> tuple[list[str], list[float]]:
+    labs: list[str] = []
+    vals: list[float] = []
+    if not isinstance(items, list):
+        return labs, vals
+    for item in items[:cap]:
+        if not isinstance(item, dict):
+            continue
+        lbl = str(item.get(key_label) or "").strip()[:48]
+        cnt = item.get(key_count)
+        if lbl and isinstance(cnt, (int, float)) and cnt > 0:
+            labs.append(lbl)
+            vals.append(float(cnt))
+    return labs, vals
+
+
+def _merge_labeled_counts_tail(
+    pairs: list[tuple[str, float]], *, max_items: int
+) -> list[tuple[str, float]]:
+    if len(pairs) <= max_items:
+        return pairs
+    head = pairs[: max_items - 1]
+    rest = sum(c for _, c in pairs[max_items - 1 :])
+    if rest > 0:
+        head.append(("其他", rest))
+    return head
+
+
+def _merge_tail_as_other(
+    labels: list[str], values: list[float], *, max_slices: int
+) -> tuple[list[str], list[float]]:
+    pairs = [(l, v) for l, v in zip(labels, values) if v > 0]
+    if not pairs:
+        return [], []
+    if len(pairs) <= max_slices:
+        return [p[0] for p in pairs], [p[1] for p in pairs]
+    head = pairs[: max_slices - 1]
+    rest = sum(v for _, v in pairs[max_slices - 1 :])
+    labs = [p[0] for p in head]
+    vals = [p[1] for p in head]
+    if rest > 0:
+        labs.append("其他")
+        vals.append(rest)
+    return labs, vals
+
+
+# 已不再写入报告正文的旧版图，避免 run_dir 里残留误导性 PNG
+_OBSOLETE_REPORT_ASSETS: frozenset[str] = frozenset(
+    {
+        "chart_focus_keywords_bar.png",
+        "chart_usage_scenarios.png",
+        "chart_usage_scenarios_pie.png",
+        "chart_focus_keywords_pie.png",
+    }
+)
+
+
+def _cleanup_obsolete_report_assets(out_dir: Path) -> None:
+    """删除历史版本生成的、当前报告不再引用的插图文件。"""
+    if not out_dir.is_dir():
+        return
+    for name in _OBSOLETE_REPORT_ASSETS:
+        fp = out_dir / name
+        if fp.is_file():
+            try:
+                fp.unlink()
+            except OSError:
+                pass
+    for fp in out_dir.glob("chart_usage_scenarios_pie__*.png"):
+        try:
+            fp.unlink()
+        except OSError:
+            pass
+
+
+def generate_report_charts(run_dir: Path, brief: dict[str, Any]) -> list[str]:
+    """生成扇形/条形 PNG。返回已写入的文件名列表（不含路径）。"""
+    _setup_matplotlib_cjk()
+    import matplotlib.pyplot as plt
+
+    out_dir = Path(run_dir).resolve() / "report_assets"
+    out_dir.mkdir(parents=True, exist_ok=True)
+    _cleanup_obsolete_report_assets(out_dir)
+    created: list[str] = []
+
+    def save_bar_h(
+        labels: list[str],
+        values: list[float],
+        title: str,
+        fname: str,
+        xlabel: str = "",
+    ) -> None:
+        if not labels or not values or max(values) <= 0:
+            return
+        n = len(labels)
+        fig_h = max(3.2, min(14.0, 0.38 * n + 1.5))
+        fig, ax = plt.subplots(figsize=(8.2, fig_h))
+        y_pos = range(n)
+        ax.barh(list(y_pos), values, color="#2563eb", height=0.65)
+        ax.set_yticks(list(y_pos))
+        ax.set_yticklabels(labels, fontsize=9)
+        ax.invert_yaxis()
+        ax.set_title(title, fontsize=12, pad=10)
+        if xlabel:
+            ax.set_xlabel(xlabel, fontsize=9)
+        fig.tight_layout()
+        path = out_dir / fname
+        fig.savefig(path, dpi=130, bbox_inches="tight")
+        plt.close(fig)
+        created.append(fname)
+
+    def save_bar_h_share_of_text(
+        labels: list[str],
+        counts: list[float],
+        n_texts: int,
+        title: str,
+        fname: str,
+    ) -> None:
+        """
+        横轴 = count / n_texts * 100，与报告表格「占有效文本比例」一致（多标签下各柱比例可相加 >100%）。
+        """
+        if not labels or not counts or n_texts <= 0 or max(counts) <= 0:
+            return
+        pcts = [100.0 * c / n_texts for c in counts]
+        n_b = len(labels)
+        fig_h = max(3.2, min(14.0, 0.38 * n_b + 1.8))
+        fig, ax = plt.subplots(figsize=(8.8, fig_h))
+        y_pos = range(n_b)
+        bars = ax.barh(list(y_pos), pcts, color="#2563eb", height=0.65)
+        ax.set_yticks(list(y_pos))
+        ax.set_yticklabels(labels, fontsize=9)
+        ax.invert_yaxis()
+        ax.set_title(title, fontsize=12, pad=10)
+        ax.set_xlabel("占有效评价文本比例（%）", fontsize=9)
+        xmax = max(pcts) * 1.12 + 4.0
+        ax.set_xlim(0, max(xmax, max(pcts) + 10.0, 24.0))
+        for bar, c, p in zip(bars, counts, pcts):
+            ax.text(
+                min(bar.get_width() + 0.6, ax.get_xlim()[1] * 0.97),
+                bar.get_y() + bar.get_height() / 2,
+                f"{int(c)}条 · {p:.1f}%",
+                va="center",
+                fontsize=8,
+            )
+        fig.tight_layout()
+        path = out_dir / fname
+        fig.savefig(path, dpi=130, bbox_inches="tight")
+        plt.close(fig)
+        created.append(fname)
+
+    def save_pie(
+        labels: list[str],
+        values: list[float],
+        title: str,
+        fname: str,
+        *,
+        max_slices: int = 8,
+    ) -> None:
+        labs, vals = _merge_tail_as_other(labels, values, max_slices=max_slices)
+        if not labs or not vals or sum(vals) <= 0:
+            return
+        fig, ax = plt.subplots(figsize=(7.2, 5.4))
+        colors = plt.cm.Set3(range(len(labs)))
+        wedges, _t, autotexts = ax.pie(
+            vals,
+            labels=None,
+            autopct=lambda p: f"{p:.1f}%" if p >= 3.5 else "",
+            pctdistance=0.72,
+            colors=colors,
+            startangle=90,
+        )
+        for t in autotexts:
+            t.set_fontsize(8)
+        ax.legend(
+            wedges,
+            labs,
+            loc="center left",
+            bbox_to_anchor=(1.02, 0.5),
+            fontsize=8,
+            frameon=False,
+        )
+        ax.set_title(title, fontsize=12, pad=12)
+        fig.tight_layout()
+        path = out_dir / fname
+        fig.savefig(path, dpi=130, bbox_inches="tight")
+        plt.close(fig)
+        created.append(fname)
+
+    mix = brief.get("category_mix_top") or []
+    labs_m, vals_m = _label_count_pairs(mix)
+    save_pie(
+        labs_m,
+        vals_m,
+        "类目/可读名称分布（列表行占比）",
+        "chart_category_mix_pie.png",
+    )
+    save_bar_h(
+        labs_m[:15],
+        vals_m[:15],
+        "类目分布（行数，Top）",
+        "chart_category_mix.png",
+        "行数",
+    )
+
+    brand_mix = brief.get("list_brand_mix_top") or []
+    lb, vb = _label_count_pairs(brand_mix, key_label="label")
+    save_pie(
+        lb,
+        vb,
+        "品牌列表曝光占比",
+        "chart_brand_rows_pie.png",
+    )
+
+    shop_mix = brief.get("list_shop_mix_top") or []
+    ls, vs = _label_count_pairs(shop_mix, key_label="label")
+    save_pie(
+        ls,
+        vs,
+        "店铺列表曝光占比",
+        "chart_shop_rows_pie.png",
+    )
+
+    def scenario_group_asset_slug(group: str, index: int) -> str:
+        """与 ``jd_competitor_report._scenario_group_asset_slug`` 保持一致。"""
+        raw = (group or "").strip()
+        core = re.sub(r"[^\w\u4e00-\u9fff-]", "", raw)[:20]
+        if not core:
+            core = "group"
+        return f"i{index:02d}_{core}"
+
+    by_grp = brief.get("usage_scenarios_by_matrix_group") or []
+    if isinstance(by_grp, list):
+        for item in by_grp:
+            if not isinstance(item, dict):
+                continue
+            slug = (item.get("chart_slug") or "").strip()
+            gname = str(item.get("group") or "").strip()[:24]
+            idx = item.get("matrix_group_index")
+            if not slug and gname != "" and isinstance(idx, int):
+                slug = scenario_group_asset_slug(gname, idx)
+            if not slug:
+                continue
+            scen_rows = item.get("scenarios") or []
+            n_unit = int(item.get("effective_text_units") or 0)
+            gpairs: list[tuple[str, float]] = []
+            if isinstance(scen_rows, list):
+                for r in scen_rows:
+                    if not isinstance(r, dict):
+                        continue
+                    lb = str(r.get("scenario") or "").strip()[:48]
+                    c = r.get("count")
+                    if lb and isinstance(c, (int, float)) and c > 0:
+                        gpairs.append((lb, float(c)))
+            gpairs = _merge_labeled_counts_tail(gpairs, max_items=14)
+            if gpairs and n_unit > 0:
+                gl = [p[0] for p in gpairs]
+                gv = [p[1] for p in gpairs]
+                title_base = f"「{gname}」· 场景/用途" if gname else "细类 · 场景/用途"
+                save_bar_h_share_of_text(
+                    gl,
+                    gv,
+                    n_unit,
+                    f"{title_base}（占有效评价文本比例）",
+                    f"chart_usage_scenarios_bar__{slug}.png",
+                )
+
+    fb = brief.get("consumer_feedback_by_matrix_group") or []
+    if isinstance(fb, list):
+        for item in fb:
+            if not isinstance(item, dict):
+                continue
+            slug = (item.get("chart_slug") or "").strip()
+            gname = str(item.get("group") or "").strip()[:24]
+            idx = item.get("matrix_group_index")
+            if not slug and gname != "" and isinstance(idx, int):
+                slug = scenario_group_asset_slug(gname, idx)
+            if not slug:
+                continue
+            hk = item.get("focus_keyword_hits") or []
+            wl: list[str] = []
+            vl: list[float] = []
+            if isinstance(hk, list):
+                for row in hk[:20]:
+                    if not isinstance(row, dict):
+                        continue
+                    w = str(row.get("word") or "").strip()[:32]
+                    c = row.get("count")
+                    if w and isinstance(c, (int, float)) and c > 0:
+                        wl.append(w)
+                        vl.append(float(c))
+            wl = wl[:18]
+            vl = vl[:18]
+            if not wl:
+                continue
+            tkw = f"「{gname}」· 关注词命中次数" if gname else "细类 · 关注词命中次数"
+            save_bar_h(wl, vl, tkw, f"chart_focus_keywords_bar__{slug}.png", "命中次数")
+
+    sent = brief.get("comment_sentiment_lexicon") or {}
+    if isinstance(sent, dict):
+        pie_labs = ["偏正向", "偏负向", "正负混合", "中性/空"]
+        pie_vals = [
+            float(sent.get("positive_only") or 0),
+            float(sent.get("negative_only") or 0),
+            float(sent.get("mixed_positive_and_negative") or 0),
+            float(sent.get("neutral_or_empty") or 0),
+        ]
+        pl = [a for a, b in zip(pie_labs, pie_vals) if b > 0]
+        pv = [b for b in pie_vals if b > 0]
+        save_pie(pl, pv, "评价语气四象限占比", "chart_sentiment_overview_pie.png")
+        save_bar_h(
+            pl,
+            pv,
+            "评价正负面粗判（条数）",
+            "chart_sentiment.png",
+            "条数",
+        )
+
+        pos_h = sent.get("positive_tone_lexeme_hits") or []
+        neg_h = sent.get("negative_tone_lexeme_hits") or []
+        plx, pvx = _label_count_pairs(
+            pos_h, key_label="word", key_count="texts_matched", cap=16
+        )
+        save_bar_h(
+            plx,
+            pvx,
+            "正向/混合语境 · 正向口语短语命中条数",
+            "chart_positive_lexemes_bar.png",
+            "条数",
+        )
+        nlx, nvx = _label_count_pairs(
+            neg_h, key_label="word", key_count="texts_matched", cap=16
+        )
+        save_bar_h(
+            nlx,
+            nvx,
+            "负向/混合语境 · 负向口语短语命中条数",
+            "chart_negative_lexemes_bar.png",
+            "条数",
+        )
+
+    return created

backend/pipeline/row_serialize.py

@@ -0,0 +1,46 @@
+"""任务数据集 ORM 行 ↔ API / 导出用的扁平字典。"""
+from __future__ import annotations
+
+from typing import Any
+
+from .csv_schema import (
+    COMMENT_CSV_COLUMNS,
+    COMMENT_CSV_TO_FIELD,
+    DETAIL_CSV_COLUMNS,
+    DETAIL_CSV_TO_FIELD,
+    JD_SEARCH_INTERNAL_KEYS,
+    MERGED_INTERNAL_KEYS,
+)
+from .models import JdJobCommentRow, JdJobDetailRow, JdJobMergedRow, JdJobSearchRow
+
+DETAIL_FIELDS_ORDER: tuple[str, ...] = tuple(DETAIL_CSV_TO_FIELD[c] for c in DETAIL_CSV_COLUMNS)
+COMMENT_FIELDS_ORDER: tuple[str, ...] = tuple(COMMENT_CSV_TO_FIELD[c] for c in COMMENT_CSV_COLUMNS)
+MERGED_FIELDS_ORDER: tuple[str, ...] = MERGED_INTERNAL_KEYS
+
+
+def search_row_to_dict(r: JdJobSearchRow) -> dict[str, Any]:
+    out: dict[str, Any] = {"id": r.id, "row_index": r.row_index}
+    for k in JD_SEARCH_INTERNAL_KEYS:
+        out[k] = getattr(r, k) or ""
+    return out
+
+
+def detail_row_to_dict(r: JdJobDetailRow) -> dict[str, Any]:
+    out: dict[str, Any] = {"id": r.id, "row_index": r.row_index}
+    for k in DETAIL_FIELDS_ORDER:
+        out[k] = getattr(r, k) or ""
+    return out
+
+
+def comment_row_to_dict(r: JdJobCommentRow) -> dict[str, Any]:
+    out: dict[str, Any] = {"id": r.id, "row_index": r.row_index}
+    for k in COMMENT_FIELDS_ORDER:
+        out[k] = getattr(r, k) or ""
+    return out
+
+
+def merged_row_to_dict(r: JdJobMergedRow) -> dict[str, Any]:
+    out: dict[str, Any] = {"id": r.id, "row_index": r.row_index}
+    for k in MERGED_FIELDS_ORDER:
+        out[k] = getattr(r, k) or ""
+    return out

backend/pipeline/serializers.py

@@ -0,0 +1,329 @@
+import json
+from pathlib import Path
+
+from django.conf import settings
+from rest_framework import serializers
+
+from .cookie_paste import normalize_browser_cookie_paste
+from .models import JdProduct, JdProductSnapshot, JobStatus, PipelineJob
+
+# 与 views._safe_file_for_job 中 mapping 一致，供前端展示「数据源是否就绪」
+_REPORT_CONFIG_ALLOWED_KEYS = frozenset(
+    {
+        "llm_comment_sentiment",
+        "comment_focus_words",
+        "comment_scenario_groups",
+        "external_market_table_rows",
+    }
+)
+
+
+def validate_report_config_body(value: dict) -> dict:
+    if not isinstance(value, dict):
+        raise serializers.ValidationError("须为 JSON 对象")
+    extra = set(value.keys()) - _REPORT_CONFIG_ALLOWED_KEYS
+    if extra:
+        raise serializers.ValidationError(
+            f"未知字段：{', '.join(sorted(extra))}"
+        )
+    if "llm_comment_sentiment" in value and value["llm_comment_sentiment"] is not None:
+        if not isinstance(value["llm_comment_sentiment"], bool):
+            raise serializers.ValidationError("llm_comment_sentiment 须为 true 或 false")
+    raw = json.dumps(value, ensure_ascii=False)
+    if len(raw) > 120_000:
+        raise serializers.ValidationError("报告配置体积过大")
+    return value
+
+
+_ARTIFACT_FILES: tuple[tuple[str, str], ...] = (
+    ("merged", "keyword_pipeline_merged.csv"),
+    ("pc_search", "pc_search_export.csv"),
+    ("comments", "comments_flat.csv"),
+    ("detail_ware", "detail_ware_export.csv"),
+    ("report", "competitor_analysis.md"),
+)
+
+
+class PipelineJobSerializer(serializers.ModelSerializer):
+    """列表/详情不返回 cookie 正文。"""
+
+    inline_cookie_used = serializers.SerializerMethodField()
+    analysis_artifacts = serializers.SerializerMethodField()
+
+    class Meta:
+        model = PipelineJob
+        fields = [
+            "id",
+            "platform",
+            "keyword",
+            "max_skus",
+            "page_start",
+            "page_to",
+            "pipeline_run_dir",
+            "cookie_file_path",
+            "inline_cookie_used",
+            "pvid",
+            "request_delay",
+            "list_pages",
+            "scenario_filter_enabled",
+            "report_config",
+            "status",
+            "cancellation_requested",
+            "run_dir",
+            "error_message",
+            "analysis_artifacts",
+            "created_at",
+            "updated_at",
+        ]
+        read_only_fields = [
+            "id",
+            "inline_cookie_used",
+            "analysis_artifacts",
+            "status",
+            "cancellation_requested",
+            "run_dir",
+            "error_message",
+            "created_at",
+            "updated_at",
+            "report_config",
+        ]
+
+    def get_inline_cookie_used(self, obj: PipelineJob) -> bool:
+        return bool((obj.cookie_text or "").strip())
+
+    def get_analysis_artifacts(self, obj: PipelineJob) -> dict[str, bool] | None:
+        if obj.status not in (JobStatus.SUCCESS, JobStatus.CANCELLED) or not (
+            obj.run_dir or ""
+        ).strip():
+            return None
+        try:
+            base = Path(obj.run_dir).expanduser().resolve()
+            return { key: (base / name).is_file() for key, name in _ARTIFACT_FILES }
+        except (OSError, ValueError, RuntimeError):
+            return None
+
+
+class JdProductListSerializer(serializers.ModelSerializer):
+    """列表：不含整包 payload，减少流量。"""
+
+    snapshot_count = serializers.IntegerField(read_only=True, required=False)
+
+    class Meta:
+        model = JdProduct
+        fields = [
+            "id",
+            "platform",
+            "sku_id",
+            "ware_id",
+            "title",
+            "detail_brand",
+            "detail_price_final",
+            "last_captured_at",
+            "last_job",
+            "snapshot_count",
+        ]
+
+
+class JdProductDetailSerializer(serializers.ModelSerializer):
+    snapshot_count = serializers.IntegerField(read_only=True, required=False)
+
+    class Meta:
+        model = JdProduct
+        fields = [
+            "id",
+            "platform",
+            "sku_id",
+            "ware_id",
+            "title",
+            "detail_brand",
+            "detail_price_final",
+            "detail_category_path",
+            "current_payload",
+            "last_job",
+            "last_captured_at",
+            "snapshot_count",
+            "created_at",
+            "updated_at",
+        ]
+
+
+class JdProductSnapshotBriefSerializer(serializers.ModelSerializer):
+    job_keyword = serializers.CharField(source="job.keyword", read_only=True)
+
+    class Meta:
+        model = JdProductSnapshot
+        fields = ["id", "job", "job_keyword", "run_dir", "captured_at"]
+
+
+class JdProductSnapshotDetailSerializer(serializers.ModelSerializer):
+    job_keyword = serializers.CharField(source="job.keyword", read_only=True)
+    sku_id = serializers.CharField(source="product.sku_id", read_only=True)
+    platform = serializers.CharField(source="product.platform", read_only=True)
+
+    class Meta:
+        model = JdProductSnapshot
+        fields = [
+            "id",
+            "platform",
+            "sku_id",
+            "job",
+            "job_keyword",
+            "run_dir",
+            "captured_at",
+            "payload",
+        ]
+
+
+def _jd_data_root() -> Path:
+    root = (settings.LOW_GI_PROJECT_ROOT or "").strip()
+    if not root:
+        raise serializers.ValidationError("服务器未配置 LOW_GI_PROJECT_ROOT")
+    return (Path(root) / "data" / "JD").resolve()
+
+
+class CreatePipelineJobSerializer(serializers.Serializer):
+    keyword = serializers.CharField(max_length=256, trim_whitespace=True)
+    platform = serializers.ChoiceField(choices=["jd"], default="jd")
+    max_skus = serializers.IntegerField(required=False, min_value=1, allow_null=True)
+    page_start = serializers.IntegerField(required=False, min_value=1, allow_null=True)
+    page_to = serializers.IntegerField(required=False, min_value=1, allow_null=True)
+    pipeline_run_dir = serializers.CharField(
+        required=False, allow_blank=True, max_length=1024, default=""
+    )
+    cookie_file_path = serializers.CharField(
+        required=False, allow_blank=True, max_length=2048, default=""
+    )
+    cookie_text = serializers.CharField(
+        required=False,
+        allow_blank=True,
+        default="",
+        max_length=500_000,
+    )
+    pvid = serializers.CharField(required=False, allow_blank=True, max_length=128, default="")
+    request_delay = serializers.CharField(
+        required=False, allow_blank=True, max_length=64, default=""
+    )
+    list_pages = serializers.CharField(
+        required=False, allow_blank=True, max_length=64, default=""
+    )
+    scenario_filter_enabled = serializers.BooleanField(required=False, allow_null=True)
+    report_config = serializers.JSONField(required=False, default=dict)
+
+    def validate_report_config(self, value):
+        if not value:
+            return {}
+        return validate_report_config_body(value)
+
+    def validate_cookie_text(self, value: str) -> str:
+        return normalize_browser_cookie_paste(value or "")
+
+    def validate_pipeline_run_dir(self, value: str) -> str:
+        v = (value or "").strip()
+        if not v:
+            return ""
+        p = Path(v).expanduser()
+        jd_root = _jd_data_root()
+        if p.is_absolute():
+            try:
+                p.resolve().relative_to(jd_root)
+            except ValueError:
+                raise serializers.ValidationError(
+                    f"绝对路径须位于京东数据目录下：{jd_root}"
+                )
+        else:
+            bad = ("..",)
+            if any(part in bad for part in p.parts):
+                raise serializers.ValidationError("路径不能包含 ..")
+        return v
+
+    def validate_cookie_file_path(self, value: str) -> str:
+        v = (value or "").strip()
+        if not v:
+            return ""
+        p = Path(v).expanduser().resolve()
+        if not p.is_file():
+            raise serializers.ValidationError(f"Cookie 文件不存在：{p}")
+        low = Path(settings.LOW_GI_PROJECT_ROOT).resolve()
+        try:
+            p.relative_to(low)
+        except ValueError:
+            raise serializers.ValidationError(
+                "Cookie 文件路径须位于 LOW_GI_PROJECT_ROOT 目录之下"
+            )
+        return str(p)
+
+
+class JobReportConfigPatchSerializer(serializers.Serializer):
+    report_config = serializers.JSONField()
+
+    def validate_report_config(self, value):
+        if not isinstance(value, dict):
+            raise serializers.ValidationError("须为 JSON 对象")
+        return validate_report_config_body(value)
+
+
+class RegenerateReportRequestSerializer(serializers.Serializer):
+    """重新生成竞品报告：规则引擎或大模型（与 ``AI_crawler.chat_completion_text`` 同一网关）。"""
+
+    generator = serializers.ChoiceField(
+        choices=["rules", "llm"],
+        default="rules",
+        required=False,
+    )
+
+
+class StrategyDraftRequestSerializer(serializers.Serializer):
+    """市场策略制定：业务备注 + 可选「决策填空/勾选」，与 competitor-brief 合并为策略向 Markdown。"""
+
+    business_notes = serializers.CharField(
+        required=False,
+        allow_blank=True,
+        default="",
+        max_length=20_000,
+        trim_whitespace=False,
+    )
+    product_role = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=500, trim_whitespace=False
+    )
+    time_horizon = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=200, trim_whitespace=False
+    )
+    success_criteria = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=2000, trim_whitespace=False
+    )
+    non_goals = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=1000, trim_whitespace=False
+    )
+    battlefield_one_line = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=1000, trim_whitespace=False
+    )
+    positioning_choice = serializers.ChoiceField(
+        choices=["", "top", "mid", "entry", "different"],
+        default="",
+        required=False,
+    )
+    competitive_stance = serializers.ChoiceField(
+        choices=["", "flank", "head_on", "both", "undecided"],
+        default="",
+        required=False,
+    )
+    pillar_product = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=800, trim_whitespace=False
+    )
+    pillar_price = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=800, trim_whitespace=False
+    )
+    pillar_channel = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=800, trim_whitespace=False
+    )
+    pillar_comm = serializers.CharField(
+        required=False, allow_blank=True, default="", max_length=800, trim_whitespace=False
+    )
+    ack_risk_keywords = serializers.BooleanField(required=False, default=False)
+    ack_risk_price = serializers.BooleanField(required=False, default=False)
+    ack_risk_concentration = serializers.BooleanField(required=False, default=False)
+    generator = serializers.ChoiceField(
+        choices=["rules", "llm"],
+        default="rules",
+        required=False,
+    )

backend/pipeline/strategy_draft.py

@@ -0,0 +1,363 @@
+"""
+市场策略 Markdown 草稿：侧重**策略制定框架**（目标、战场、定位、支柱、行动），
+基于同任务结构化摘要与可选业务备注规则生成；附录为关键数据速览。
+
+后续可接 LLM 润色；当前无模型调用，便于验收与追溯。
+"""
+from __future__ import annotations
+
+import math
+from typing import Any
+
+
+def _esc(s: Any) -> str:
+    t = "" if s is None else str(s).strip()
+    return t.replace("\r\n", "\n").replace("\r", "\n")
+
+
+def _pct(x: Any) -> str:
+    if x is None:
+        return "—"
+    try:
+        v = float(x)
+        if math.isnan(v) or math.isinf(v):
+            return "—"
+        return f"{100 * v:.1f}%"
+    except (TypeError, ValueError):
+        return "—"
+
+
+def _num(x: Any) -> str:
+    if x is None:
+        return "—"
+    if isinstance(x, bool):
+        return str(x)
+    if isinstance(x, int):
+        return str(x)
+    if isinstance(x, float):
+        if math.isnan(x) or math.isinf(x):
+            return "—"
+        if x == int(x):
+            return str(int(x))
+        return f"{x:.2f}"
+    return str(x)
+
+
+def _cr_narrative(label: str, cr1: Any, cr3: Any, top: Any) -> str | None:
+    """从集中度生成一句策略向描述，无数据则返回 None。"""
+    try:
+        c1 = float(cr1) if cr1 is not None else None
+    except (TypeError, ValueError):
+        c1 = None
+    if c1 is None and not (top or "").strip():
+        return None
+    top_s = _esc(top) or "—"
+    if c1 is not None:
+        if c1 >= 0.4:
+            tone = "偏高，头部资源集中"
+        elif c1 >= 0.25:
+            tone = "中等，存在可争夺空间"
+        else:
+            tone = "相对分散，差异化切入点可能更多"
+        return f"- **{label}**：第一大品牌/店份额 ≈ {_pct(cr1)}，前三合计份额 ≈ {_pct(cr3)}；头部为「{top_s}」。*粗判：{tone}。*"
+    return f"- **{label}**：头部标签「{top_s}」（缺少份额指标时可结合列表/商详数据补全）。"
+
+
+def _goal_bullet(label: str, user_val: str, placeholder: str) -> str:
+    v = _esc(user_val).strip()
+    if v:
+        return f"- **{label}**：{v}"
+    return f"- **{label}**：*（{placeholder}）*"
+
+
+def _pillar_cell(user_val: str) -> str:
+    v = _esc(user_val).strip()
+    return v if v else "*待填*"
+
+
+def _pos_mark(choice: str, key: str) -> str:
+    return "[x]" if choice == key else "[ ]"
+
+
+def _risk_line(checked: bool, text: str) -> str:
+    mark = "[x]" if checked else "[ ]"
+    return f"- {mark} {text}"
+
+
+def build_strategy_draft_markdown(
+    *,
+    job_id: int,
+    keyword: str,
+    brief: dict[str, Any],
+    business_notes: str = "",
+    generated_at_iso: str = "",
+    strategy_decisions: dict[str, Any] | None = None,
+) -> str:
+    """生成可下载的 Markdown：策略框架为主，附录为数据速览。"""
+    d = strategy_decisions or {}
+    pos = _esc(d.get("positioning_choice") or "").strip()
+    kw = _esc(brief.get("keyword")) or _esc(keyword) or "—"
+    lines: list[str] = [
+        f"# 市场策略制定草稿 · 「{kw}」",
+        "",
+        "> 本稿用于**辅助制定市场策略**；由规则根据本批次结构化摘要与业务备注生成，**非大模型自由发挥**，定稿前请业务修订。",
+        "",
+    ]
+    if generated_at_iso:
+        lines.append(f"> **生成时间**：{_esc(generated_at_iso)}  ·  **任务 ID**：{job_id}")
+        lines.append("")
+
+    lines.extend(
+        [
+            "---",
+            "",
+            "## 一、战略背景与目标（请业务补全）",
+            "",
+            _goal_bullet("本品角色", str(d.get("product_role") or ""), "新品 / 追赶 / 防守 / 拓品类 …"),
+            _goal_bullet("时间范围", str(d.get("time_horizon") or ""), "如：本季度 / 未来 12 周"),
+            _goal_bullet(
+                "成功标准（可量化）",
+                str(d.get("success_criteria") or ""),
+                "如：搜索位次、转化率、声量、复购 …",
+            ),
+            _goal_bullet("非目标（明确不做什么）", str(d.get("non_goals") or ""), "可选"),
+            "",
+        ]
+    )
+
+    scope = brief.get("scope") or {}
+    merged_n = scope.get("merged_sku_count")
+    comm_n = scope.get("comment_flat_rows")
+    lines.extend(
+        [
+            "## 二、战场界定（监测语境）",
+            "",
+            f"- **监测关键词 / 货架语境**：{kw}",
+            f"- **批次**：{_esc(brief.get('batch_label')) or '—'}",
+        ]
+    )
+    if merged_n is not None or comm_n is not None:
+        lines.append(
+            f"- **深入样本规模**：深入 SKU ≈ {_num(merged_n)}；评价扁平条数 ≈ {_num(comm_n)}。"
+            "*策略含义：样本越大，以下「假设」越需抽样复核原评论。*"
+        )
+    bf = _esc(d.get("battlefield_one_line") or "").strip()
+    if bf:
+        lines.append(f"- **一句话战场**：{bf}")
+    else:
+        lines.append(
+            "- **一句话战场**：*（请用业务语言写：我们在哪个需求场景、与谁抢同一批用户？）*"
+        )
+    lines.append("")
+
+    conc = brief.get("concentration") or {}
+    shops = conc.get("shops_from_list") or {}
+    dbrand = conc.get("detail_brand_among_merged") or {}
+    lines.extend(["## 三、竞争格局 → 策略含义", ""])
+    n_shop = _cr_narrative("列表侧店铺集中度", shops.get("cr1"), shops.get("cr3"), shops.get("top_label"))
+    n_brand = _cr_narrative("深入样本内品牌集中度", dbrand.get("cr1"), dbrand.get("cr3"), dbrand.get("top_label"))
+    if n_shop:
+        lines.append(n_shop)
+    if n_brand:
+        lines.append(n_brand)
+    if not n_shop and not n_brand:
+        lines.append("*本摘要未含集中度指标，请结合本批次竞争结构数据补全后再写判断。*")
+    lines.extend(
+        [
+            "",
+            "**可下判断的提问（自测）**",
+            "",
+            "- 若头部已占稳心智，本品是**侧翼**还是**正面替代**？",
+            "- 店铺/品牌分散时，是否适合用**细分场景**或**内容教育**切入？",
+            "",
+        ]
+    )
+    stance = _esc(d.get("competitive_stance") or "").strip()
+    stance_line = {
+        "flank": "- **本品倾向**：倾向**侧翼切入**，避免与头部正面硬碰。",
+        "head_on": "- **本品倾向**：倾向**正面替代**，对标头部主战场。",
+        "both": "- **本品倾向**：计划**分层推进**（部分场景侧翼、部分场景正面）。",
+        "undecided": "- **本品倾向**：**尚未拍板**，需在会议中对齐后再定主战场叙事。",
+    }.get(stance)
+    if stance_line:
+        lines.append(stance_line)
+        lines.append("")
+
+    mix = brief.get("category_mix_top") or []
+    if mix:
+        lines.append("### 类目结构提示（Top）")
+        lines.append("")
+        lines.append("*以下仅作「货架长什么样」的速记。*")
+        for item in mix[:6]:
+            if isinstance(item, dict):
+                lines.append(f"- {_esc(item.get('label'))}：{_num(item.get('count'))}")
+        lines.append("")
+
+    pst = brief.get("price_stats") or {}
+    lines.extend(["## 四、价格带与定位选项（启发式）", ""])
+    if pst.get("n"):
+        src = _esc(brief.get("price_stats_source")) or "—"
+        lines.extend(
+            [
+                f"- **统计口径**：{src}，有效价样本 n = {_num(pst.get('n'))}。",
+                f"- **展示价区间**：{_num(pst.get('min'))} ～ {_num(pst.get('max'))}；**中位数** {_num(pst.get('median'))}。",
+                "",
+                "**定位选项（请勾一条或改写，并写明理由）**",
+                "",
+                f"- {_pos_mark(pos, 'top')} **贴顶**：对标中高位或头部价位带，强调品质/成分/背书。",
+                f"- {_pos_mark(pos, 'mid')} **卡腰**：围绕中位数一带，强调性价比与场景匹配。",
+                f"- {_pos_mark(pos, 'entry')} **下探**：贴近区间下限，强调入门与拉新（注意毛利与品牌调性）。",
+                f"- {_pos_mark(pos, 'different')} **另起带**：刻意避开主价格带，用规格/组合/服务差异化。",
+                "",
+            ]
+        )
+    else:
+        lines.append("*摘要中无价带统计，请结合本批次价格相关数据补全后再填上表。*")
+        lines.append("")
+        lines.extend(
+            [
+                "**定位选项（请勾一条或改写，并写明理由）**",
+                "",
+                f"- {_pos_mark(pos, 'top')} **贴顶**：对标中高位或头部价位带，强调品质/成分/背书。",
+                f"- {_pos_mark(pos, 'mid')} **卡腰**：围绕中位数一带，强调性价比与场景匹配。",
+                f"- {_pos_mark(pos, 'entry')} **下探**：贴近区间下限，强调入门与拉新（注意毛利与品牌调性）。",
+                f"- {_pos_mark(pos, 'different')} **另起带**：刻意避开主价格带，用规格/组合/服务差异化。",
+                "",
+            ]
+        )
+
+    ckw = brief.get("comment_focus_keywords") or []
+    usc = brief.get("usage_scenarios") or []
+    lines.extend(["## 五、用户需求与场景 — 可写成策略的假设", ""])
+    lines.append(
+        "*下列由关注词/场景**计数**转化而来，是「待验证假设」而非结论；请结合评价原文抽样修订。*"
+    )
+    lines.append("")
+    if ckw:
+        for item in ckw[:8]:
+            if isinstance(item, dict):
+                w = _esc(item.get("word"))
+                c = _num(item.get("count"))
+                lines.append(
+                    f"- **假设**：用户决策中「{w}」被频繁提及（约 {c} 次统计命中）—— "
+                    f"*可追问：本品故事是否正面回应？传播关键词是否覆盖？*"
+                )
+    if usc:
+        for item in usc[:6]:
+            if isinstance(item, dict):
+                sc = _esc(item.get("scenario"))
+                cn = _num(item.get("count"))
+                sh = _pct(item.get("share_of_text_units"))
+                lines.append(
+                    f"- **场景命题**：「{sc}」在预设场景中约 {cn} 条、约占 {sh} 文本单元—— "
+                    f"*可追问：主图/详情/客服话术是否对齐该场景？*"
+                )
+    if not ckw and not usc:
+        lines.append("*摘要中无关注词/场景组结果，请补全评论侧分析后再写本节。*")
+    lines.append("")
+
+    hints = brief.get("strategy_hints") or []
+    lines.extend(
+        [
+            "## 六、机会方向与策略支柱（草案）",
+            "",
+            "### 规则引擎提示（来自摘要 `strategy_hints`）",
+            "",
+        ]
+    )
+    if hints:
+        for h in hints:
+            lines.append(f"- {_esc(h)}")
+    else:
+        lines.append("*（当前无自动线索，请结合本批次结论手写 3～5 条机会）*")
+    pp = str(d.get("pillar_product") or "")
+    pr = str(d.get("pillar_price") or "")
+    pch = str(d.get("pillar_channel") or "")
+    pcm = str(d.get("pillar_comm") or "")
+    lines.extend(
+        [
+            "",
+            "### 策略支柱 — 请业务逐项填空",
+            "",
+            "| 支柱 | 本品打算怎么做 | 与头部差异 | 证据 / 出处 |",
+            "|------|----------------|------------|-------------|",
+            f"| 产品 | {_pillar_cell(pp)} | *待填* | *§* |",
+            f"| 价格 | {_pillar_cell(pr)} | *待填* | *§* |",
+            f"| 渠道/触点 | {_pillar_cell(pch)} | *待填* | *§* |",
+            f"| 传播与内容 | {_pillar_cell(pcm)} | *待填* | *§* |",
+            "",
+        ]
+    )
+
+    rk = bool(d.get("ack_risk_keywords"))
+    rp = bool(d.get("ack_risk_price"))
+    rc = bool(d.get("ack_risk_concentration"))
+    lines.extend(
+        [
+            "## 七、风险与待证伪",
+            "",
+            _risk_line(rk, "关注词/场景是否**以偏概全**？（需原评论抽样）"),
+            _risk_line(rp, "价格带是否含大促/异常挂价？（需核对清洗口径）"),
+            _risk_line(rc, "列表集中度与深入样本品牌是否**矛盾**？（需解释渠道差异）"),
+            "",
+        ]
+    )
+
+    notes = _esc(business_notes)
+    lines.extend(
+        [
+            "## 八、业务约束与内部判断",
+            "",
+            (notes if notes else "*（未填写。建议补充：渠道红线、价位策略、竞品对标名单、预算量级等。）*"),
+            "",
+        ]
+    )
+
+    lines.extend(
+        [
+            "## 九、建议下一步（策略向）",
+            "",
+            "- [ ] 开会对齐：**§一** 目标与 **§八** 约束，确认 1～2 条主策略命题。",
+            "- [ ] 为 **§六** 策略支柱表格每一行各找 **1 条数据证据**（注明出处）。",
+            "- [ ] 产出 **12 周节奏表**（里程碑 + 负责人），与本品排期挂钩。",
+            "- [ ] 定义 **3 个可观测指标**（周或双周复盘）。",
+            "",
+            "---",
+            "",
+            "## 附录 · 本任务关键数据速览",
+            "",
+            f"- **关键词**：{kw}  ·  **摘要版本**：v{_num(brief.get('schema_version'))}",
+        ]
+    )
+    meta = brief.get("meta")
+    meta_labels = {
+        "page_start": "起始页",
+        "page_to": "采集至页",
+        "max_skus_config": "SKU 上限",
+        "scenario_filter_enabled": "场景筛选",
+    }
+    if isinstance(meta, dict) and meta:
+        bits = []
+        for k in ("page_start", "page_to", "max_skus_config", "scenario_filter_enabled"):
+            if k in meta:
+                label = meta_labels.get(k, k)
+                bits.append(f"{label}={_esc(meta.get(k))}")
+        if bits:
+            lines.append(f"- **采集参数快照**：{'; '.join(bits)}")
+    raw = brief.get("pc_search_raw") or {}
+    if raw.get("result_count_consensus") is not None:
+        lines.append(
+            f"- **列表申报规模（resultCount）**：{_num(raw.get('result_count_consensus'))}"
+        )
+    lines.extend(
+        [
+            "",
+            "*同目录含本批次 CSV 与分析产出，可对照使用。*",
+            "",
+            "---",
+            "",
+            "*本稿由工作台「市场策略制定」生成；与同任务结构化分析数据一致。*",
+            "",
+        ]
+    )
+    return "\n".join(lines)

backend/pipeline/tasks.py

@@ -0,0 +1,151 @@
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+import time
+import traceback
+from pathlib import Path
+
+from django.conf import settings
+from django.utils import timezone
+
+from .cookie_paste import normalize_browser_cookie_paste
+from .ingest import try_ingest_job_full
+from .jd_runner import (
+    resolve_pipeline_run_directory_for_job,
+    try_write_competitor_report_if_merged_exists,
+)
+from .models import JobStatus, PipelineJob
+
+
+def execute_job(job_id: int) -> None:
+    job = PipelineJob.objects.filter(pk=job_id).first()
+    if not job:
+        return
+
+    job.status = JobStatus.RUNNING
+    job.error_message = ""
+    job.save(update_fields=["status", "error_message", "updated_at"])
+
+    job.refresh_from_db()
+    if job.cancellation_requested:
+        job.status = JobStatus.CANCELLED
+        job.cancellation_requested = False
+        job.error_message = "已终止（任务开始后立即收到终止请求）。"
+        job.updated_at = timezone.now()
+        job.save(
+            update_fields=[
+                "status",
+                "cancellation_requested",
+                "error_message",
+                "updated_at",
+            ],
+        )
+        PipelineJob.objects.filter(pk=job_id).update(cookie_text="")
+        return
+
+    cookie_temp: Path | None = None
+    try:
+        if job.platform != "jd":
+            raise ValueError(f"暂不支持平台: {job.platform}")
+
+        cookie_path_for_pipeline: str | None = None
+        _cookie_body = normalize_browser_cookie_paste(job.cookie_text or "")
+        if _cookie_body:
+            runtime_dir = Path(settings.BASE_DIR) / "runtime_cookies"
+            runtime_dir.mkdir(parents=True, exist_ok=True)
+            cookie_temp = (runtime_dir / f"job_{job_id}_cookie.txt").resolve()
+            cookie_temp.write_text(_cookie_body, encoding="utf-8")
+            cookie_path_for_pipeline = str(cookie_temp)
+        elif (job.cookie_file_path or "").strip():
+            cookie_path_for_pipeline = job.cookie_file_path.strip()
+
+        rc_cfg = job.report_config if isinstance(job.report_config, dict) else {}
+        run_dir_path = resolve_pipeline_run_directory_for_job(job)
+        run_dir_path.mkdir(parents=True, exist_ok=True)
+
+        manage_py = Path(settings.BASE_DIR) / "manage.py"
+        env = os.environ.copy()
+        env["PIPELINE_JOB_RUN_DIR"] = str(run_dir_path.resolve())
+        if cookie_path_for_pipeline:
+            env["PIPELINE_JOB_COOKIE_PATH"] = str(cookie_path_for_pipeline)
+
+        proc = subprocess.Popen(
+            [sys.executable, str(manage_py), "run_pipeline_job", str(job_id)],
+            cwd=str(Path(settings.BASE_DIR).resolve()),
+            env=env,
+            stdin=subprocess.DEVNULL,
+        )
+        user_terminated = False
+        while True:
+            if proc.poll() is not None:
+                break
+            time.sleep(0.25)
+            if PipelineJob.objects.filter(
+                pk=job_id, cancellation_requested=True
+            ).exists():
+                user_terminated = True
+                proc.terminate()
+                break
+
+        if proc.poll() is None:
+            try:
+                proc.wait(timeout=25)
+            except subprocess.TimeoutExpired:
+                proc.kill()
+                try:
+                    proc.wait(timeout=15)
+                except subprocess.TimeoutExpired:
+                    pass
+        else:
+            proc.wait()
+
+        returncode = proc.returncode
+        if returncode is None:
+            returncode = -1
+
+        if user_terminated:
+            job.status = JobStatus.CANCELLED
+            job.run_dir = str(run_dir_path.resolve())
+            job.cancellation_requested = False
+            job.error_message = (
+                "已终止：已结束采集子进程（与在终端对脚本按 Ctrl+C 类似，可能留下部分文件）。"
+            )
+            try_write_competitor_report_if_merged_exists(
+                run_dir_path,
+                (job.keyword or "").strip(),
+                report_config=rc_cfg or None,
+            )
+        elif returncode == 0:
+            job.status = JobStatus.SUCCESS
+            job.run_dir = str(run_dir_path.resolve())
+            job.error_message = ""
+            job.cancellation_requested = False
+        else:
+            job.status = JobStatus.FAILED
+            job.run_dir = str(run_dir_path.resolve())
+            job.cancellation_requested = False
+            job.error_message = f"流水线子进程异常退出（exit {returncode}）。"
+    except Exception as e:
+        job.status = JobStatus.FAILED
+        job.cancellation_requested = False
+        job.error_message = f"{e}\n\n{traceback.format_exc()}"
+    job.updated_at = timezone.now()
+    job.save(
+        update_fields=[
+            "status",
+            "run_dir",
+            "error_message",
+            "cancellation_requested",
+            "updated_at",
+        ],
+    )
+    if job.status == JobStatus.SUCCESS:
+        try_ingest_job_full(PipelineJob.objects.get(pk=job_id))
+    if cookie_temp is not None and cookie_temp.is_file():
+        try:
+            cookie_temp.unlink()
+        except OSError:
+            pass
+    PipelineJob.objects.filter(pk=job_id).update(cookie_text="")

backend/pipeline/tests/test_brief_compact.py

@@ -0,0 +1,49 @@
+"""brief_compact：矩阵裁剪后仍保留 matrix_overview_for_llm。"""
+from __future__ import annotations
+
+from django.test import SimpleTestCase
+
+from pipeline.brief_compact import compact_brief_for_llm, matrix_overview_for_llm
+
+
+class BriefCompactTests(SimpleTestCase):
+    def test_matrix_overview_always_in_compact_output(self) -> None:
+        brief = {
+            "keyword": "测",
+            "matrix_by_group": [
+                {
+                    "group": "饼干",
+                    "sku_count": 2,
+                    "skus": [
+                        {"brand": "A牌", "sku_id": "1"},
+                        {"brand": "B牌", "sku_id": "2"},
+                    ],
+                }
+            ],
+        }
+        out = compact_brief_for_llm(brief, max_chars=120_000)
+        self.assertEqual(len(out["matrix_overview_for_llm"]), 1)
+        self.assertEqual(out["matrix_overview_for_llm"][0]["group"], "饼干")
+        self.assertIn("A牌", out["matrix_overview_for_llm"][0]["distinct_brands_sample"])
+
+    def test_overview_preserved_when_matrix_omitted(self) -> None:
+        brief = {
+            "matrix_by_group": [
+                {
+                    "group": f"G{i}",
+                    "sku_count": 2,
+                    "skus": [
+                        {"brand": "B", "sku_id": str(j)}
+                        for j in range(2)
+                    ],
+                }
+                for i in range(40)
+            ],
+            "consumer_feedback_by_matrix_group": [],
+        }
+        out = compact_brief_for_llm(brief, max_chars=200)
+        self.assertTrue(out.get("matrix_by_group_omitted"))
+        self.assertEqual(len(out["matrix_overview_for_llm"]), 40)
+
+    def test_matrix_overview_for_llm_empty_without_matrix(self) -> None:
+        self.assertEqual(matrix_overview_for_llm({}), [])

backend/pipeline/tests/test_brief_pack.py

@@ -0,0 +1,51 @@
+"""简报包 ZIP 与要点摘录 Markdown。"""
+from __future__ import annotations
+
+import json
+import zipfile
+from io import BytesIO
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+from django.test import SimpleTestCase
+
+from pipeline.brief_pack import (
+    build_brief_pack_zip_bytes,
+    markdown_summary_from_brief,
+)
+
+
+class BriefPackTests(SimpleTestCase):
+    def test_markdown_summary_contains_keyword_and_hints(self) -> None:
+        md = markdown_summary_from_brief(
+            {
+                "keyword": "测试词",
+                "batch_label": "batch1",
+                "scope": {"merged_sku_count": 3, "comment_flat_rows": 10},
+                "strategy_hints": ["提示一行"],
+            }
+        )
+        self.assertIn("测试词", md)
+        self.assertIn("提示一行", md)
+        self.assertIn("深入 SKU 数", md)
+
+    def test_zip_contains_expected_entries(self) -> None:
+        brief = {"keyword": "k", "schema_version": 1}
+        with TemporaryDirectory() as td:
+            p = Path(td) / "competitor_analysis.md"
+            p.write_text("# 报告\n", encoding="utf-8")
+            raw = build_brief_pack_zip_bytes(Path(td), brief)
+        buf = BytesIO(raw)
+        with zipfile.ZipFile(buf, "r") as zf:
+            names = set(zf.namelist())
+            data = json.loads(zf.read("02_结构化摘要.json").decode())
+        self.assertIn("01_竞品分析报告.md", names)
+        self.assertIn("02_结构化摘要.json", names)
+        self.assertIn("03_要点摘录.md", names)
+        self.assertIn("00_说明.txt", names)
+        self.assertEqual(data["keyword"], "k")
+
+    def test_zip_raises_without_report_file(self) -> None:
+        with TemporaryDirectory() as td:
+            with self.assertRaises(FileNotFoundError):
+                build_brief_pack_zip_bytes(Path(td), {})

backend/pipeline/tests/test_competitor_brief.py

@@ -0,0 +1,67 @@
+"""结构化竞品摘要：空样本烟测（不依赖真实 run_dir CSV）。"""
+from __future__ import annotations
+
+import sys
+import tempfile
+from pathlib import Path
+
+from django.conf import settings
+from django.test import SimpleTestCase
+
+
+class BuildCompetitorBriefTests(SimpleTestCase):
+    def test_empty_merged_json_safe(self) -> None:
+        root = Path(settings.CRAWLER_JD_ROOT).resolve()
+        if str(root) not in sys.path:
+            sys.path.insert(0, str(root))
+        import jd_competitor_report as jcr  # noqa: WPS433
+
+        with tempfile.TemporaryDirectory() as td:
+            run_dir = Path(td)
+            (run_dir / "pc_search_raw").mkdir(parents=True)
+
+            out = jcr.build_competitor_brief(
+                run_dir=run_dir,
+                keyword="测试",
+                merged_rows=[],
+                search_export_rows=[],
+                comment_rows=[],
+                meta=None,
+            )
+
+        self.assertEqual(out["schema_version"], 1)
+        self.assertEqual(out["scope"]["merged_sku_count"], 0)
+        self.assertIsInstance(out["strategy_hints"], list)
+        self.assertEqual(out["matrix_by_group"], [])
+        self.assertIn("comment_sentiment_lexicon", out)
+        self.assertEqual(out["comment_sentiment_lexicon"].get("text_units"), 0)
+        import json
+
+        json.dumps(out)
+
+    def test_custom_focus_words_in_report_config(self) -> None:
+        root = Path(settings.CRAWLER_JD_ROOT).resolve()
+        if str(root) not in sys.path:
+            sys.path.insert(0, str(root))
+        import jd_competitor_report as jcr  # noqa: WPS433
+
+        with tempfile.TemporaryDirectory() as td:
+            run_dir = Path(td)
+            (run_dir / "pc_search_raw").mkdir(parents=True)
+
+            out = jcr.build_competitor_brief(
+                run_dir=run_dir,
+                keyword="测试",
+                merged_rows=[],
+                search_export_rows=[],
+                comment_rows=[
+                    {
+                        "tagCommentContent": "自定义词阿尔法出现两次 自定义词阿尔法",
+                    }
+                ],
+                meta=None,
+                report_config={"comment_focus_words": ["自定义词阿尔法"]},
+            )
+
+        words = {x["word"] for x in out["comment_focus_keywords"]}
+        self.assertIn("自定义词阿尔法", words)

backend/pipeline/tests/test_strategy_draft.py

@@ -0,0 +1,56 @@
+"""市场策略草稿 Markdown（规则，无 LLM）。"""
+from __future__ import annotations
+
+from django.test import SimpleTestCase
+
+from pipeline.strategy_draft import build_strategy_draft_markdown
+
+
+class StrategyDraftTests(SimpleTestCase):
+    def test_build_contains_sections_and_notes(self) -> None:
+        brief = {
+            "schema_version": 1,
+            "keyword": "测试K",
+            "batch_label": "b1",
+            "scope": {"merged_sku_count": 2, "comment_flat_rows": 5},
+            "strategy_hints": ["假设A"],
+        }
+        md = build_strategy_draft_markdown(
+            job_id=99,
+            keyword="测试K",
+            brief=brief,
+            business_notes="重点：华东",
+            generated_at_iso="2026-04-09T12:00:00",
+        )
+        self.assertIn("测试K", md)
+        self.assertIn("任务 ID**：99", md)
+        self.assertIn("假设A", md)
+        self.assertIn("重点：华东", md)
+        self.assertIn("战略背景与目标", md)
+        self.assertIn("市场策略制定草稿", md)
+
+    def test_strategy_decisions_merge(self) -> None:
+        brief = {"schema_version": 1, "keyword": "K", "batch_label": "b"}
+        decisions = {
+            "product_role": "追赶型",
+            "positioning_choice": "mid",
+            "competitive_stance": "flank",
+            "pillar_product": "做低糖配方",
+            "ack_risk_keywords": True,
+            "ack_risk_price": False,
+            "ack_risk_concentration": True,
+        }
+        md = build_strategy_draft_markdown(
+            job_id=1,
+            keyword="K",
+            brief=brief,
+            strategy_decisions=decisions,
+        )
+        self.assertIn("**本品角色**：追赶型", md)
+        self.assertIn("- [x] **卡腰**", md)
+        self.assertIn("- [ ] **贴顶**", md)
+        self.assertIn("侧翼切入", md)
+        self.assertIn("| 产品 | 做低糖配方 |", md)
+        self.assertIn("- [x] 关注词/场景是否**以偏概全**", md)
+        self.assertIn("- [ ] 价格带是否含大促", md)
+        self.assertIn("- [x] 列表集中度与深入样本品牌是否**矛盾**", md)

backend/pipeline/urls.py

@@ -0,0 +1,101 @@
+from django.urls import path
+
+from . import views
+
+urlpatterns = [
+    path(
+        "report-config-defaults/",
+        views.ReportConfigDefaultsView.as_view(),
+        name="report-config-defaults",
+    ),
+    path("jobs/", views.JobListCreateView.as_view(), name="job-list-create"),
+    path("jobs/<int:pk>/", views.JobDetailView.as_view(), name="job-detail"),
+    path(
+        "jobs/<int:pk>/cancel/",
+        views.JobCancelView.as_view(),
+        name="job-cancel",
+    ),
+    path("jobs/<int:pk>/download/", views.JobDownloadView.as_view(), name="job-download"),
+    path("jobs/<int:pk>/preview/", views.JobPreviewView.as_view(), name="job-preview"),
+    path(
+        "jobs/<int:pk>/dataset/summary/",
+        views.JobDatasetSummaryView.as_view(),
+        name="job-dataset-summary",
+    ),
+    path(
+        "jobs/<int:pk>/dataset/search/",
+        views.JobDatasetSearchView.as_view(),
+        name="job-dataset-search",
+    ),
+    path(
+        "jobs/<int:pk>/dataset/detail/",
+        views.JobDatasetDetailView.as_view(),
+        name="job-dataset-detail",
+    ),
+    path(
+        "jobs/<int:pk>/dataset/comments/",
+        views.JobDatasetCommentsView.as_view(),
+        name="job-dataset-comments",
+    ),
+    path(
+        "jobs/<int:pk>/dataset/merged/",
+        views.JobDatasetMergedView.as_view(),
+        name="job-dataset-merged",
+    ),
+    path(
+        "jobs/<int:pk>/export/",
+        views.JobDatasetExportView.as_view(),
+        name="job-dataset-export",
+    ),
+    path(
+        "jobs/<int:pk>/regenerate-report/",
+        views.JobRegenerateReportView.as_view(),
+        name="job-regenerate-report",
+    ),
+    path(
+        "jobs/<int:pk>/competitor-brief/",
+        views.JobCompetitorBriefView.as_view(),
+        name="job-competitor-brief",
+    ),
+    path(
+        "jobs/<int:pk>/competitor-brief-pack/",
+        views.JobCompetitorBriefPackView.as_view(),
+        name="job-competitor-brief-pack",
+    ),
+    path(
+        "jobs/<int:pk>/strategy-draft/",
+        views.JobStrategyDraftView.as_view(),
+        name="job-strategy-draft",
+    ),
+    path(
+        "jobs/<int:pk>/export-document/",
+        views.JobExportDocumentView.as_view(),
+        name="job-export-document",
+    ),
+    path(
+        "jobs/<int:pk>/report-asset/",
+        views.JobReportAssetView.as_view(),
+        name="job-report-asset",
+    ),
+    path(
+        "jobs/<int:pk>/ingest-merged/",
+        views.JobImportMergedView.as_view(),
+        name="job-ingest-merged",
+    ),
+    path("jd/products/", views.JdProductListView.as_view(), name="jd-product-list"),
+    path(
+        "jd/products/<str:sku_id>/",
+        views.JdProductDetailView.as_view(),
+        name="jd-product-detail",
+    ),
+    path(
+        "jd/products/<str:sku_id>/snapshots/",
+        views.JdProductSnapshotListView.as_view(),
+        name="jd-product-snapshots",
+    ),
+    path(
+        "jd/snapshots/<int:pk>/",
+        views.JdProductSnapshotDetailView.as_view(),
+        name="jd-snapshot-detail",
+    ),
+]

backend/pipeline/views.py

@@ -0,0 +1,858 @@
+from __future__ import annotations
+
+import mimetypes
+import threading
+from pathlib import Path
+from typing import Any
+
+import requests
+from django.conf import settings
+from django.db.models import Count, Q
+from django.http import FileResponse, Http404, HttpResponse
+from django.utils import timezone
+from django.utils.decorators import method_decorator
+from django.views.decorators.csrf import csrf_exempt
+from rest_framework import status
+from rest_framework.response import Response
+from rest_framework.views import APIView
+
+from .dataset_nonempty import (
+    comment_columns_for_api,
+    detail_columns_for_api,
+    merged_columns_for_api,
+    search_columns_for_api,
+)
+from .export_job import build_csv_bytes, build_json_bytes, build_xlsx_bytes
+from .row_serialize import (
+    comment_row_to_dict,
+    detail_row_to_dict,
+    merged_row_to_dict,
+    search_row_to_dict,
+)
+from .brief_pack import build_brief_pack_zip_bytes
+from .strategy_draft import build_strategy_draft_markdown
+from .ingest import ingest_job_full
+from .jd_runner import (
+    build_competitor_brief_for_job,
+    get_default_report_config,
+    merge_llm_supplement_with_rules_report,
+    regenerate_competitor_report,
+    write_competitor_analysis_markdown,
+)
+from .llm_generate import (
+    generate_competitor_report_markdown_llm,
+    generate_strategy_draft_markdown_llm,
+)
+from .md_document_export import markdown_to_docx_bytes, markdown_to_pdf_bytes
+from .models import (
+    JdJobCommentRow,
+    JdJobDetailRow,
+    JdJobMergedRow,
+    JdJobSearchRow,
+    JdProduct,
+    JdProductSnapshot,
+    JobStatus,
+    PipelineJob,
+)
+from .serializers import (
+    CreatePipelineJobSerializer,
+    JdProductDetailSerializer,
+    JdProductListSerializer,
+    JdProductSnapshotBriefSerializer,
+    JdProductSnapshotDetailSerializer,
+    JobReportConfigPatchSerializer,
+    PipelineJobSerializer,
+    RegenerateReportRequestSerializer,
+    StrategyDraftRequestSerializer,
+)
+from .tasks import execute_job
+
+# 在线预览最大字节（超出则截断并提示下载）
+_PREVIEW_MAX_BYTES = 2 * 1024 * 1024
+
+# 允许下载的相对文件名（均在 run_dir 下）
+_DOWNLOAD_NAMES = frozenset(
+    {
+        "merged",
+        "pc_search",
+        "comments",
+        "detail_ware",
+        "report",
+    }
+)
+
+
+def _jd_data_root() -> Path:
+    root = (settings.LOW_GI_PROJECT_ROOT or "").strip()
+    if not root:
+        raise RuntimeError("LOW_GI_PROJECT_ROOT 未配置")
+    return (Path(root) / "data" / "JD").resolve()
+
+
+def _safe_file_for_job(run_dir_str: str, name: str) -> Path:
+    if name not in _DOWNLOAD_NAMES:
+        raise Http404("unknown file")
+    base = Path(run_dir_str).resolve()
+    jd_root = _jd_data_root().resolve()
+    try:
+        base.relative_to(jd_root)
+    except ValueError:
+        raise Http404("invalid run_dir")
+
+    mapping = {
+        "merged": "keyword_pipeline_merged.csv",
+        "pc_search": "pc_search_export.csv",
+        "comments": "comments_flat.csv",
+        "detail_ware": "detail_ware_export.csv",
+        "report": "competitor_analysis.md",
+    }
+    f = base / mapping[name]
+    if not f.is_file():
+        raise Http404("file not found")
+    return f
+
+
+def _job_run_dir_usable(job: PipelineJob) -> bool:
+    """成功或已终止但已写入 run_dir 时，可预览/下载批次文件。"""
+    return bool((job.run_dir or "").strip()) and job.status in (
+        JobStatus.SUCCESS,
+        JobStatus.CANCELLED,
+    )
+
+
+@method_decorator(csrf_exempt, name="dispatch")
+class JobListCreateView(APIView):
+    def get(self, request):
+        qs = PipelineJob.objects.all()[:200]
+        return Response(PipelineJobSerializer(qs, many=True).data)
+
+    def post(self, request):
+        if not (settings.LOW_GI_PROJECT_ROOT or "").strip():
+            return Response(
+                {"detail": "请先在 market_assistant/.env 中配置 LOW_GI_PROJECT_ROOT"},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+        ser = CreatePipelineJobSerializer(data=request.data)
+        ser.is_valid(raise_exception=True)
+        data = ser.validated_data
+        job = PipelineJob.objects.create(
+            platform=data["platform"],
+            keyword=data["keyword"],
+            max_skus=data.get("max_skus"),
+            page_start=data.get("page_start"),
+            page_to=data.get("page_to"),
+            pipeline_run_dir=data.get("pipeline_run_dir") or "",
+            cookie_file_path=data.get("cookie_file_path") or "",
+            cookie_text=data.get("cookie_text") or "",
+            pvid=data.get("pvid") or "",
+            request_delay=data.get("request_delay") or "",
+            list_pages=data.get("list_pages") or "",
+            scenario_filter_enabled=data.get("scenario_filter_enabled"),
+            report_config=data.get("report_config") or {},
+            status=JobStatus.PENDING,
+        )
+        t = threading.Thread(target=execute_job, args=(job.id,), daemon=True)
+        t.start()
+        return Response(
+            PipelineJobSerializer(job).data,
+            status=status.HTTP_201_CREATED,
+        )
+
+
+@method_decorator(csrf_exempt, name="dispatch")
+class JobDetailView(APIView):
+    def get(self, request, pk: int):
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        return Response(PipelineJobSerializer(job).data)
+
+    def patch(self, request, pk: int):
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        ser = JobReportConfigPatchSerializer(data=request.data)
+        ser.is_valid(raise_exception=True)
+        job.report_config = ser.validated_data["report_config"]
+        job.save(update_fields=["report_config", "updated_at"])
+        return Response(PipelineJobSerializer(job).data)
+
+
+@method_decorator(csrf_exempt, name="dispatch")
+class JobCancelView(APIView):
+    """
+    终止：将 ``cancellation_requested`` 置位后，执行线程会尽快 ``terminate`` 采集子进程
+    （效果接近在终端对脚本按 Ctrl+C），并保留已写入运行目录的文件。
+    """
+
+    def post(self, request, pk: int):
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if job.status not in (JobStatus.PENDING, JobStatus.RUNNING):
+            return Response(
+                {"detail": "仅待执行或执行中的任务可终止"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        job.cancellation_requested = True
+        job.save(update_fields=["cancellation_requested", "updated_at"])
+        return Response(PipelineJobSerializer(job).data)
+
+
+class ReportConfigDefaultsView(APIView):
+    """返回 ``jd_competitor_report`` 中与脚本常量一致的默认报告调参 JSON。"""
+
+    def get(self, request):
+        try:
+            return Response(get_default_report_config())
+        except FileNotFoundError as e:
+            return Response(
+                {"detail": str(e)},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+
+
+class JobDownloadView(APIView):
+    def get(self, request, pk: int):
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job or not _job_run_dir_usable(job):
+            raise Http404()
+        name = (request.query_params.get("name") or "").strip().lower()
+        path = _safe_file_for_job(job.run_dir, name)
+        return FileResponse(
+            path.open("rb"),
+            as_attachment=True,
+            filename=path.name,
+        )
+
+
+class JobPreviewView(APIView):
+    """浏览器内联查看产出（CSV / Markdown 文本），大文件截断。"""
+
+    def get(self, request, pk: int):
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job or not _job_run_dir_usable(job):
+            raise Http404()
+        name = (request.query_params.get("name") or "").strip().lower()
+        fpath = _safe_file_for_job(job.run_dir, name)
+        raw = fpath.read_bytes()
+        truncated = len(raw) > _PREVIEW_MAX_BYTES
+        if truncated:
+            raw = raw[:_PREVIEW_MAX_BYTES]
+        text = raw.decode("utf-8-sig", errors="replace")
+        if truncated:
+            text += "\n\n... [内容已截断，完整文件请使用下载]\n"
+
+        if name == "report":
+            ctype = "text/markdown; charset=utf-8"
+        else:
+            ctype = "text/csv; charset=utf-8"
+        resp = HttpResponse(text, content_type=ctype)
+        resp["X-Preview-Truncated"] = "1" if truncated else "0"
+        resp["X-Preview-Filename"] = fpath.name
+        return resp
+
+
+@method_decorator(csrf_exempt, name="dispatch")
+class JobRegenerateReportView(APIView):
+    """基于任务已有 ``run_dir`` 内 CSV 重新生成 ``competitor_analysis.md``（不重新爬取）。"""
+
+    def post(self, request, pk: int):
+        if not (settings.LOW_GI_PROJECT_ROOT or "").strip():
+            return Response(
+                {"detail": "请先在 market_assistant/.env 中配置 LOW_GI_PROJECT_ROOT"},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if job.status != JobStatus.SUCCESS or not (job.run_dir or "").strip():
+            return Response(
+                {"detail": "仅可对已成功且已写入 run_dir 的任务重新生成报告"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        ser = RegenerateReportRequestSerializer(data=request.data or {})
+        ser.is_valid(raise_exception=True)
+        generator = ser.validated_data.get("generator") or "rules"
+        rc = job.report_config if isinstance(job.report_config, dict) else None
+        if generator == "llm":
+            try:
+                regenerate_competitor_report(
+                    job.run_dir, job.keyword, report_config=rc
+                )
+                rules_md = (
+                    Path(job.run_dir) / "competitor_analysis.md"
+                ).read_text(encoding="utf-8")
+                brief = build_competitor_brief_for_job(
+                    job.run_dir, job.keyword, report_config=rc
+                )
+                md = generate_competitor_report_markdown_llm(brief, job.keyword)
+                md = merge_llm_supplement_with_rules_report(md, rules_md)
+                write_competitor_analysis_markdown(job.run_dir, md)
+            except FileNotFoundError as e:
+                return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+            except ValueError as e:
+                return Response({"detail": str(e)}, status=status.HTTP_503_SERVICE_UNAVAILABLE)
+            except requests.RequestException as e:
+                return Response(
+                    {"detail": f"大模型网关错误：{e}"},
+                    status=status.HTTP_502_BAD_GATEWAY,
+                )
+        else:
+            try:
+                regenerate_competitor_report(job.run_dir, job.keyword, report_config=rc)
+            except FileNotFoundError as e:
+                return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+            except ValueError as e:
+                return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+        return Response(PipelineJobSerializer(job).data)
+
+
+class JobCompetitorBriefView(APIView):
+    """单次任务的结构化竞品摘要（JSON，与 ``competitor_analysis.md`` 统计口径一致，规则驱动无 LLM）。"""
+
+    def get(self, request, pk: int):
+        if not (settings.LOW_GI_PROJECT_ROOT or "").strip():
+            return Response(
+                {"detail": "请先在 market_assistant/.env 中配置 LOW_GI_PROJECT_ROOT"},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if job.status != JobStatus.SUCCESS or not (job.run_dir or "").strip():
+            return Response(
+                {"detail": "仅可对已成功且含 run_dir 的任务获取竞品摘要"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        try:
+            data = build_competitor_brief_for_job(
+                job.run_dir,
+                job.keyword,
+                report_config=job.report_config
+                if isinstance(job.report_config, dict)
+                else None,
+            )
+        except FileNotFoundError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+        except ValueError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+        return Response(data)
+
+
+class JobCompetitorBriefPackView(APIView):
+    """ZIP：完整 Markdown 报告 + 结构化 JSON + 要点摘录 Markdown + 说明文本。"""
+
+    def get(self, request, pk: int):
+        if not (settings.LOW_GI_PROJECT_ROOT or "").strip():
+            return Response(
+                {"detail": "请先在 market_assistant/.env 中配置 LOW_GI_PROJECT_ROOT"},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if job.status != JobStatus.SUCCESS or not (job.run_dir or "").strip():
+            return Response(
+                {"detail": "仅可对已成功且含 run_dir 的任务导出简报包"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        try:
+            brief = build_competitor_brief_for_job(
+                job.run_dir,
+                job.keyword,
+                report_config=job.report_config
+                if isinstance(job.report_config, dict)
+                else None,
+            )
+            zip_bytes = build_brief_pack_zip_bytes(Path(job.run_dir), brief)
+        except FileNotFoundError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+        except ValueError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+
+        filename_ascii = f"job_{pk}_competitor_brief_pack.zip"
+        resp = HttpResponse(zip_bytes, content_type="application/zip")
+        resp["Content-Disposition"] = f'attachment; filename="{filename_ascii}"'
+        return resp
+
+
+@method_decorator(csrf_exempt, name="dispatch")
+class JobStrategyDraftView(APIView):
+    """
+    市场策略制定 Markdown：策略框架 + 附录；默认规则生成，可选 ``generator=llm``（``AI_crawler.chat_completion_text``）。
+    """
+
+    def post(self, request, pk: int):
+        if not (settings.LOW_GI_PROJECT_ROOT or "").strip():
+            return Response(
+                {"detail": "请先在 market_assistant/.env 中配置 LOW_GI_PROJECT_ROOT"},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if job.status != JobStatus.SUCCESS or not (job.run_dir or "").strip():
+            return Response(
+                {"detail": "仅可对已成功且含 run_dir 的任务生成策略制定稿"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        ser = StrategyDraftRequestSerializer(data=request.data or {})
+        ser.is_valid(raise_exception=True)
+        vd = ser.validated_data
+        notes = (vd.get("business_notes") or "").strip()
+        strategy_decisions = {
+            "product_role": vd.get("product_role") or "",
+            "time_horizon": vd.get("time_horizon") or "",
+            "success_criteria": vd.get("success_criteria") or "",
+            "non_goals": vd.get("non_goals") or "",
+            "battlefield_one_line": vd.get("battlefield_one_line") or "",
+            "positioning_choice": vd.get("positioning_choice") or "",
+            "competitive_stance": vd.get("competitive_stance") or "",
+            "pillar_product": vd.get("pillar_product") or "",
+            "pillar_price": vd.get("pillar_price") or "",
+            "pillar_channel": vd.get("pillar_channel") or "",
+            "pillar_comm": vd.get("pillar_comm") or "",
+            "ack_risk_keywords": bool(vd.get("ack_risk_keywords")),
+            "ack_risk_price": bool(vd.get("ack_risk_price")),
+            "ack_risk_concentration": bool(vd.get("ack_risk_concentration")),
+        }
+        try:
+            brief = build_competitor_brief_for_job(
+                job.run_dir,
+                job.keyword,
+                report_config=job.report_config
+                if isinstance(job.report_config, dict)
+                else None,
+            )
+        except FileNotFoundError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+        except ValueError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+
+        gen_at = timezone.now().isoformat()
+        generator = (vd.get("generator") or "rules").strip()
+        try:
+            if generator == "llm":
+                md = generate_strategy_draft_markdown_llm(
+                    job_id=job.id,
+                    keyword=job.keyword,
+                    brief=brief,
+                    business_notes=notes,
+                    generated_at_iso=gen_at,
+                    strategy_decisions=strategy_decisions,
+                )
+                src = "llm_text_ai_crawler_v1"
+            else:
+                md = build_strategy_draft_markdown(
+                    job_id=job.id,
+                    keyword=job.keyword,
+                    brief=brief,
+                    business_notes=notes,
+                    generated_at_iso=gen_at,
+                    strategy_decisions=strategy_decisions,
+                )
+                src = "structured_summary_rules_v1"
+        except ValueError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_503_SERVICE_UNAVAILABLE)
+        except requests.RequestException as e:
+            return Response(
+                {"detail": f"大模型网关错误：{e}"},
+                status=status.HTTP_502_BAD_GATEWAY,
+            )
+        return Response(
+            {
+                "schema_version": 1,
+                "job_id": job.id,
+                "keyword": job.keyword,
+                "generated_at": gen_at,
+                "source": src,
+                "markdown": md,
+            }
+        )
+
+
+@method_decorator(csrf_exempt, name="dispatch")
+class JobExportDocumentView(APIView):
+    """
+    将 Markdown 导出为 Word（.docx）或简易 PDF。
+    - GET：``kind=report``，读取 ``run_dir/competitor_analysis.md``。
+    - POST：``kind=strategy``，请求体 JSON 字段 ``markdown`` 为策略稿正文（与前端 sessionStorage 一致）。
+    PDF 依赖本机中文字体或环境变量 ``MA_PDF_FONT`` 指向 .ttf。
+    """
+
+    def get(self, request, pk: int):
+        if not (settings.LOW_GI_PROJECT_ROOT or "").strip():
+            return Response(
+                {"detail": "请先在 market_assistant/.env 中配置 LOW_GI_PROJECT_ROOT"},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if not _job_run_dir_usable(job):
+            return Response(
+                {"detail": "仅可对已成功或已终止且含 run_dir 的任务导出"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        fmt = (request.query_params.get("fmt") or "docx").strip().lower()
+        kind = (request.query_params.get("kind") or "report").strip().lower()
+        if kind != "report":
+            return Response(
+                {"detail": "GET 仅支持 kind=report；策略稿请用 POST 提交 markdown"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        if fmt not in ("docx", "pdf"):
+            return Response(
+                {"detail": "fmt 须为 docx 或 pdf"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        path = Path(job.run_dir) / "competitor_analysis.md"
+        if not path.is_file():
+            return Response(
+                {"detail": "报告文件不存在，请先在「报告生成」重新生成"},
+                status=status.HTTP_404_NOT_FOUND,
+            )
+        md = path.read_text(encoding="utf-8")
+        asset_root = Path(job.run_dir).resolve()
+        try:
+            if fmt == "docx":
+                data = markdown_to_docx_bytes(md, asset_root=asset_root)
+                ct = "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
+                fn = f"job_{pk}_competitor_report.docx"
+            else:
+                data = markdown_to_pdf_bytes(md, asset_root=asset_root)
+                ct = "application/pdf"
+                fn = f"job_{pk}_competitor_report.pdf"
+        except ValueError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_503_SERVICE_UNAVAILABLE)
+        resp = HttpResponse(data, content_type=ct)
+        resp["Content-Disposition"] = f'attachment; filename="{fn}"'
+        return resp
+
+    def post(self, request, pk: int):
+        if not (settings.LOW_GI_PROJECT_ROOT or "").strip():
+            return Response(
+                {"detail": "请先在 market_assistant/.env 中配置 LOW_GI_PROJECT_ROOT"},
+                status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            )
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if not _job_run_dir_usable(job):
+            return Response(
+                {"detail": "仅可对已成功或已终止且含 run_dir 的任务导出"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        body = request.data if isinstance(request.data, dict) else {}
+        kind = (body.get("kind") or "strategy").strip().lower()
+        fmt = (body.get("fmt") or "docx").strip().lower()
+        md = (body.get("markdown") or "").strip()
+        if kind != "strategy":
+            return Response(
+                {"detail": "POST 仅支持 kind=strategy"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        if not md:
+            return Response(
+                {"detail": "markdown 不能为空"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        if fmt not in ("docx", "pdf"):
+            return Response(
+                {"detail": "fmt 须为 docx 或 pdf"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        try:
+            if fmt == "docx":
+                data = markdown_to_docx_bytes(md)
+                ct = "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
+                fn = f"job_{pk}_strategy_draft.docx"
+            else:
+                data = markdown_to_pdf_bytes(md)
+                ct = "application/pdf"
+                fn = f"job_{pk}_strategy_draft.pdf"
+        except ValueError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_503_SERVICE_UNAVAILABLE)
+        resp = HttpResponse(data, content_type=ct)
+        resp["Content-Disposition"] = f'attachment; filename="{fn}"'
+        return resp
+
+
+class JobReportAssetView(APIView):
+    """安全读取 ``run_dir/report_assets/*`` 下的 PNG 等（供 Markdown 预览插图）。"""
+
+    def get(self, request, pk: int):
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job or not _job_run_dir_usable(job):
+            raise Http404()
+        rel = (request.query_params.get("path") or "").strip().replace("\\", "/")
+        if not rel or ".." in Path(rel).parts:
+            return Response(
+                {"detail": "path 非法"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        base = Path(job.run_dir).resolve()
+        assets_root = (base / "report_assets").resolve()
+        target = (base / rel).resolve()
+        try:
+            target.relative_to(assets_root)
+        except ValueError:
+            raise Http404()
+        if not target.is_file():
+            raise Http404()
+        ctype, _ = mimetypes.guess_type(str(target))
+        return FileResponse(
+            target.open("rb"),
+            content_type=ctype or "application/octet-stream",
+        )
+
+
+def _dataset_job(pk: int) -> PipelineJob:
+    job = PipelineJob.objects.filter(pk=pk).first()
+    if not job:
+        raise Http404()
+    return job
+
+
+def _read_page_params(request) -> tuple[int, int]:
+    page_size = min(max(int(request.query_params.get("page_size", 50)), 1), 200)
+    page = max(int(request.query_params.get("page", 1)), 1)
+    return page, page_size
+
+
+class JobDatasetSummaryView(APIView):
+    """任务在库中的搜索/详情/评价行数（入库后可用）。"""
+
+    def get(self, request, pk: int):
+        job = _dataset_job(pk)
+        return Response(
+            {
+                "job_id": job.id,
+                "keyword": job.keyword,
+                "status": job.status,
+                "search_rows": JdJobSearchRow.objects.filter(job=job).count(),
+                "detail_rows": JdJobDetailRow.objects.filter(job=job).count(),
+                "comment_rows": JdJobCommentRow.objects.filter(job=job).count(),
+                "merged_rows": JdJobMergedRow.objects.filter(job=job).count(),
+                "search_columns": search_columns_for_api(job),
+                "detail_columns": detail_columns_for_api(job),
+                "comment_columns": comment_columns_for_api(job),
+                "merged_columns": merged_columns_for_api(job),
+            }
+        )
+
+
+class JobDatasetSearchView(APIView):
+    def get(self, request, pk: int):
+        job = _dataset_job(pk)
+        page, page_size = _read_page_params(request)
+        qs = JdJobSearchRow.objects.filter(job=job)
+        total = qs.count()
+        start = (page - 1) * page_size
+        rows = qs.order_by("row_index")[start : start + page_size]
+        return Response(
+            {
+                "total": total,
+                "page": page,
+                "page_size": page_size,
+                "results": [search_row_to_dict(r) for r in rows],
+            }
+        )
+
+
+class JobDatasetDetailView(APIView):
+    def get(self, request, pk: int):
+        job = _dataset_job(pk)
+        page, page_size = _read_page_params(request)
+        qs = JdJobDetailRow.objects.filter(job=job)
+        total = qs.count()
+        start = (page - 1) * page_size
+        rows = qs.order_by("row_index")[start : start + page_size]
+        return Response(
+            {
+                "total": total,
+                "page": page,
+                "page_size": page_size,
+                "results": [detail_row_to_dict(r) for r in rows],
+            }
+        )
+
+
+class JobDatasetCommentsView(APIView):
+    def get(self, request, pk: int):
+        job = _dataset_job(pk)
+        page, page_size = _read_page_params(request)
+        sku_id = (request.query_params.get("sku_id") or "").strip()
+        qs = JdJobCommentRow.objects.filter(job=job)
+        if sku_id:
+            qs = qs.filter(sku_id=sku_id)
+        total = qs.count()
+        start = (page - 1) * page_size
+        rows = qs.order_by("row_index")[start : start + page_size]
+        return Response(
+            {
+                "total": total,
+                "page": page,
+                "page_size": page_size,
+                "sku_filter": sku_id or None,
+                "results": [comment_row_to_dict(r) for r in rows],
+            }
+        )
+
+
+class JobDatasetMergedView(APIView):
+    def get(self, request, pk: int):
+        job = _dataset_job(pk)
+        page, page_size = _read_page_params(request)
+        qs = JdJobMergedRow.objects.filter(job=job)
+        total = qs.count()
+        start = (page - 1) * page_size
+        rows = qs.order_by("row_index")[start : start + page_size]
+        return Response(
+            {
+                "total": total,
+                "page": page,
+                "page_size": page_size,
+                "results": [merged_row_to_dict(r) for r in rows],
+            }
+        )
+
+
+class JobDatasetExportView(APIView):
+    """下载：kind=search|detail|comments|merged|all，export_fmt=json|csv|xlsx。
+
+    ``merged``：库内合并宽表行（与 lean 合并 CSV 列一致，入库后导出）。
+    注意：勿使用查询参数名 ``format``，DRF 会将其用于内容协商，非 json 时易在进视图前 404。
+    """
+
+    def get(self, request, pk: int):
+        job = _dataset_job(pk)
+        kind = (request.query_params.get("kind") or "search").strip().lower()
+        fmt = (request.query_params.get("export_fmt") or "json").strip().lower()
+        if kind not in ("search", "detail", "comments", "all", "merged"):
+            return Response(
+                {"detail": "kind 须为 search / detail / comments / all / merged"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        if fmt not in ("json", "csv", "xlsx"):
+            return Response(
+                {"detail": "export_fmt 须为 json / csv / xlsx"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        try:
+            if fmt == "json":
+                data, filename = build_json_bytes(job=job, kind=kind)
+                resp = HttpResponse(data, content_type="application/json; charset=utf-8")
+            elif fmt == "csv":
+                data, filename = build_csv_bytes(job=job, kind=kind)
+                resp = HttpResponse(data, content_type="text/csv; charset=utf-8")
+            else:
+                data, filename = build_xlsx_bytes(job=job, kind=kind)
+                resp = HttpResponse(
+                    data,
+                    content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+                )
+        except ValueError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+        resp["Content-Disposition"] = f'attachment; filename="{filename}"'
+        return resp
+
+
+class JdProductListView(APIView):
+    """已入库 SKU 分页列表；支持按标题/SKU/品牌模糊搜、按合并表中的 pipeline_keyword 精确筛。"""
+
+    def get(self, request):
+        limit = min(max(int(request.query_params.get("limit", 50)), 1), 200)
+        offset = max(int(request.query_params.get("offset", 0)), 0)
+        q = (request.query_params.get("q") or "").strip()
+        kw = (request.query_params.get("keyword") or "").strip()
+        qs = JdProduct.objects.annotate(snapshot_count=Count("snapshots"))
+        if q:
+            qs = qs.filter(
+                Q(sku_id__icontains=q)
+                | Q(title__icontains=q)
+                | Q(detail_brand__icontains=q)
+            )
+        if kw:
+            qs = qs.filter(current_payload__pipeline_keyword=kw)
+        total = qs.count()
+        page = qs.order_by("-updated_at")[offset : offset + limit]
+        return Response(
+            {
+                "total": total,
+                "limit": limit,
+                "offset": offset,
+                "results": JdProductListSerializer(page, many=True).data,
+            }
+        )
+
+
+class JdProductDetailView(APIView):
+    def get(self, request, sku_id: str):
+        platform = (request.query_params.get("platform") or "jd").strip() or "jd"
+        obj = (
+            JdProduct.objects.annotate(snapshot_count=Count("snapshots"))
+            .filter(platform=platform, sku_id=sku_id)
+            .first()
+        )
+        if not obj:
+            raise Http404()
+        return Response(JdProductDetailSerializer(obj).data)
+
+
+class JdProductSnapshotListView(APIView):
+    """某 SKU 的历史快照列表（不含整包 payload，便于时间线）。"""
+
+    def get(self, request, sku_id: str):
+        platform = (request.query_params.get("platform") or "jd").strip() or "jd"
+        product = JdProduct.objects.filter(platform=platform, sku_id=sku_id).first()
+        if not product:
+            raise Http404()
+        snaps = (
+            product.snapshots.select_related("job")
+            .order_by("-captured_at")
+            .all()
+        )
+        return Response(
+            {
+                "platform": platform,
+                "sku_id": sku_id,
+                "count": snaps.count(),
+                "results": JdProductSnapshotBriefSerializer(snaps, many=True).data,
+            }
+        )
+
+
+class JdProductSnapshotDetailView(APIView):
+    """单条快照完整 payload，用于历史回放与字段级对比。"""
+
+    def get(self, request, pk: int):
+        snap = (
+            JdProductSnapshot.objects.select_related("product", "job")
+            .filter(pk=pk)
+            .first()
+        )
+        if not snap:
+            raise Http404()
+        return Response(JdProductSnapshotDetailSerializer(snap).data)
+
+
+@method_decorator(csrf_exempt, name="dispatch")
+class JobImportMergedView(APIView):
+    """将指定任务目录下搜索/详情/评价 CSV 与合并表重新写入数据库（幂等：先清空该任务三类行再全量插入）。"""
+
+    def post(self, request, pk: int):
+        job = PipelineJob.objects.filter(pk=pk).first()
+        if not job:
+            raise Http404()
+        if job.status != JobStatus.SUCCESS or not (job.run_dir or "").strip():
+            return Response(
+                {"detail": "仅可对已成功且含 run_dir 的任务执行入库"},
+                status=status.HTTP_400_BAD_REQUEST,
+            )
+        try:
+            stats = ingest_job_full(job)
+        except FileNotFoundError as e:
+            return Response({"detail": str(e)}, status=status.HTTP_400_BAD_REQUEST)
+        return Response(stats)

backend/requirements.txt

@@ -0,0 +1,9 @@
+django>=5.0,<6
+djangorestframework>=3.15
+django-cors-headers>=4.3
+python-dotenv>=1.0
+openpyxl>=3.1
+requests>=2.31
+python-docx>=1.1
+reportlab>=4.0
+matplotlib>=3.8

docs/DEPLOY_AND_GIT.md

@@ -0,0 +1,101 @@
+# 部署与 Git 仓库整理
+
+## 1. 环境变量：仅一份 `.env`
+
+| 文件 | 说明 |
+|------|------|
+| `market_assistant/.env` | 本地与服务器上的**唯一**配置（密钥、路径、Django、LLM） |
+| `market_assistant/.env.example` | 模板，可提交仓库 |
+
+已移除「仓库根目录 `.env`」与「`backend/.env`」第二加载源；Django 与 `crawler_copy/jd_pc_search/AI_crawler.py` 均从 `market_assistant/.env` 读取（`AI_crawler` 在导入时先加载该文件再解析 `LOW_GI_PROJECT_ROOT`）。
+
+部署到新机器：
+
+1. 复制 `market_assistant/.env.example` → `market_assistant/.env`
+2. **可选**：设置 `LOW_GI_PROJECT_ROOT` 为单独数据盘的绝对路径；不设置时默认为**本仓库根目录**，数据写在 `./data/JD/`（启动 Django 时会自动创建该目录）
+3. 设置 `DJANGO_SECRET_KEY`、`DJANGO_DEBUG=False`、生产域名下的 `DJANGO_ALLOWED_HOSTS` / `CORS_*` / `CSRF_*`
+4. 若使用 LLM，填写 `OPENAI_*` 或 `LLM_*`
+
+## 2. 功能是否只需本目录？
+
+**是。** `market_assistant` 内含：
+
+- Django API、流水线任务、入库与导出  
+- 前端 Vue 工作台  
+- 京东采集脚本副本 `backend/crawler_copy/jd_pc_search`（含 Node/Playwright 子进程调用）
+
+**不要求**仓库外仍存在旧的 `crawler/jd_pc_search`。  
+
+**运行时另需**（部分不在 Git 中）：
+
+- 默认可写目录为仓库根下 `data/JD/`（`.gitignore` 已忽略）；若配置了 `LOW_GI_PROJECT_ROOT` 则数据在该路径下  
+- 按任务配置放置 Cookie（路径须在有效数据根之下，如 `common/jd_cookie.txt`）  
+- 本机已装 Node、流水线所需的 Playwright 等（与现有一致）
+
+## 3. 远程 Git 只维护 `market_assistant`（推荐两种做法）
+
+> **先备份仓库**，再在副本上操作；改写历史后需与团队约定 **`git push --force`**。
+
+### 方案 A：保留历史，把子目录提成仓库根（git filter-repo）
+
+适用于「当前仓库在上一级 `Low GI/`，只想提交 `market_assistant/` 里的内容且路径变为仓库根」。
+
+1. 安装 [git-filter-repo](https://github.com/newren/git-filter-repo)（需单独安装，不是 Git 自带）。
+2. 在**原仓库克隆的副本**中执行：
+
+```bash
+cd /path/to/Low-GI-repo-copy
+git filter-repo --path market_assistant/ --path-rename market_assistant/:
+```
+
+3. 此时仓库根目录即为原 `market_assistant` 下的 `backend/`、`frontend/`、`docs/` 等。
+4. 将 `origin` 改为新远程或清空原远程后强制推送：
+
+```bash
+git remote add origin <你的新仓库 URL>
+git branch -M main
+git push -u origin main --force
+```
+
+5. 旧远程若废弃，在 Git 平台将旧库归档或删除，避免误用。
+
+### 方案 B：新仓库，不保留旧历史
+
+适用于「从零起一个干净远程，只装当前代码」。
+
+```bash
+cd market_assistant
+git init
+git add .
+git commit -m "chore: initial standalone market_assistant"
+git remote add origin <新仓库 URL>
+git branch -M main
+git push -u origin main
+```
+
+之后本地开发只在 `market_assistant` 目录内 `git pull` / `git push`。
+
+### 拆库后目录约定
+
+- 克隆下来的**仓库根** = 现在的 `market_assistant`（含 `backend/`、`frontend/`、`docs/`）。  
+- 文档中的路径仍写 `market_assistant/.env` 时，在「已拆库」情形下指**仓库根目录下的** `.env`（即 `.env` 在 clone 下来的根上）。
+
+### 已从跟踪中移除误提交文件（旧 monorepo 根目录上执行）
+
+若仍暂时保留大仓库，可在**原根目录**执行：
+
+```bash
+git rm -r --cached venv/ 2>/dev/null || true
+git rm -r --cached data/ 2>/dev/null || true
+git rm -r --cached .idea/ 2>/dev/null || true
+git rm --cached .env 2>/dev/null || true
+```
+
+## 4. 生产构建（简要）
+
+- **后端**：`pip install -r backend/requirements.txt`，`migrate`，用 gunicorn/uwsgi 等托管 WSGI，前面 Nginx 反代。  
+- **前端**：`cd frontend && npm ci && npm run build`，将 `dist/` 由 Nginx 托管静态资源，并把 `/api` 反代到 Django；同时把生产环境的 CORS/CSRF 与 `vite.config.js` 开发代理区分配置（生产一般同源或显式写 API 域名）。
+
+## 5. 与 `LOW_GI_PROJECT_ROOT` 的关系
+
+流水线 CSV、跑批目录默认写在 `LOW_GI_PROJECT_ROOT/data/JD/...`。该路径**可以**在服务器上位于 Web 代码库之外（例如单独数据盘），只要在 `.env` 中指向正确绝对路径即可。

docs/Market-Assistant-项目进展与里程碑.md

@@ -0,0 +1,172 @@
+# Market-Assistant（前台事业部 AI 增强启动器）— 项目进展与里程碑
+
+> 文档版本：2026-04-09（末次修订：**阶段 2.1～2.2 MVP** 市场策略制定已接入工作台；阶段 1.4 仍待业务侧脱敏截图/录屏）  
+> 目标：市场策略、竞品分析、文档体系、营销内容一键生成；手工作业投入降低 **≥50%**（需先定义基线与度量方式）。
+
+---
+
+## 一、总体目标与成功标准
+
+| 维度 | 说明 |
+|------|------|
+| **产品范围** | 面向前台事业部的统一入口：采集/分析 → 策略与文档 → 营销素材，尽量少切换工具。 |
+| **效率目标** | 在约定场景下（如：单次品类竞品简报、周报型材料），端到端耗时与人工步骤较基线下降 **≥50%**。 |
+| **质量目标** | 输出可追溯（数据来源、生成时间、模型/规则版本）；关键结论可人工复核。 |
+| **工程目标** | 每个里程碑结束：**代码提交并推送到远程仓库**（见文末 Git 检查清单）。 |
+
+---
+
+## 二、现状盘点（已有什么）
+
+以下基于 **Market-Assistant** 子项目当前能力整理。
+
+| 模块 | 已有能力 | 备注 / 缺口 |
+|------|-----------|-------------|
+| **后端** | Web 框架 + 任务创建、详情、下载、在线预览；后台线程执行任务 | 需明确生产部署、队列/进程模型是否从 Thread 演进 |
+| **配置** | 环境变量；数据根目录指向低 GI 项目数据区 | 文档化部署步骤与必填项 |
+| **京东链路** | 搜索/商详/评论等采集脚本及可选 AI 辅助 | 与「策略/文档/营销」产品化能力可继续加深 |
+| **前端** | 京东工作台：**搜索采集**、**任务列表**、**库内数据浏览**、**报告生成**、**报告查看**、**市场策略制定**；报告在线预览；报告统计项表单化 | 淘宝/天猫为占位页；缺少「一键 Word/PPT」与「营销内容」独立工作流 |
+| **数据落盘** | 任务产物含合并表、搜索导出、评价、商详、报告等 | lean 合并表与商详导出与库表对齐；全量归档规范仍可加强 |
+
+**京东 / 流水线数据层（2026-04-09 前后已落地）**
+
+- **合并宽表（lean）**：搜索全列 + 固定商详子集（品牌、到手价、店铺、类目路径、参数、配料）+ 评论摘要；已去掉入库与导出不需要的 HTTP 状态类字段。  
+- **商详导出（lean）**：与合并表商详子集一致；支持离线重写商详表再入库。  
+- **库表迁移**：已做合并表与商详行相关迁移；对旧库尽量可安全执行。  
+- **采集解析**：PC 搜索卖点等兜底；竞品报告类目读中文类目列与路径。  
+
+**结论**：**数据采集与部分分析骨架已在**，数据层在京东单次任务链路上 **schema 与库表已收敛一层**；下一步重点是 **产品化工作流**、**模板与一键生成**、**基线度量** 与 **多平台/多场景扩展**。
+
+**进展摘要（2026-04-09 前后已落地，竞品分析子闭环）**
+
+| 方向 | 内容 |
+|------|------|
+| **任务模型** | 任务上可配置报告调参（关注词、场景组、外部市场表）；创建时可带入，成功后也可单独修改再重生成。 |
+| **报告与摘要** | 按任务配置与内置默认合并后生成在线分析报告与结构化摘要；重新生成报告与拉取摘要均读当前任务配置。 |
+| **前端信息架构** | 菜单拆分：**报告生成**与**报告查看**；查看页 **一键下载简报包**（ZIP）。 |
+| **可用性** | 搜索采集页配置采集参数，与报告统计解耦；报告侧为中文表单 + 可选高级项；外部市场表列宽已约束。 |
+| **工程** | 含报告调参相关迁移与自动化测试（关注词等烟测）。 |
+| **演示与基线（文档）** | 演示走查、脱敏说明、手工基线表模板；工程说明含线程执行任务与生产演进提示。 |
+| **市场策略制定（阶段 2 MVP）** | 中文策略输出模板；工作台一键生成策略框架稿 + 附录速览（可选业务备注）；文档体系轻量说明 | Word 导出、LLM 润色、完整向导式多步表单为后续项 |
+
+---
+
+## 三、差距与下一步（要做什么）
+
+按目标拆成四条主线，可并行但建议优先级如下。
+
+1. **竞品分析闭环**：从「能跑任务」到「固定输入 → 固定输出（简报/PPT 大纲/对比表）」+ 引用数据源。  
+2. **市场策略**：在竞品与内部假设输入之上，结构化输出（定位、人群、渠道、节奏）并可导出 Word 等文稿。  
+3. **文档体系**：目录模板、命名规范、版本记录；与仓库或对象存储对齐（可选）。  
+4. **营销内容一键生成**：基于同一事实层生成多体裁（标题包、详情页卖点、短视频脚本提纲等），并支持人工编辑再导出。
+
+**阶段 1 当前建议的「下一步」（优先序）**
+
+1. **对齐阶段 0**：若尚未完成，补 **0.1～0.3**（标准场景、手工基线、远程仓库首次 push）。  
+2. **阶段 1.4 收口（业务侧）**：按演示走查打勾并附 **2～3 张脱敏截图** 或 **短录屏**（仓库可不存大 ZIP；脱敏见同目录说明）。  
+3. **阶段 2 持续推进**：模板与规则草稿已可用；可补 Word 导出、变更日志、向导增强（见下表阶段 2）。  
+4. **可选增强**：简报包内 **打印友好 HTML** 或 **Word 导出**。  
+5. **工程债**：线程模型已记在工程说明中；生产部署再扩写部署文档。
+
+---
+
+## 四、阶段规划、小任务与时间节点
+
+> 起始参考日：**2026-04-09**。若启动日晚于该日，整体顺延；**每个阶段结束日 = 必须完成「提交 + 推送」的截止日**。
+
+### 阶段 0：启动与基线（约 1 周）
+
+| 序号 | 任务 | 产出 | 建议完成日 |
+|------|------|------|------------|
+| 0.1 | 与业务方确认 2～3 个**标准场景**（如：低 GI 品类京东竞品周报） | 场景说明文档（可放本目录或 Confluence） | **2026-04-11** |
+| 0.2 | 为每个场景记录**当前手工步骤**与平均耗时 | 基线表（步骤数、分钟数、责任人） | **2026-04-14** |
+| 0.3 | 远程仓库初始化：忽略规则、分支策略、最小运行说明 | 可克隆可运行说明 + **首次 push** | **2026-04-16** |
+
+**阶段 0 里程碑 Git**：合并「基线说明 + README」等到主分支并 **push**。
+
+---
+
+### 阶段 1：竞品分析 MVP（约 2～3 周）
+
+| 序号 | 任务 | 产出 | 建议完成日 |
+|------|------|------|------------|
+| 1.1 | 统一单次任务的**输出规范**（字段、报告结构、失败重试） | 流水线输出说明 + OpenAPI 子集 + 示例 JSON（**已落盘**；含报告调参、更新配置、默认模板等） | **2026-04-23** |
+| 1.2 | 后端：从任务结果生成**结构化竞品摘要**（规则 + 可选 LLM） | 可查询的结构化摘要能力（**规则版已可用**；LLM 润色为后续项） | **2026-04-30** |
+| 1.3 | 前端：竞品页一键「生成简报」+ 在线阅读/下载 | UI 联调 | **2026-05-07**（**已交付**：报告查看页「一键下载简报包」ZIP） |
+| 1.4 | 联调与演示数据集（脱敏） | 演示录屏或截图 + 样例数据 | **2026-05-09**（**文档已备**；**待业务补**：脱敏截图/录屏；大样本勿提交 Git） |
+
+**阶段 1 里程碑 Git**：打 tag 可选，如 `v0.1-competitor-mvp`，并 **push**。
+
+---
+
+### 阶段 2：市场策略 + 文档体系（约 2～3 周）
+
+| 序号 | 任务 | 产出 | 建议完成日 |
+|------|------|------|------------|
+| 2.1 | 定义「市场策略」**输出模板**（章节、必填项、可选 AI 补全） | 中文策略模板（**已落盘**；Word 版可选后续） | **2026-05-16** |
+| 2.2 | 后端：策略生成能力（输入：结构化摘要 + 业务约束） | 策略制定接口（**已落盘**，规则无 LLM） | **2026-05-23** |
+| 2.3 | 文档体系：文件夹模板、命名规则、变更日志或版本页 | 文档体系轻量说明（**已落盘**）；变更日志待团队约定 | **2026-05-28** |
+| 2.4 | 前端：「市场策略制定」向导式表单 + 预览导出 | 工作台任务选择 + 备注 + 预览/下载（**已落盘**）；多步向导可后续增强 | **2026-05-30** |
+
+**阶段 2 里程碑 Git**：**push**；建议特性分支合并后主分支可演示。
+
+---
+
+### 阶段 3：营销内容一键生成（约 2～3 周）
+
+| 序号 | 任务 | 产出 | 建议完成日 |
+|------|------|------|------------|
+| 3.1 | 定义营销**体裁清单**（标题、卖点、FAQ、短视频提纲等）与字数约束 | 提示词/规则配置 | **2026-06-06** |
+| 3.2 | 后端：批量生成接口（同一事实层 → 多体裁） | API + 限流/缓存策略 | **2026-06-13** |
+| 3.3 | 前端：一键生成 + 分 tab 展示 + 复制/导出 | UI | **2026-06-18** |
+| 3.4 | 与阶段 0 基线对比，粗测耗时下降比例 | 内部复盘表 | **2026-06-20** |
+
+**阶段 3 里程碑 Git**：**push**；若达到 ≥50% 目标则记录证据（前后对比表）。
+
+---
+
+### 阶段 4：硬化与推广（约 1～2 周）
+
+| 序号 | 任务 | 产出 | 建议完成日 |
+|------|------|------|------------|
+| 4.1 | 权限、密钥、日志与审计（按公司规范） | 配置说明 | **2026-06-25** |
+| 4.2 | 部署文档（Docker/主机二选一） | README 或单独部署说明 | **2026-06-27** |
+| 4.3 | 前台事业部试用反馈一轮 | 问题清单 + 下一迭代 backlog | **2026-06-30** |
+
+**阶段 4 里程碑 Git**：**push**；可选 `v1.0-internal-release`。
+
+---
+
+## 五、每个节点：提交与推送仓库（必做）
+
+建议在每次「任务行」完成或「阶段结束日」执行：
+
+1. 确认工作区仅包含预期变更。  
+2. 提交并写清说明（团队统一前缀更好）。  
+3. 推送到远程约定分支。  
+4. 若使用 PR：合并后再 **push** 更新后的主分支。  
+5. 重要里程碑可在远程打 **tag** 便于回溯。
+
+**原则**：不以「本地只有」为完成标准；**远程仓库可见**才算里程碑闭合。
+
+---
+
+## 六、风险与依赖
+
+| 风险 | 应对 |
+|------|------|
+| 爬虫稳定性 / 合规 | 明确使用范围；失败降级为「人工上传 CSV」 |
+| LLM 成本与质量波动 | 规则优先 + LLM 增强；版本化 prompt；抽样质检 |
+| 「50%」无基线 | 阶段 0 必须完成基线表，否则目标不可验收 |
+| 多人协作冲突 | 按阶段切分支；文档与接口先行 |
+
+---
+
+## 七、文档维护
+
+- 每次阶段结束后更新本文件「二、现状盘点」与「四、阶段规划」中的**实际完成日**与**偏差说明**；数据层/爬虫有结构变更时在「京东 / 流水线数据层」小节补一句即可。  
+- 负责人、会议纪要与更细任务可链接到内部 wiki，本文件保持**一页纸可读完**。
+
+---
+
+*项目进展与里程碑（Market-Assistant）*

docs/demo/README.md

@@ -0,0 +1,9 @@
+# 演示与基线（阶段 0 / 1.4）
+
+| 材料 | 用途 |
+|------|------|
+| 竞品分析演示走查 | 按菜单逐步验收采集、报告、简报包 |
+| 脱敏与样例说明 | 对外分享前须处理的敏感项与仓库边界 |
+| 手工基线表模板 | 阶段 0.2：记录当前手工耗时，便于算「50%」 |
+
+工程约束（线程执行任务）见同目录上一级的 **工程说明**。

docs/demo/手工基线表模板.md

@@ -0,0 +1,32 @@
+# 手工基线表模板（阶段 0.2）
+
+> 与业务确认「标准场景」后，为**每个场景**填一张，用于后续验收「效率提升 ≥50%」。  
+> 仅模板，**不含真实数据**；可打印或复制到 Excel。
+
+## 场景名称
+
+（例如：低 GI 品类 · 京东竞品周报）
+
+## 当前手工流程
+
+| 序号 | 步骤简述 | 责任人角色 | 单次耗时（分钟） | 工具 |
+|------|----------|------------|------------------|------|
+| 1 | | | | |
+| 2 | | | | |
+| 3 | | | | |
+| … | | | | |
+
+**合计单次总耗时（分钟）**：_____  
+**每月大致次数**：_____
+
+## 痛点（可选）
+
+- 
+
+## 备注
+
+- 与 Market-Assistant 自动化流程对照时，用「步骤数、总分钟数」做前后对比。
+
+---
+
+*手工基线表模板*

docs/demo/竞品分析演示走查.md

@@ -0,0 +1,38 @@
+# 竞品分析能力 — 演示走查（内部）
+
+> 用途：业务或新同事**不依赖录屏**也能按步骤验收「采集 → 报告 → 简报包」。  
+> 数据须**脱敏**：勿在共享环境使用真实 Cookie、真实手机号/地址；关键词可用「测试词」类替代。
+
+## 前置
+
+- 已在环境变量中配置**数据根目录**，且本机可写入京东数据区。  
+- 后端已迁移并启动开发服务。  
+- 前端开发服务或构建页可打开 Market-Assistant 京东工作台。
+
+## 走查步骤（建议顺序）
+
+| 步骤 | 菜单 / 页面 | 操作 | 预期 |
+|------|-------------|------|------|
+| 1 | **搜索采集** | 填搜索词；按需配置登录与请求节奏（测试用 Cookie） | 提交成功，可进入任务列表 |
+| 2 | **任务列表** | 刷新，观察状态直至成功或失败（失败则查看错误说明） | 成功后可见本批输出目录 |
+| 3 | **报告查看** | 选该任务 → **重新加载报告** | 出现报告正文预览（若无文件则按提示去报告生成） |
+| 4 | **报告生成**（若 3 无文件） | 同一任务 → **重新生成报告** | 返回报告查看再加载，应有预览 |
+| 5 | **报告查看** | **一键下载简报包** | 浏览器得到 ZIP 压缩包 |
+| 6 | 解压 ZIP | 打开说明、要点摘录、分析报告 | 内容完整、中文可读；结构化摘要可被 JSON 工具打开 |
+| 7 | **库内数据浏览**（可选） | 选同一任务，浏览合并表 / 搜索 / 评价等 | 行数与列说明与任务一致 |
+| 8 | **报告生成**（可选） | 改「关注词 / 场景」→ 保存 → 再 **重新生成报告** → 报告查看对比 | 统计与报告正文随配置变化（规则一致） |
+
+## 失败时快速对照
+
+- **服务不可用**：未配置数据根目录或采集副本路径不可用。  
+- **报告不存在**：分析报告文件尚未生成 → 到 **报告生成** 点「重新生成报告」。  
+- **简报包失败**：同上，须先生成分析报告。
+
+## 演示交付物建议（阶段 1.4）
+
+- 本走查表打勾截图 **2～3 张**（任务成功、报告预览、ZIP 目录列表即可）。  
+- 若对外分享：仅使用**脱敏**批次或打码后的截图，**不要**提交含 Cookie 的仓库文件。
+
+---
+
+*竞品分析演示走查*

docs/demo/脱敏与样例说明.md

@@ -0,0 +1,25 @@
+# 脱敏与对外样例说明
+
+## 哪些数据敏感
+
+| 类型 | 说明 | 处理建议 |
+|------|------|----------|
+| **Cookie / Token** | 等同登录凭证 | 永不入库到 Git；演示用临时 Cookie 或本地专用账号 |
+| **用户地址、电话、真实姓名** | 可能出现在评价正文 | 对外导出前抽样删改或不用含 PII 的批次 |
+| **内部绝对路径** | 批次目录可能含本机用户名 | 对外截图或文档中可打码或改为「…」示意 |
+| **未公开商业数据** | 若关键词、品类属保密 | 演示关键词改为「测试品类 A」等 |
+
+## 「样例数据」在仓库中的边界
+
+- **推荐**：仓库内仅保留 **小规模、无隐私的结构化示例** 与 **字段说明**，不提交完整跑批结果目录。  
+- **若必须附小样本 CSV**：行数 ≤ 20，且对 `tagCommentContent`、`comment_preview` 等列做**虚构替换**或清空。  
+- **简报包 ZIP**：默认不提交；走查时各人本地生成即可。
+
+## 对外演示话术（可选）
+
+- 「数据来自京东 PC 公开列表与商详，统计规则见报告第一章方法说明。」  
+- 「要点摘录与 JSON 为规则引擎生成，定稿需业务复核。」
+
+---
+
+*脱敏与样例说明*

docs/doc-system/README.md

@@ -0,0 +1,34 @@
+# 文档体系（轻量说明）
+
+> 面向协作：哪些材料放哪、命名习惯、与里程碑如何对齐。**不**替代研发内部接口契约（后者单独维护）。
+
+## 推荐阅读顺序
+
+| 文档 | 用途 |
+|------|------|
+| 项目进展与里程碑 | 一页纸：目标、阶段、优先级 |
+| 流水线单次任务输出说明 | 任务产物、状态、接口能力（偏研发） |
+| OpenAPI 子集 | 可导入 API 调试工具 |
+| 示例数据 | JSON 形状参考 |
+| 演示与脱敏 | 走查步骤、基线模板、样例约束 |
+| 策略类模板 | 人工撰写或对照自动草稿章节 |
+| 工程说明 | 实现上的约束（如任务执行方式） |
+
+## 命名与存放
+
+- 说明类：文本文档，中文文件名可接受。  
+- 大体积数据、含隐私的跑批结果：**勿提交** Git；用内部网盘或对象存储，并在说明里写清获取方式。  
+- 对外演示：只用脱敏截图/录屏 + 小规模样例。
+
+## 版本与变更
+
+- 产品级大变更：在里程碑文档中记一句；若团队使用变更日志，可在仓库根后续追加（当前未强制）。
+
+## 常见产出形态
+
+- **市场策略制定稿**：策略框架与待填命题；下载或复制后由业务放入 Confluence / Word 再定稿。  
+- **竞品简报包**：ZIP，内含报告、结构化摘要、要点摘录等，详见流水线输出说明。
+
+---
+
+*文档体系说明（Market-Assistant）*

docs/engineering-notes.md

@@ -0,0 +1,16 @@
+# 工程说明（单机演示 vs 生产）
+
+## 任务执行模型
+
+- **当前实现**：创建京东流水线任务后，后端在 **后台线程** 中执行采集与报告步骤。  
+- **含义**：适合**本机或单进程演示**；进程重启会中断未完成任务。  
+- **生产演进建议**：改为 **Celery / RQ / Django-Q** 等任务队列 + 独立 worker 进程；并配置 **超时、重试、并发上限** 与日志采集。
+
+## 与里程碑文档的关系
+
+- 阶段 1「竞品分析 MVP」在单机环境下可完整演示。  
+- 对外部试点或 SLA 承诺前，应补充 **部署文档**（进程守护、环境变量、静态资源）与 **队列化** 方案。
+
+---
+
+*工程说明（Market-Assistant）*

docs/examples/competitor_brief_schema_v1.json

@@ -0,0 +1,108 @@
+{
+  "schema_version": 1,
+  "keyword": "低GI",
+  "batch_label": "20260409 154041",
+  "run_dir": "<本机批次输出目录，示例略>",
+  "scope": {
+    "merged_sku_count": 5,
+    "comment_flat_rows": 112,
+    "structure_source_rows": 117,
+    "uses_pc_search_list_export": true
+  },
+  "meta": {
+    "page_start": 1,
+    "page_to": 2,
+    "merged_csv_mode": "lean"
+  },
+  "pc_search_raw": {
+    "result_count_consensus": 12345,
+    "list_keyword": "低GI",
+    "result_count_uniques": [12340, 12345],
+    "raw_json_files_scanned": 2
+  },
+  "list_visibility_proxy": {
+    "total_rows": 117,
+    "unique_skus": 95
+  },
+  "concentration": {
+    "shops_from_list": {
+      "cr1": 0.12,
+      "cr3": 0.35,
+      "top_label": "某旗舰店",
+      "top_share_pct": "12.0%"
+    },
+    "list_brand_field": null,
+    "detail_brand_among_merged": {
+      "cr1": 0.2,
+      "cr3": 0.6,
+      "top_label": "品牌A",
+      "top_share_pct": "20.0%"
+    }
+  },
+  "category_mix_top": [{ "label": "休闲食品", "count": 40 }],
+  "price_stats": {
+    "min": 14.9,
+    "max": 39.9,
+    "mean": 21.5,
+    "n": 117,
+    "median": 16.9
+  },
+  "price_stats_source": "pc_search_export_all_rows",
+  "price_stats_merged_sample": {
+    "min": 14.9,
+    "max": 34.9,
+    "mean": 22.1,
+    "n": 5,
+    "median": 16.9
+  },
+  "price_stats_list_export": {
+    "min": 14.9,
+    "max": 39.9,
+    "mean": 21.5,
+    "n": 117,
+    "median": 16.9
+  },
+  "comment_focus_keywords": [{ "word": "口感", "count": 48 }],
+  "usage_scenarios": [
+    { "scenario": "控糖/血糖相关", "count": 12, "share_of_text_units": 0.15 }
+  ],
+  "strategy_hints": ["样本内…（待验证）"],
+  "matrix_by_group": [
+    {
+      "group": "饼干",
+      "sku_count": 3,
+      "skus": [
+        {
+          "sku_id": "100065199809",
+          "title": "示例标题",
+          "brand": "示例品牌",
+          "list_price_show": "16.90",
+          "coupon_or_detail_price": "",
+          "detail_price_final": "16.41",
+          "shop": "示例店",
+          "category": "休闲食品 > 饼干 > 粗粮饼干",
+          "selling_point": "",
+          "comment_fuzzy": "20万+"
+        }
+      ]
+    }
+  ],
+  "consumer_feedback_by_matrix_group": [
+    {
+      "group": "饼干",
+      "comment_rows": 40,
+      "focus_keyword_hits": [{ "word": "口感", "count": 10 }],
+      "scenarios_top": [
+        {
+          "scenario": "早餐/代餐",
+          "count": 5,
+          "share_of_text_units": 0.2
+        }
+      ]
+    }
+  ],
+  "notes": [
+    "与在线分析报告统计口径一致；主题词与场景为预设词表，非 NLP 主题模型。",
+    "价格来自展示字段抽取，含促销与规格差异。"
+  ]
+}

docs/examples/dataset_summary.json

@@ -0,0 +1,31 @@
+{
+  "job_id": 5,
+  "keyword": "低GI",
+  "status": "success",
+  "search_rows": 117,
+  "detail_rows": 5,
+  "comment_rows": 112,
+  "merged_rows": 5,
+  "search_columns": [
+    { "key": "sku_id", "label": "SKU(skuId)" },
+    { "key": "title", "label": "标题(wareName)" }
+  ],
+  "detail_columns": [
+    { "key": "sku_id", "label": "skuId" },
+    { "key": "detail_brand", "label": "detail_brand" },
+    { "key": "detail_price_final", "label": "detail_price_final" },
+    { "key": "detail_shop_name", "label": "detail_shop_name" },
+    { "key": "detail_category_path", "label": "detail_category_path" },
+    { "key": "detail_product_attributes", "label": "detail_product_attributes" },
+    { "key": "detail_body_ingredients", "label": "detail_body_ingredients" }
+  ],
+  "comment_columns": [
+    { "key": "sku_id", "label": "sku" },
+    { "key": "tag_comment_content", "label": "tagCommentContent" }
+  ],
+  "merged_columns": [
+    { "key": "pipeline_keyword", "label": "pipeline_keyword" },
+    { "key": "sku_id", "label": "SKU(skuId)" },
+    { "key": "detail_brand", "label": "detail_brand" }
+  ]
+}

docs/examples/pipeline_job_success.json

@@ -0,0 +1,27 @@
+{
+  "id": 5,
+  "platform": "jd",
+  "keyword": "低GI",
+  "max_skus": 5,
+  "page_start": 1,
+  "page_to": 2,
+  "pipeline_run_dir": "",
+  "cookie_file_path": "",
+  "inline_cookie_used": false,
+  "pvid": "",
+  "request_delay": "30-60",
+  "list_pages": "1-2",
+  "scenario_filter_enabled": true,
+  "status": "success",
+  "run_dir": "<本机批次输出目录，示例略>",
+  "error_message": "",
+  "analysis_artifacts": {
+    "merged": true,
+    "pc_search": true,
+    "comments": true,
+    "detail_ware": true,
+    "report": true
+  },
+  "created_at": "2026-04-09T12:00:00+08:00",
+  "updated_at": "2026-04-09T12:45:00+08:00"
+}

docs/openapi/pipeline-jobs.openapi.yaml

@@ -0,0 +1,587 @@
+openapi: 3.0.3
+info:
+  title: Market-Assistant — Pipeline & JD 数据集 API（子集）
+  description: |
+    与 `docs/pipeline-job-output-spec.md` 一致的核心路径；完整行为以 `pipeline/views.py` 为准。
+  version: "1.0.0"
+
+servers:
+  - url: http://127.0.0.1:8000
+    description: 本地 Django 默认端口
+
+tags:
+  - name: jobs
+    description: 京东流水线任务
+  - name: dataset
+    description: 任务入库后的分页与导出
+  - name: jd
+    description: 全局 SKU 主档与快照
+
+paths:
+  /api/jobs/:
+    get:
+      tags: [jobs]
+      summary: 任务列表（最近 200 条）
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  $ref: "#/components/schemas/PipelineJob"
+    post:
+      tags: [jobs]
+      summary: 创建任务并异步执行
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/CreatePipelineJob"
+      responses:
+        "201":
+          description: Created
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/PipelineJob"
+        "503":
+          description: LOW_GI_PROJECT_ROOT 未配置
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ErrorDetail"
+
+  /api/jobs/{id}/:
+    get:
+      tags: [jobs]
+      summary: 任务详情
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/PipelineJob"
+        "404":
+          description: Not found
+    patch:
+      tags: [jobs]
+      summary: 仅更新 report_config（报告关注词/场景/外部市场表等）
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/PatchReportConfig"
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/PipelineJob"
+        "400":
+          description: 校验失败（未知键或体积过大等）
+        "404":
+          description: Not found
+
+  /api/jobs/{id}/cancel/:
+    post:
+      tags: [jobs]
+      summary: 请求终止待执行/执行中的采集任务（协作式，在可停点结束）
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      responses:
+        "200":
+          description: OK（cancellation_requested=true，随后状态变为 cancelled）
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/PipelineJob"
+        "400":
+          description: 任务已结束，不可再终止
+        "404":
+          description: Not found
+
+  /api/report-config-defaults/:
+    get:
+      tags: [jobs]
+      summary: 报告调参默认模板（与 jd_competitor_report 脚本常量一致）
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ReportConfig"
+        "503":
+          description: 爬虫副本路径不可用等
+
+  /api/jobs/{id}/download/:
+    get:
+      tags: [jobs]
+      summary: 下载 run_dir 内产物
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+        - name: name
+          in: query
+          required: true
+          schema:
+            type: string
+            enum: [merged, pc_search, comments, detail_ware, report]
+      responses:
+        "200":
+          description: 文件流
+        "404":
+          description: 任务未成功或文件不存在
+
+  /api/jobs/{id}/dataset/summary/:
+    get:
+      tags: [dataset]
+      summary: 库内行数与列说明
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/DatasetSummary"
+
+  /api/jobs/{id}/dataset/merged/:
+    get:
+      tags: [dataset]
+      summary: 合并宽表分页
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+        - $ref: "#/components/parameters/Page"
+        - $ref: "#/components/parameters/PageSize"
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/PaginatedMergedRows"
+
+  /api/jobs/{id}/export/:
+    get:
+      tags: [dataset]
+      summary: 导出 JSON / CSV / xlsx
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+        - name: kind
+          in: query
+          schema:
+            type: string
+            enum: [search, detail, comments, merged, all]
+        - name: export_fmt
+          in: query
+          schema:
+            type: string
+            enum: [json, csv, xlsx]
+      responses:
+        "200":
+          description: 文件下载
+
+  /api/jobs/{id}/regenerate-report/:
+    post:
+      tags: [jobs]
+      summary: 根据已有 CSV 重写 competitor_analysis.md（不重新爬取）；可选大模型生成（AI_crawler 同网关）
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      requestBody:
+        required: false
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/RegenerateReportRequest"
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/PipelineJob"
+        "400":
+          description: 任务状态或路径无效
+        "502":
+          description: 大模型网关 HTTP 错误
+        "503":
+          description: 未配置 OPENAI_* / LLM_* 等（generator=llm 时）
+
+  /api/jobs/{id}/competitor-brief/:
+    get:
+      tags: [jobs]
+      summary: 结构化竞品摘要 JSON（与 Markdown 报告统计口径一致）
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      responses:
+        "200":
+          description: schema_version=1 的规则摘要
+          content:
+            application/json:
+              schema:
+                type: object
+                additionalProperties: true
+        "400":
+          description: 任务未成功或缺少 run_dir / 合并表
+
+  /api/jobs/{id}/competitor-brief-pack/:
+    get:
+      tags: [jobs]
+      summary: 一键简报包（ZIP：报告 md + 摘要 json + 要点摘录 md + 说明）
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      responses:
+        "200":
+          description: application/zip 附件
+          content:
+            application/zip:
+              schema:
+                type: string
+                format: binary
+        "400":
+          description: 缺少 competitor_analysis.md 等
+        "503":
+          description: LOW_GI_PROJECT_ROOT 未配置
+
+  /api/jobs/{id}/strategy-draft/:
+    post:
+      tags: [jobs]
+      summary: 市场策略 Markdown 草稿（规则或可选大模型润色，网关同 AI_crawler）
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      requestBody:
+        required: false
+        content:
+          application/json:
+            schema:
+              $ref: "#/components/schemas/StrategyDraftRequest"
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/StrategyDraftResponse"
+        "400":
+          description: 任务未成功、缺少 run_dir，或 brief 构建失败
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ErrorDetail"
+        "404":
+          description: Not found
+        "502":
+          description: 大模型网关 HTTP 错误
+        "503":
+          description: LOW_GI_PROJECT_ROOT 未配置，或大模型凭证未配置（generator=llm）
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/ErrorDetail"
+
+  /api/jobs/{id}/ingest-merged/:
+    post:
+      tags: [dataset]
+      summary: 从 run_dir 将 CSV 全量写入库（幂等：先清空该任务四类行）
+      parameters:
+        - $ref: "#/components/parameters/JobId"
+      responses:
+        "200":
+          description: 统计信息（dataset + merged 块）
+
+  /api/jd/products/:
+    get:
+      tags: [jd]
+      summary: SKU 主档列表
+      responses:
+        "200":
+          description: OK
+
+components:
+  parameters:
+    JobId:
+      name: id
+      in: path
+      required: true
+      schema:
+        type: integer
+    Page:
+      name: page
+      in: query
+      schema:
+        type: integer
+        default: 1
+    PageSize:
+      name: page_size
+      in: query
+      schema:
+        type: integer
+        default: 20
+
+  schemas:
+    ErrorDetail:
+      type: object
+      properties:
+        detail:
+          type: string
+
+    CreatePipelineJob:
+      type: object
+      required: [platform, keyword]
+      properties:
+        platform:
+          type: string
+          example: jd
+        keyword:
+          type: string
+        max_skus:
+          type: integer
+        page_start:
+          type: integer
+        page_to:
+          type: integer
+        pipeline_run_dir:
+          type: string
+        cookie_file_path:
+          type: string
+        cookie_text:
+          type: string
+        pvid:
+          type: string
+        request_delay:
+          type: string
+        list_pages:
+          type: string
+        scenario_filter_enabled:
+          type: boolean
+        report_config:
+          $ref: "#/components/schemas/ReportConfig"
+
+    ReportConfig:
+      type: object
+      description: |
+        允许键仅限下列三项；可全部省略或 `{}` 表示使用脚本内置默认。
+      properties:
+        comment_focus_words:
+          type: array
+          items:
+            type: string
+        comment_scenario_groups:
+          type: array
+          items:
+            type: object
+            properties:
+              label:
+                type: string
+              triggers:
+                type: array
+                items:
+                  type: string
+        external_market_table_rows:
+          type: array
+          items:
+            type: object
+            properties:
+              indicator:
+                type: string
+              value_and_scope:
+                type: string
+              source:
+                type: string
+              year:
+                type: string
+
+    PatchReportConfig:
+      type: object
+      required: [report_config]
+      properties:
+        report_config:
+          $ref: "#/components/schemas/ReportConfig"
+
+    PipelineJob:
+      type: object
+      properties:
+        id:
+          type: integer
+        platform:
+          type: string
+        keyword:
+          type: string
+        status:
+          type: string
+          enum: [pending, running, success, failed, cancelled]
+        cancellation_requested:
+          type: boolean
+        run_dir:
+          type: string
+        error_message:
+          type: string
+        analysis_artifacts:
+          type: object
+          additionalProperties:
+            type: boolean
+          nullable: true
+        created_at:
+          type: string
+          format: date-time
+        updated_at:
+          type: string
+          format: date-time
+        report_config:
+          $ref: "#/components/schemas/ReportConfig"
+
+    ColumnMeta:
+      type: object
+      properties:
+        key:
+          type: string
+        label:
+          type: string
+
+    DatasetSummary:
+      type: object
+      properties:
+        job_id:
+          type: integer
+        keyword:
+          type: string
+        status:
+          type: string
+        search_rows:
+          type: integer
+        detail_rows:
+          type: integer
+        comment_rows:
+          type: integer
+        merged_rows:
+          type: integer
+        search_columns:
+          type: array
+          items:
+            $ref: "#/components/schemas/ColumnMeta"
+        detail_columns:
+          type: array
+          items:
+            $ref: "#/components/schemas/ColumnMeta"
+        comment_columns:
+          type: array
+          items:
+            $ref: "#/components/schemas/ColumnMeta"
+        merged_columns:
+          type: array
+          items:
+            $ref: "#/components/schemas/ColumnMeta"
+
+    PaginatedMergedRows:
+      type: object
+      properties:
+        total:
+          type: integer
+        page:
+          type: integer
+        page_size:
+          type: integer
+        results:
+          type: array
+          items:
+            type: object
+            additionalProperties: true
+
+    RegenerateReportRequest:
+      type: object
+      properties:
+        generator:
+          type: string
+          enum: [rules, llm]
+          default: rules
+          description: rules=jd_competitor_report 规则引擎；llm=结构化摘要 JSON + AI_crawler 文本接口
+
+    StrategyDraftRequest:
+      type: object
+      properties:
+        generator:
+          type: string
+          enum: [rules, llm]
+          default: rules
+          description: llm 时在规则底稿与同任务摘要基础上由大模型润色全文
+        business_notes:
+          type: string
+          maxLength: 20000
+          description: 可选业务约束/假设，写入 §八
+        product_role:
+          type: string
+          maxLength: 500
+        time_horizon:
+          type: string
+          maxLength: 200
+        success_criteria:
+          type: string
+          maxLength: 2000
+        non_goals:
+          type: string
+          maxLength: 1000
+        battlefield_one_line:
+          type: string
+          maxLength: 1000
+        positioning_choice:
+          type: string
+          enum: ['', 'top', 'mid', 'entry', 'different']
+          description: 价格带主定位；空字符串表示文稿中四项均为未勾选
+        competitive_stance:
+          type: string
+          enum: ['', 'flank', 'head_on', 'both', 'undecided']
+        pillar_product:
+          type: string
+          maxLength: 800
+        pillar_price:
+          type: string
+          maxLength: 800
+        pillar_channel:
+          type: string
+          maxLength: 800
+        pillar_comm:
+          type: string
+          maxLength: 800
+        ack_risk_keywords:
+          type: boolean
+          default: false
+        ack_risk_price:
+          type: boolean
+          default: false
+        ack_risk_concentration:
+          type: boolean
+          default: false
+
+    StrategyDraftResponse:
+      type: object
+      required:
+        - schema_version
+        - job_id
+        - keyword
+        - generated_at
+        - source
+        - markdown
+      properties:
+        schema_version:
+          type: integer
+          example: 1
+        job_id:
+          type: integer
+        keyword:
+          type: string
+        generated_at:
+          type: string
+          format: date-time
+        source:
+          type: string
+          description: structured_summary_rules_v1 或 llm_text_ai_crawler_v1
+          example: structured_summary_rules_v1
+        markdown:
+          type: string

docs/pipeline-job-output-spec.md

@@ -0,0 +1,92 @@
+# 京东流水线单次任务 — 输出规范（v1）
+
+> 与阶段 1「竞品分析 MVP」任务 **1.1** 对齐：约定任务产物、入库数据与接口能力。  
+> **列与字段**：与任务产物入库及导出一致，以后端当前版本为准（研发可查内部说明）。
+
+---
+
+## 1. 任务生命周期与状态
+
+| 状态 | 含义 |
+|------|------|
+| 待执行 | 已创建，等待运行 |
+| 执行中 | 正在跑采集与报告 |
+| 成功 | 本批次输出已写入指定目录 |
+| 失败 | 可阅读错误说明排查 |
+
+- **创建任务**：通过任务创建接口提交；执行在后台进行，需轮询任务详情直至结束。  
+- **失败重试**：当前为「新建任务」模式；同一关键词可再次提交，**不会**对同一任务 ID 自动重试。  
+- **环境**：数据根目录未配置时，创建任务会返回服务不可用类错误。
+
+### 1.1 报告调参（可选）
+
+任务可附带一份 **报告调参配置**（缺省为空对象）。为空时表示生成**在线分析报告**与**结构化摘要**时一律使用**系统内置默认**（关注词、场景组、外部市场表等）。
+
+| 能力 | 说明 |
+|------|------|
+| **创建时带入** | 创建任务时可一并提交调参（仅允许约定字段；体积与未知字段由后端校验）。 |
+| **成功后修改** | 可在任务成功后更新调参；更新后需再执行「重新生成报告」或刷新结构化摘要，才会反映到新文稿或接口结果。 |
+| **默认模板** | 「报告调参默认」接口返回与内置规则一致的示例，供界面「填入推荐示例」。 |
+
+可调字段包括：评价关注词列表、用途/场景分组、外部市场表行等，具体键名与结构以实现为准。
+
+---
+
+## 2. 本批次输出目录与可下载内容
+
+成功任务会在数据根下生成**本批次输出目录**（绝对路径，由系统分配）。
+
+可下载/预览的**内容类型**包括（名称以实现为准）：
+
+| 类型 | 说明 |
+|------|------|
+| 合并宽表 | 搜索 + 商详子集 + 评论摘要（精简列） |
+| 搜索列表导出 | 列表侧表格数据 |
+| 商详导出 | 与合并表对齐的商详子集 |
+| 评价扁平 | 评价明细表 |
+| **分析报告** | 在线阅读用的主报告文稿 |
+
+- **下载 / 预览**：通过任务对应的下载、预览能力获取（仅成功且文件已生成时可用）。  
+- **重新生成报告**（不重新爬取）：仅用本批次已有表格与元数据刷新**分析报告**正文。  
+- **结构化竞品摘要**：规则生成的 JSON，与在线分析报告**统计口径一致**；价格等指标在能解析列表价时以列表为准，否则以深入样本为准，响应内会标明口径。  
+- **一键简报包**：ZIP，内含完整分析报告稿、结构化摘要、要点摘录与说明；须已生成主报告。  
+- **市场策略制定**：提交可选业务备注后返回策略向文稿（策略框架 + 附录数据速览），数据与同任务结构化摘要一致；仅成功且数据齐全时可用。  
+- **商详表离线修正再入库**：由运维/研发在侧链完成后再走全量入库能力。
+
+---
+
+## 3. 宽表与商详导出（精简列）
+
+- 合并表列顺序与内部键由系统定义。  
+- 商详表为 SKU 及与合并表对齐的商详子集。  
+- 精简导出**不含** HTTP 状态类字段。
+
+---
+
+## 4. 入库后的浏览与导出
+
+任务成功后自动入库（也可手动全量入库）。支持：
+
+- **数据集摘要**：各表行数与列说明。  
+- **分页浏览**：搜索 / 商详 / 评价 / 合并表等。  
+- **导出**：按种类与格式（如 JSON、CSV、表格）；导出时**勿使用**名为「format」的查询参数（与框架保留字冲突）。
+
+分页结果中的字段名与库表一致。
+
+---
+
+## 5. 全局商品与快照
+
+商品主档、单 SKU、按任务的历史快照等能力，与 OpenAPI 中路径一致。
+
+---
+
+## 6. 机器可读约定
+
+- **OpenAPI**：仓库内提供核心路径子集，可导入调试工具。  
+- **示例数据**：仓库内提供 JSON 形状示例。  
+- 与线上行为不一致时，**以当前部署版本为准**，并回写说明与 OpenAPI。
+
+---
+
+*流水线单次任务输出规范（v1）*

docs/templates/market-strategy-brief.zh.md

@@ -0,0 +1,33 @@
+# 市场策略输出模板（中文）
+
+> 本文件供**人工撰写**或**对照**工作台「市场策略制定」自动草稿使用。  
+> 自动草稿在**同一任务的 Web 工作台**中一键生成，与同任务结构化分析数据一致；**规则拼接、非 LLM**，定稿须业务修订。
+
+## 自动草稿章节
+
+| 章节 | 作用 |
+|------|------|
+| 一、战略背景与目标 | 待业务补全（角色、周期、成功标准） |
+| 二、战场界定 | 关键词、样本规模、一句话战场（待填） |
+| 三、竞争格局 → 策略含义 | 集中度粗判 + 自测提问；类目 Top 仅作提示 |
+| 四、价格带与定位选项 | 区间/中位数 + 定位勾选框架 |
+| 五、用户需求与场景假设 | 由关注词/场景转为**待验证命题** |
+| 六、机会与策略支柱 | 规则提示 + 四支柱表格（待填） |
+| 七、风险与待证伪 | 抽样与数据质量自检清单 |
+| 八、业务约束与判断 | 工作台 **业务备注** |
+| 九、建议下一步 | 会议对齐、证据挂钩、节奏与指标 |
+| 附录 | 本任务关键数据速览 |
+
+## 可选：完全手写时的建议目录
+
+1. 背景与目标（本品、时间范围、成功标准）  
+2. 市场与用户（细分、痛点、购买路径）  
+3. 竞争格局（价位、渠道、头部玩法）  
+4. 机会与风险  
+5. 策略支柱（产品 / 价格 / 渠道 / 传播）  
+6. 节奏与资源（12 周视图）  
+7. 指标与复盘  
+
+---
+
+*策略输出模板（中文）*

frontend/.gitignore

@@ -0,0 +1,24 @@
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+lerna-debug.log*
+
+node_modules
+dist
+dist-ssr
+*.local
+
+# Editor directories and files
+.vscode/*
+!.vscode/extensions.json
+.idea
+.DS_Store
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?

frontend/README.md

@@ -0,0 +1,19 @@
+# Market-Assistant 前端（Vue 3 + Vite）
+
+## 启动命令
+
+```bash
+npm install
+npm run dev
+```
+
+默认本地开发端口见终端提示。请先启动后端开发服务，以便 API 代理可用。
+
+## 其他脚本
+
+- **npm run build** — 生产构建  
+- **npm run preview** — 本地预览构建结果  
+
+## 项目说明
+
+完整说明见上一级目录的 **README**。

frontend/index.html

@@ -0,0 +1,13 @@
+<!doctype html>
+<html lang="zh-CN">
+  <head>
+    <meta charset="UTF-8" />
+    <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>Market-Assistant</title>
+  </head>
+  <body>
+    <div id="app"></div>
+    <script type="module" src="/src/main.js"></script>
+  </body>
+</html>

frontend/package.json

@@ -0,0 +1,26 @@
+{
+  "name": "frontend",
+  "private": true,
+  "version": "0.0.0",
+  "type": "module",
+  "scripts": {
+    "dev": "vite",
+    "build": "vite build",
+    "preview": "vite preview"
+  },
+  "dependencies": {
+    "dompurify": "^3.2.2",
+    "github-markdown-css": "^5.8.1",
+    "marked": "^15.0.7",
+    "papaparse": "^5.5.2",
+    "vue": "^3.5.13",
+    "vue-router": "^4.4.5"
+  },
+  "devDependencies": {
+    "@vitejs/plugin-vue": "^5.2.1",
+    "vite": "^5.4.11"
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  }
+}