Merge pull request #723 from Colin-XKL/dev

merge dev into main

Author: Colin-XKL

.env.example

@@ -15,6 +15,10 @@ FC_DB_SQLITE_PATH=./db
 # Cache Configuration
 FC_REDIS_URI=redis://localhost:6379/
 
+# Outbound HTTP User-Agent defaults
+FC_HTTP_USER_AGENT_FEED="FeedCraft/2.0"
+FC_HTTP_USER_AGENT_HTML="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36"
+
 # LLM Configuration
 FC_LLM_API_BASE=https://api.openai.com/v1
 FC_LLM_API_KEY=sk-your-api-key-here
@@ -28,4 +32,4 @@ FC_PUPPETEER_HTTP_ENDPOINT=http://localhost:3000
 # Frontend Configuration
 FRONTEND_PORT=5173
 FRONTEND_HOST=localhost
-VITE_API_BASE_URL=http://localhost:8080
+VITE_API_BASE_URL=http://localhost:8080

.github/workflows/docker-publish.yml

@@ -99,7 +99,7 @@ jobs:
       # https://github.com/docker/build-push-action
       - name: Build and push Docker image
         id: build-and-push
-        uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
+        uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
         with:
           file: build/Dockerfile
           context: .

.github/workflows/lint.yml

@@ -26,7 +26,7 @@ jobs:
         with:
           version: latest
 
-      - uses: pnpm/action-setup@v3
+      - uses: pnpm/action-setup@v6
         with:
           version: 9
 

README.md

@@ -19,17 +19,17 @@ Doc: [English](https://feed-craft-doc.vercel.app/en) | [简体中文](https://fe
 
 **FeedCraft** is a simple, powerful RSS feed processing tool.  
 It can serve as middleware to handle your RSS feeds; you can use it to extract the main text, perform intelligent translation, generate article summaries through AI, filter articles using natural language, and more.  
-It includes a built-in visual RSS generator (HTML/Curl/Search to RSS) that can turn web pages, API responses, or search results into RSS feeds.
+It includes a built-in visual RSS generator (HTML/JSON/Search to RSS) that can turn web pages, JSON API responses, or search results into RSS feeds.
 
 **FeedCraft** 是一个简单、强大的 RSS 源处理工具.
 他可以作为一个中间件处理你的 RSS 源, 你可以用它来提取正文、智能翻译、通过 AI 生成文章摘要、通过自然语言筛选文章等.
-它内置了可视化 RSS 生成器 (HTML/Curl/Search to RSS)，支持将网页、API (Curl) 或搜索结果转换为 RSS 订阅源。
+它内置了可视化 RSS 生成器 (HTML/JSON/Search to RSS)，支持将网页、JSON API (Curl) 或搜索结果转换为 RSS 订阅源。
 
 ## 核心特性
 
 - 开源可自部署. 可以作为中间件与现有的任何 RSS 阅读器协同使用
 - AI Power, 可以接入 Open AI 接口兼容的 LLM 对 RSS 进行处理, 可自定义 prompt
-- **HTML/Curl/Search to RSS**: 内置可视化 RSS 生成器，支持将网页、API (Curl) 或搜索结果转换为 RSS 订阅源
+- **HTML/JSON/Search to RSS**: 内置可视化 RSS 生成器，支持将网页、JSON API (支持 Curl 语句导入) 或搜索结果转换为 RSS 订阅源
 - 支持**便携模式**(portable mode, 即用即走, 只需要在原 RSS 地址前面加个前缀即可), 和**高级模式**(dock mode,
   在后台页面自定义 RSS 地址和各类深度加工参数)
 
@@ -87,7 +87,6 @@ https://feed-craft.colinx.one
 控制台默认账号`admin`, 密码 `adminadmin`, 登陆后请尽快修改默认密码
 
 ```yaml
-version: "3"
 services:
   app.feed-craft:
     image: ghcr.io/colin-xkl/feed-craft
@@ -129,6 +128,7 @@ services:
       FC_LLM_API_MODEL: gemini-pro/chatgpt-3.5/... # 默认使用的模型
       FC_LLM_API_TYPE: openai # openai 或 ollama
       FC_DEFAULT_TARGET_LANG: zh-CN # (Optional) LLM 处理任务的默认目标语言
+      LOG_LEVEL: info # (Optional) 日志级别，如 info, debug, trace
   service.redis:
     image: redis:6-alpine
     container_name: feedcraft_redis

doc-site/src/content/docs/en/guides/advanced/customization.md

@@ -65,12 +65,15 @@ You can configure FeedCraft using environment variables in `docker-compose.yml`.
 
 - **FC_PUPPETEER_HTTP_ENDPOINT**: Address of the Browserless/Chrome instance. Required for `fulltext-plus`.
 - **FC_REDIS_URI**: Redis connection address. Used for caching to speed up processing and reduce AI token consumption.
+- **FC_HTTP_USER_AGENT_FEED**: (Optional) Default `User-Agent` for feed-style outbound requests, such as fetching RSS/XML resources. Search provider requests are temporarily grouped into this same rule.
+- **FC_HTTP_USER_AGENT_HTML**: (Optional) Default `User-Agent` for HTML page fetches, such as fulltext extraction and the HTML-to-RSS tooling. **Note:** If the value contains spaces or parentheses, it must be enclosed in quotes.
 - **FC_LLM_API_KEY**: API Key for OpenAI or compatible services (like DeepSeek, Gemini, etc.).
 - **FC_LLM_API_MODEL**: Default model to use (e.g., `gemini-pro`, `gpt-3.5-turbo`). **Multiple Models Support:** You can provide a comma-separated list of models (e.g., `gpt-3.5-turbo,gpt-4`). FeedCraft will randomly select a model for each request and automatically retry with others if a call fails.
 - **FC_LLM_API_BASE**: API endpoint address. For OpenAI-compatible APIs, usually ends with `/v1`.
 - **FC_LLM_API_TYPE**: (Optional) `openai` (default) or `ollama`.
 - **FC_LLM_MAX_CONCURRENCY**: (Optional) Global maximum concurrency for LLM requests (default: `3`). Limits concurrent API calls to prevent rate limits.
 - **FC_DOMAIN_MAX_CONCURRENCY**: (Optional) Maximum concurrent requests per target domain during web scraping like fulltext extraction (default: `3`). Prevents overwhelming target servers.
+- **LOG_LEVEL**: (Optional) Log level for the backend application (e.g., `info`, `debug`, `trace`). Overrides the default level set by `ENV`.
 
 ### External Services
 

doc-site/src/content/docs/en/guides/advanced/html-to-rss.md

@@ -11,7 +11,7 @@ sidebar:
 FeedCraft includes a visual **HTML to RSS** tool that allows you to generate selectors for creating RSS feeds from websites that don't provide them natively.
 
 :::note
-This tool is designed for HTML pages. If you need to process a JSON API, use the [CURL to RSS](/en/guides/advanced/curl-to-rss/) instead.
+This tool is designed for HTML pages. If you need to process a JSON API, use the [JSON to RSS](/en/guides/advanced/json-to-rss/) instead.
 :::
 
 ## Overview

…t/docs/en/guides/advanced/curl-to-rss.md …t/docs/en/guides/advanced/json-to-rss.mddoc-site/src/content/docs/en/guides/advanced/curl-to-rss.md renamed to doc-site/src/content/docs/en/guides/advanced/json-to-rss.md doc-site/src/content/docs/en/guides/advanced/curl-to-rss.md renamed to doc-site/src/content/docs/en/guides/advanced/json-to-rss.md

@@ -1,5 +1,5 @@
 ---
-title: CURL to RSS
+title: JSON to RSS
 description: Convert any JSON API response into an RSS feed with jq selectors and optional templates.
 sidebar:
   order: 3
@@ -8,11 +8,11 @@ sidebar:
     variant: success
 ---
 
-FeedCraft includes a **CURL to RSS** tool that allows you to fetch data from JSON APIs, extract fields with `jq`, and optionally post-process them with templates before generating an RSS feed.
+FeedCraft includes a **JSON to RSS** tool that allows you to fetch data from JSON APIs, extract fields with `jq`, and optionally post-process them with templates before generating an RSS feed.
 
 ## Overview
 
-The CURL to RSS tool helps you:
+The JSON to RSS tool helps you:
 
 1.  **Fetch** JSON data from an API endpoint (supporting custom headers and methods).
 2.  **Parse** the JSON structure using `jq` syntax, then optionally use templates to build the final RSS fields.
@@ -21,13 +21,13 @@ The CURL to RSS tool helps you:
 
 ## How to use
 
-Navigate to **Worktable > Curl to RSS** in the admin dashboard.
+Navigate to **Worktable > JSON to RSS** in the admin dashboard.
 
 ### Step 1: Request Configuration
 
 You need to define how to fetch the JSON data.
 
-- **Import from Curl**: You can paste a `curl` command to automatically populate the URL, method, headers, and body. This is useful if you copy the request from your browser's Developer Tools.
+- **Import from cURL**: You can paste a `curl` command to automatically populate the URL, method, headers, and body. This is useful if you copy the request from your browser's Developer Tools.
 - **Method**: Select `GET` or `POST`.
 - **URL**: The API endpoint URL.
 - **Headers**: Add any necessary headers (e.g., `Authorization`, `Content-Type`).
@@ -50,6 +50,28 @@ The tool uses **[jq](https://jqlang.github.io/jq/)** syntax for querying JSON, a
 - **Date Selector**: (Optional) Path to the publication date.
 - **Content Selector**: (Optional) Path to the full content or summary.
 
+#### Using Templates (Optional)
+
+You can use [Go Templates](https://pkg.go.dev/text/template) to further process extracted values.
+
+**Available Variables:**
+
+- `.Fields`: The parsed field values (e.g., `.Fields.Title`, `.Fields.Link`, `.Fields.Date`, `.Fields.Description`).
+- `.Item`: The raw JSON item object (e.g., `.Item.id`, `.Item.author.name`).
+
+**Built-in Functions:**
+
+- `trimSpace`: Removes leading and trailing whitespace.
+- `trim`: Removes specified leading and trailing characters.
+- `default`: Provides a fallback value if the field is empty.
+
+**Examples:**
+
+- **Clean up whitespace in title**: `{{ .Fields.Title | trimSpace }}`
+- **Build absolute URLs**: `https://example.com/article/{{ .Item.id }}`
+- **Remove specific prefixes**: `{{ .Fields.Description | trim "Prefix: " }}`
+- **Fallback values**: `{{ default .Fields.Description "No summary available" }}`
+
 Click **Run Preview** to verify your selectors, then click **Next Step**.
 
 ### Step 3: Feed Metadata

doc-site/src/content/docs/en/guides/advanced/tools.md

@@ -48,6 +48,21 @@ The **Craft Dependencies** (System Health) tool visualizes the internal relation
 If you encounter errors like "Craft not found", use this tool to trace the broken link in your configuration.
 :::
 
+## System Runtime
+
+The **System Runtime** (Observability) tool provides a comprehensive dashboard for monitoring the health and execution status of your resources.
+
+- **Usage**:
+  1. Navigate to **Tools > System Runtime**.
+- **Features**:
+  - **Resource Health**: View the current status (Healthy, Degraded, Paused) of Recipes and other components, including consecutive failures.
+  - **Execution Logs**: Track detailed execution history, success rates, and specific error types (e.g., Timeout, Network, Parse) across all runs.
+  - **System Notifications**: Review automated alerts regarding resource state transitions (e.g., when a Recipe becomes degraded). You can also subscribe to these alerts via the built-in RSS feed at `/system/notifications/rss`.
+
+:::tip
+If a Recipe fails repeatedly and becomes "Paused", you can use the System Runtime dashboard to manually "Resume" it after fixing the underlying issue.
+:::
+
 ## Debug Tools
 
 ### LLM Debug

doc-site/src/content/docs/en/guides/start/concepts.md

@@ -57,6 +57,11 @@ You can configure processors for a Topic Feed to automatically handle the combin
 - **Limit**: Keeps only the most recent items.
 
 **Managing Topic Feeds:**
+
+:::caution
+The Topic Feed feature is currently under active development. Its UI entry point in the admin dashboard has been temporarily hidden until the feature is stable and ready for general use.
+:::
+
 Navigate to **Worktable > Topic Feed** to create and manage topics.
 
 - **Create**: Define a title, add multiple input URIs (e.g. `feedcraft://recipe/my-recipe` or external RSS URLs), and set your aggregator config.

doc-site/src/content/docs/zh-tw/guides/advanced/customization.md

@@ -65,12 +65,15 @@ sidebar:
 
 - **FC_PUPPETEER_HTTP_ENDPOINT**: Browserless/Chrome 實例的地址。`fulltext-plus` 功能必須。
 - **FC_REDIS_URI**: Redis 連線地址。用於快取，加快處理速度並減少 AI Token 消耗。
+- **FC_HTTP_USER_AGENT_FEED**: （可選）feed 類外部請求的預設 `User-Agent`，例如抓取 RSS/XML 資源時使用。搜尋提供方請求目前也暫時歸入這一規則。
+- **FC_HTTP_USER_AGENT_HTML**: （可選）HTML 頁面抓取的預設 `User-Agent`，例如全文提取和 HTML 轉 RSS 工具使用。**注意：** 如果該值包含空格或括號，必須使用引號括起來。
 - **FC_LLM_API_KEY**: OpenAI 或相容服務（如 DeepSeek, Gemini 等）的 API Key。
 - **FC_LLM_API_MODEL**: 預設使用的模型（如 `gemini-pro`, `gpt-3.5-turbo`）。**支援多個模型：** 你可以提供一個逗號分隔的模型列表（例如 `gpt-3.5-turbo,gpt-4`）。FeedCraft 會為每個請求隨機選擇一個模型，如果調用失敗，會自動重試列表中的其他模型。
 - **FC_LLM_API_BASE**: API 介面地址。如果是相容 OpenAI 的 API，通常以 `/v1` 結尾。
 - **FC_LLM_API_TYPE**: (可選) `openai` (預設) 或 `ollama`.
 - **FC_LLM_MAX_CONCURRENCY**: (可選) 全局最大 LLM 併發請求數（預設: `3`）。用於限制併發請求數量以防止觸發 API 速率限制。
 - **FC_DOMAIN_MAX_CONCURRENCY**: (可選) 網頁抓取（如全文提取）時每個目標域名的最大併發數（預設: `3`）。防止抓取目標伺服器負載過高。
+- **LOG_LEVEL**: (可選) 後端應用的日誌級別 (例如 `info`, `debug`, `trace`)。覆蓋 `ENV` 設定的預設級別。
 
 ### 外部服務