-
Notifications
You must be signed in to change notification settings - Fork 9.8k
feat: add GXNAS blog route #21982
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Franklittleboy
wants to merge
15
commits into
DIYgod:master
Choose a base branch
from
Franklittleboy:feat/gxnas-route
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+142
−39
Open
feat: add GXNAS blog route #21982
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
6800f03
feat: add GXNAS blog route
Franklittleboy f461e2e
feat: add GXNAS blog route - index
Franklittleboy dcf1c4e
fix: resolve oxlint errors and mark antiCrawler feature
Franklittleboy 6ae8224
fix(gxnas): 使用 FlareSolverr 绕过 Cloudflare JS Challenge
Franklittleboy b950d62
fix: 改为使用 franksun123/rsshub 镜像
Franklittleboy bf5880d
chore: 简化 docker-compose,去掉 browserless 和 redis
Franklittleboy f84d063
fix(gxnas): 使用 FlareSolverr session 复用浏览器实例解 Cloudflare challenge
Franklittleboy 3cc3262
chore: docker-compose 改为 host 网络模式
Franklittleboy 32913b4
fix(gxnas): FlareSolverr 超时改为180秒
Franklittleboy fa64d45
fix(route/gxnas): replace FlareSolverr with built-in Playwright for C…
Franklittleboy 850be73
fix(route/gxnas): sort imports for oxlint simple-import-sort rule
Franklittleboy 700b9dc
fix(route/gxnas): fix simple-import-sort order - type imports after v…
Franklittleboy d17b411
fix(route/gxnas): correct simple-import-sort order per eslint autofix
Franklittleboy e6ec27c
fix(route/gxnas): add blank line between external and internal import…
Franklittleboy 3263c9a
revert(route/gxnas): restore FlareSolverr approach - Playwright fails…
Franklittleboy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,50 +1,22 @@ | ||
| version: '3.8' | ||
| services: | ||
| rsshub: | ||
| # two ways to enable Playwright: | ||
| # * comment out marked lines, then use this image instead: diygod/rsshub:chromium-bundled | ||
| # * (consumes more disk space and memory) leave everything unchanged | ||
| image: diygod/rsshub # or ghcr.io/diygod/rsshub | ||
| image: franksun123/rsshub:latest | ||
| container_name: rsshubfrank | ||
| restart: always | ||
| ports: | ||
| - '1200:1200' | ||
| network_mode: host | ||
| environment: | ||
| NODE_ENV: production | ||
| CACHE_TYPE: redis | ||
| REDIS_URL: 'redis://redis:6379/' | ||
| PLAYWRIGHT_WS_ENDPOINT: 'ws://browserless:3000' # marked | ||
| healthcheck: | ||
| test: ['CMD', 'curl', '-f', 'http://localhost:1200/healthz'] | ||
| interval: 30s | ||
| timeout: 10s | ||
| retries: 3 | ||
| - NODE_ENV=production | ||
| - CACHE_TYPE=redis | ||
| - REDIS_URL=redis://127.0.0.1:6379/0 | ||
| - FLARESOLVERR_URL=http://127.0.0.1:8191/v1 | ||
| depends_on: | ||
| - redis | ||
| - browserless # marked | ||
|
|
||
| browserless: # marked | ||
| image: browserless/chrome # marked | ||
| restart: always # marked | ||
| ulimits: # marked | ||
| core: # marked | ||
| hard: 0 # marked | ||
| soft: 0 # marked | ||
| healthcheck: # marked | ||
| test: ['CMD', 'curl', '-f', 'http://localhost:3000/pressure'] # marked | ||
| interval: 30s # marked | ||
| timeout: 10s # marked | ||
| retries: 3 # marked | ||
|
|
||
| redis: | ||
| image: redis:alpine | ||
| container_name: rsshub-redis | ||
| restart: always | ||
| network_mode: host | ||
| volumes: | ||
| - redis-data:/data | ||
| healthcheck: | ||
| test: ['CMD', 'redis-cli', 'ping'] | ||
| interval: 30s | ||
| timeout: 10s | ||
| retries: 5 | ||
| start_period: 5s | ||
|
|
||
| volumes: | ||
| redis-data: | ||
| - ./redis-data:/data |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| import { load } from 'cheerio'; | ||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
github-advanced-security[bot] marked this conversation as resolved.
Fixed
github-advanced-security[bot] marked this conversation as resolved.
Fixed
|
||
|
|
||
| import type { DataItem, Route } from '@/types'; | ||
| import logger from '@/utils/logger'; | ||
| import ofetch from '@/utils/ofetch'; | ||
| import { parseDate } from '@/utils/parse-date'; | ||
|
|
||
| const FLARESOLVERR_URL = process.env.FLARESOLVERR_URL || 'http://flaresolverr:8191/v1'; | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do not read |
||
| const SESSION_ID = 'gxnas_session'; | ||
|
|
||
| // Session singleton: create once, reuse across requests | ||
| let sessionCreated = false; | ||
|
|
||
| async function ensureSession() { | ||
| if (sessionCreated) { | ||
| return; | ||
| } | ||
| try { | ||
| logger.debug(`Creating FlareSolverr session: ${SESSION_ID}`); | ||
| await ofetch(FLARESOLVERR_URL, { | ||
| method: 'POST', | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| body: { cmd: 'sessions.create', session: SESSION_ID }, | ||
| }); | ||
| sessionCreated = true; | ||
| logger.debug(`FlareSolverr session created: ${SESSION_ID}`); | ||
| } catch (e) { | ||
Check failureCode scanning / oxlint eslint-plugin-unicorn(catch-error-name) Error
The catch parameter "e" should be named "error"
Rename e to error |
||
|
|
||
| // Session might already exist from a previous run | ||
| logger.debug(`Session creation failed (may already exist): ${(e as Error).message}`); | ||
| sessionCreated = true; | ||
| } | ||
| } | ||
|
|
||
| export const route: Route = { | ||
| path: '/', | ||
| categories: ['blog'], | ||
| example: '/gxnas', | ||
| radar: [ | ||
| { | ||
| source: ['wp.gxnas.com/'], | ||
| }, | ||
| ], | ||
| url: 'wp.gxnas.com/', | ||
| name: '最新文章', | ||
| maintainers: ['Franklittleboy'], | ||
| handler, | ||
| description: `:::warning | ||
| 该网站受 Cloudflare 保护,需要配置 FlareSolverr 服务([文档](https://github.com/FlareSolverr/FlareSolverr))。 | ||
| 环境变量 \`FLARESOLVERR_URL\` 默认为 \`http://flaresolverr:8191/v1\`。 | ||
| :::`, | ||
| features: { | ||
| requireConfig: false, | ||
| requirePuppeteer: false, | ||
| antiCrawler: true, | ||
| supportBT: false, | ||
| supportPodcast: false, | ||
| supportScihub: false, | ||
| }, | ||
| }; | ||
|
|
||
| async function handler() { | ||
| const rootUrl = 'https://wp.gxnas.com'; | ||
|
|
||
| await ensureSession(); | ||
|
|
||
| logger.debug(`Fetching ${rootUrl} via FlareSolverr session ${SESSION_ID}`); | ||
| const result = await ofetch(FLARESOLVERR_URL, { | ||
| method: 'POST', | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| body: { | ||
| cmd: 'request.get', | ||
| url: rootUrl, | ||
| session: SESSION_ID, | ||
| maxTimeout: 180000, | ||
| }, | ||
| }); | ||
|
|
||
| const html = result?.solution?.response; | ||
| if (!html) { | ||
| throw new Error('FlareSolverr 返回内容为空'); | ||
| } | ||
|
|
||
| const $ = load(html); | ||
|
|
||
| const items = $('.article-panel') | ||
| .toArray() | ||
| .map((el) => { | ||
| const $el = $(el); | ||
| const titleEl = $el.find('.header .title a'); | ||
| const title = titleEl.text().trim(); | ||
| const link = titleEl.attr('href'); | ||
| const categoryEl = $el.find('.header .label'); | ||
| const category = categoryEl.text().trim(); | ||
| const contentEl = $el.find('.content'); | ||
| const description = contentEl.html() || ''; | ||
| const thumbEl = $el.find('.a-thumb img'); | ||
| const image = thumbEl.attr('src'); | ||
|
|
||
| // Date format: 2023年11月17日 | ||
| const dateText = $el.find('.a-meta span').first().text().trim(); | ||
| let pubDate: Date | undefined; | ||
| const m = dateText.match(/(\d{4})年(\d{1,2})月(\d{1,2})日/); | ||
| if (m) { | ||
| pubDate = new Date(Number.parseInt(m[1]), Number.parseInt(m[2]) - 1, Number.parseInt(m[3])); | ||
| } | ||
|
|
||
| return { | ||
| title, | ||
| link, | ||
| description: image ? `<img src="${image}"><br>${description}` : description, | ||
| category, | ||
| pubDate: pubDate && parseDate(pubDate), | ||
| } as DataItem; | ||
| }) | ||
| .filter((item) => item.title && item.link); | ||
|
|
||
| return { | ||
| title: 'GXNAS博客', | ||
| link: rootUrl, | ||
| item: items, | ||
| description: 'GXNAS博客 - NAS博客|NAS社区|NAS交流|NAS技术|群晖教程|软路由', | ||
| }; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| import type { Namespace } from '@/types'; | ||
|
|
||
| export const namespace: Namespace = { | ||
| name: 'GXNAS博客', | ||
| url: 'wp.gxnas.com', | ||
| description: 'GXNAS博客 - NAS技术交流', | ||
| lang: 'zh-CN', | ||
| }; |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not modify the docker compose file.