Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions lib/routes/farsnews/namespace.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import type { Namespace } from '@/types';

export const namespace: Namespace = {
name: 'Fars News',
url: 'farsnews.ir',
lang: 'fa',
};
84 changes: 84 additions & 0 deletions lib/routes/farsnews/showcase.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import { load } from 'cheerio';

Check failure

Code scanning / oxlint

simple-import-sort(imports) Error

Run autofix to sort these imports!
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
import type { Route } from '@/types';
import cache from '@/utils/cache';
import got from '@/utils/got';
import { parseDate } from '@/utils/parse-date';

export const route: Route = {
path: '/showcase/:category?',
categories: ['traditional-media'],
example: '/farsnews/showcase',
parameters: { category: 'Category slug from farsnews.ir/showcase URL' },
features: {
requireConfig: false,
requirePuppeteer: false,
antiCrawler: true,
supportBT: false,
supportPodcast: false,
supportScihub: false,
},
radar: [{
source: ['farsnews.ir/showcase'],
target: '/showcase',
}],
name: 'Showcase',
maintainers: [],
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing maintainer GitHub ID.

handler,
description: 'Fars News showcase articles. Persian news agency.',
};

async function handler(ctx) {
const category = ctx.req.param('category') ?? '';
const baseUrl = 'https://farsnews.ir';
const currentUrl = category ? `${baseUrl}/showcase/${category}` : `${baseUrl}/showcase`;

const response = await got({ method: 'get', url: currentUrl });
const $ = load(response.data);

const items = $('a[href^="/"]')
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a textbook example of the first anti-pattern listed.

.toArray()
.map((item) => {
item = $(item);
const href = item.attr('href');
const title = item.find('h2, h3').first().text().trim() || item.text().trim();

if (!href || !title || !/^\/[^/]+\/\d+\//.test(href)) {
return null;
}

return {
title,
link: `${baseUrl}${href}`,
};
})
.filter((item) => item !== null)
.filter((item, index, self) => self.findIndex((i) => i.link === item.link) === index);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the embeded data from window.__hydrationDataString instead.


const processedItems = await Promise.all(
items.map((item) =>
cache.tryGet(item.link, async () => {
try {
const detailResponse = await got({ method: 'get', url: item.link });
const detail$ = load(detailResponse.data);

const desc = detail$('meta[name="description"]').attr('content') || '';
item.description = desc;

const timeText = detail$('time').attr('datetime') || detail$('.text-gray-400').first().text();
if (timeText) {
item.pubDate = parseDate(timeText);
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the embeded data from window.__hydrationDataString instead.

} catch {
// Silently continue if detail fetch fails
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no error emitted from all 127 requests. Provide the URL that throws an error which requires using try-catch.

return item;
})
)
);

return {
title: 'Fars News - Showcase',
link: currentUrl,
item: processedItems,
};
}