RYDE-WORK/MediaCrawler
1
0
Fork 0
mirror of https://github.com/RYDE-WORK/MediaCrawler.git synced 2026-01-19 13:33:27 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

douyin搜索页为空跳下一关键词

Browse Source 
预计页数没到，空了也跳
This commit is contained in:
crpa33 2025-03-27 23:32:21 +08:00 committed by GitHub
parent 3c72fc48b0
commit 2c4af2337e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
media_platform/douyin/core.py
View File

@ -108,6 +108,9 @@ class DouYinCrawler(AbstractCrawler):
publish_time=PublishTimeType(config.PUBLISH_TIME_TYPE),
search_id=dy_search_id
)
if posts_res.get("data") is None or posts_res.get("data") == []:
utils.logger.info(f"[DouYinCrawler.search] search douyin keyword: {keyword}, page: {page} is empty,{posts_res.get('data')}`")
break
except DataFetchError:
utils.logger.error(f"[DouYinCrawler.search] search douyin keyword: {keyword} failed")
break