RYDE-WORK/MediaCrawler
1
0
Fork 0
mirror of https://github.com/RYDE-WORK/MediaCrawler.git synced 2026-01-19 13:33:27 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 7ed6621933
         Merge pull request #603 from Bowenwin/fix_words main 程序员阿江(Relakkes) 2025-05-19 23:16:12 +08:00
  • 703a6e84cb fix_words Bowenwin 2025-05-19 20:07:20 +08:00
  • 144b8bec6a fix_words Bowenwin 2025-05-19 20:04:00 +08:00
  • 654260cbce docs: update README.md Relakkes 2025-05-13 18:42:58 +08:00
  • 79a9824f6a fix: modify dy schema Relakkes 2025-04-30 16:47:13 +08:00
  • 67d31bf42a fix: dy update fp params Relakkes 2025-04-30 13:26:22 +08:00
  • 2a41b684ad
         Merge pull request #590 from 2513502304/main 程序员阿江(Relakkes) 2025-04-20 14:14:55 +08:00
  • af5a393a7a
         Update core.py，删除了其它代码贡献者所添加的try-catch语句，该段try-catch语句将会影响其代码的最终逻辑并令其失效，使其仅能爬取当天一天数据而无法跳转到下一天（原先的逻辑就是try-catch捕获异常从而进入下一天，不要再向该语句中添加捕获异常操作或者finally语句！） 翟持江 2025-04-19 04:34:24 +08:00
  • b675547aab
         Update __init__.py，为bilibili的视频信息、up主信息、评论信息添加额外字段 翟持江 2025-04-19 02:29:22 +08:00
  • ec97001451
         Update tables.sql 翟持江 2025-04-19 02:22:22 +08:00
  • 9935a07279
         Add files via upload 翟持江 2025-04-19 02:18:52 +08:00
  • cb2ae6cbab docs: add sponsor Relakkes 2025-04-16 16:49:49 +08:00
  • 0d715a9f32 fix: bili qrcode login fix Relakkes 2025-04-08 21:11:40 +08:00
  • 660fd18a95 fix: dy login fix Relakkes 2025-04-08 20:58:04 +08:00
  • afbd4ec1bf
         Merge pull request #572 from crpa33/main 程序员阿江(Relakkes) 2025-04-02 13:34:41 +08:00
  • 274d64aefc
         处理xhs意外的评论信息为空的情况 crpa33 2025-04-02 11:59:27 +08:00
  • a39b571d27
         输出到日志-处理视频搜索页任务列表构造的错误 crpa33 2025-04-02 11:57:28 +08:00
  • 413d91a520
         输出到日志-author被封禁或存在错误 crpa33 2025-04-02 11:52:36 +08:00
  • eaf14721f8
         输出到日志-NoneType导致的推导式错误 crpa33 2025-04-02 11:48:36 +08:00
  • 2c4af2337e
         douyin搜索页为空跳下一关键词 crpa33 2025-03-27 23:32:21 +08:00
  • 3c72fc48b0
         保护author为None但未被识别的情况 crpa33 2025-03-27 23:22:47 +08:00
  • 6b6e2b8ba0
         修复NoneType导致的推导式错误 crpa33 2025-03-27 23:18:01 +08:00
  • dfddfa7fdc docs: update README.md Relakkes 2025-03-23 20:35:11 +08:00
  • daaea7155b feat: add uv tool to manage project Relakkes 2025-03-23 18:13:13 +08:00
  • 8030d2a02f docs: removed sponsor Relakkes 2025-03-13 15:07:54 +08:00
  • 061d1c15e2 feat: kuaishou search params update Relakkes 2025-03-11 23:42:34 +08:00
  • f2cf864c27 fix: zhihu article url error #564 Relakkes 2025-03-03 18:18:41 +08:00
  • b43d6b7b91 chore: update config Relakkes 2025-02-12 10:58:48 +08:00
  • 66a7ab1db8 refactor: bibi default to get without time data Relakkes 2025-02-12 10:58:15 +08:00
  • 678ce1bfac fix: bilibili bugfix Relakkes 2025-02-10 17:13:37 +08:00
  • 457205efd8 docs: add sponsor Relakkes 2025-02-08 15:28:18 +08:00
  • 38f2b36bf5
         Merge pull request #542 from 2513502304/main 程序员阿江(Relakkes) 2025-01-20 19:30:10 +08:00
  • 0364b23b5b
         Update core.py，为爬取类型为detail和creator的任务，添加了和search任务一样的，用于转存up主信息的bilibili_store.update_up_info的函数调用 翟持江 2025-01-19 19:55:18 +08:00
  • 4b63ea68ec
         Merge pull request #538 from 2513502304/main 程序员阿江(Relakkes) 2025-01-17 19:43:57 +08:00
  • 2d93ec5a82
         Update core.py，更改了错误的缩进 翟持江 2025-01-15 18:33:12 +08:00
  • 8741952cb5
         Update requirements.txt，添加了pandas模块，datetime为Python标准库模块无需添加 翟持江 2025-01-15 18:27:40 +08:00
  • d2ecd3b11d
         Update client.py，将search_video_by_keyword中post_data错误的请求参数进行更新 翟持江 2025-01-15 18:21:03 +08:00
  • f2b41b573b
         Update core.py，以实现按照 START_DAY 至 END_DAY ，每一天进行筛选，这样能够突破 1000 条视频的限制，最大程度爬取该关键词下的所有视频 翟持江 2025-01-15 18:18:36 +08:00
  • bf87821d4b
         为 base_config.py 添加了是否开启按每一天进行爬取的选项 翟持江 2025-01-15 18:06:16 +08:00
  • 30d0e733d5 feat: douyin adds comment images Relakkes 2025-01-15 14:49:55 +08:00
  • a8e796e690
         Merge pull request #536 from 2513502304/main 程序员阿江(Relakkes) 2025-01-13 13:58:15 +08:00
  • 0118621a79
         将微博评论爬取函数get_note_all_comments的max_id_type便为可变请求参数 翟持江 2025-01-10 19:20:01 +08:00
  • 17b40b8bb4
         Delete client.py 翟持江 2025-01-10 19:13:03 +08:00
  • 8967d3721c
         将微博评论爬取函数get_note_all_comments的max_id_type便为可变请求参数 翟持江 2025-01-10 19:08:57 +08:00
  • 5d3e12169e fix: move dirctory Relakkes 2025-01-10 16:53:20 +08:00
  • b89a8e5dda docs: add github issue template Relakkes 2025-01-10 16:51:30 +08:00
  • 792d3bbc34 docs: update README.md Relakkes 2025-01-05 19:02:10 +08:00
  • 5f7c8aea5b docs: 添加11-12月份捐赠者名单 Relakkes 2025-01-02 20:37:28 +08:00
  • fbbead814a fix: 贴吧创作者bug修复 Relakkes 2025-01-02 20:29:05 +08:00
  • 7ce1273386
         Merge pull request #528 from HuiGitH/hui/fix-xhs-gender 程序员阿江(Relakkes) 2024-12-28 21:01:58 +08:00
  • d929ad16ae fix xhs get gender HuiLong 2024-12-28 20:24:37 +08:00
  • 79bf9fc05d chore: add xhs field comment for issue #526 Relakkes 2024-12-26 18:28:23 +08:00
  • 29ab6feded fix: 修复数据库密码中有特殊符号导致db连接不上的bug Relakkes 2024-12-26 18:13:13 +08:00
  • ea5223c708 feat: 知乎支持详情模式 Relakkes 2024-12-26 17:36:33 +08:00
  • dc9116e098
         Merge pull request #513 from liudongkai/feature_xhs 程序员阿江(Relakkes) 2024-12-06 13:01:16 +08:00
  • 33e7ef016d feat: xhs 非代理模式下增加随机等待间隔, db存储模式下增加存储xsec_token字段 liudongkai 2024-12-05 21:10:31 +08:00
  • 9c7e1d499b
         Merge pull request #509 from leantli/feat/xhs_comments_upgrade 程序员阿江(Relakkes) 2024-12-03 18:34:56 +08:00
  • e830ada574 feat: xhs comments add xsec_token leantli 2024-12-03 18:25:21 +08:00
  • 6001fc8a52
         Merge pull request #506 from JianxunRao/main 程序员阿江(Relakkes) 2024-11-29 11:07:58 +08:00
  • f9eedc59b1 fix：微博根据creator爬取note时，爬取评论失败。原因是解析的参数key有误 Trojx 2024-11-29 10:47:40 +08:00
  • 453ea642fb docs: update README.md Relakkes 2024-11-29 10:40:52 +08:00
  • ca9b47ef63 fix: xhs 帖子详情优化 Relakkes 2024-11-27 09:41:24 +08:00
  • 43dffeb2d1 feat: xhs帖子详情获取优化 Relakkes 2024-11-26 13:37:53 +08:00
  • de32d06815 docs: update README.md Relakkes 2024-11-19 12:58:28 +08:00
  • 935a928f90 docs: update README.md Relakkes 2024-11-17 07:00:55 +08:00
  • 8ab4c67443 feat: 抖音支持评论点赞数量 #495 Relakkes 2024-11-16 00:37:48 +08:00
  • c12a63acb2 fix: weibo creator duplicate bug Relakkes 2024-11-13 22:28:54 +08:00
  • f9e7d56d4f chore: remove sponsor Relakkes 2024-11-08 22:27:03 +08:00
  • 3db2edd479 fix: weibo创作者bugfix Relakkes 2024-11-06 16:00:41 +08:00
  • 3567193a98
         Merge pull request #479 from lyx0727/main 程序员阿江(Relakkes) 2024-11-01 23:09:01 +08:00
  • 705b810269 fix: xhs验证码处理，跳转链接修复 lyx0727 2024-11-01 22:08:46 +08:00
  • 1a37df4d5e fix: xhs出现验证码时报错，增加用户手动验证 lyx0727 2024-11-01 18:52:16 +08:00
  • ef4eba121c fix: 兼容windows编码问题 Relakkes 2024-10-28 15:26:49 +08:00
  • 9bf6ea3650 docs: update README.md Relakkes 2024-10-24 18:18:53 +08:00
  • 6a96c00b4b chore: config update & xinqqiu img update Relakkes 2024-10-24 15:35:34 +08:00
  • 5a27ad089c
         Merge pull request #462 from FloRainRJY/xiaohongshu_comment_number_restrict 程序员阿江(Relakkes) 2024-10-24 15:31:13 +08:00
  • 0cd01fbf8c
         Merge pull request #463 from liugangdao/main 程序员阿江(Relakkes) 2024-10-24 15:13:05 +08:00
  • 78c09c4ae1 fix:翻页时search id不变 liugangdao 2024-10-24 13:57:27 +08:00
  • 7e53c4acfc All_platform_comments_restrict unknown 2024-10-23 16:32:02 +08:00
  • fa2bcc4181
         Merge pull request #461 from FloRainRJY/xiaohongshu_comment_number_restrict 程序员阿江(Relakkes) 2024-10-23 13:35:10 +08:00
  • 19269c66fd xiaohongshu_comment_number_restrict unknown 2024-10-22 20:33:10 +08:00
  • 0bb9298d26 docs: 添加捐赠名单 Relakkes 2024-10-21 21:17:28 +08:00
  • dc11d39185 docs: update README.md Relakkes 2024-10-20 01:05:57 +08:00
  • 03e393949a fix: xhs帖子详情问题更新 Relakkes 2024-10-20 00:59:08 +08:00
  • 9fe3e47b0f chore: 增加代码学习声明，严格禁止非法、禁止商业、不当用途 Relakkes 2024-10-20 00:43:25 +08:00
  • ac8537c296
         Update LICENSE 程序员阿江-Relakkes 2024-10-18 03:27:00 +08:00
  • aa0f920369 feat: B站搜索接口增加发布日期筛选 Relakkes 2024-10-17 15:11:25 +08:00
  • fa1932e105 docs: 添加14群交流二维码 Relakkes 2024-10-17 14:54:06 +08:00
  • 96508fb050 docs: 知乎支持创作者主页数据 Relakkes 2024-10-16 21:03:06 +08:00
  • da8f1c62b8 feat: 知乎支持创作者主页数据爬取(回答、文章、视频) Relakkes 2024-10-16 21:02:27 +08:00
  • af9d2d8e84 docs: update Sponsor Relakkes 2024-10-10 11:28:37 +08:00
  • 9046074909 docs: update README.md Relakkes 2024-09-28 12:48:43 +08:00
  • 7b5b099636 feat: update douyin abogus params Relakkes 2024-09-27 14:58:10 +08:00
  • 17e70ac2c1 chore: update ads wrong text Relakkes 2024-09-26 15:41:32 +08:00
  • 340926bcd3 chore: add vitepress doc custom theme Relakkes 2024-09-26 15:38:18 +08:00
  • 4e10e65216 docs: 增加捐赠名单 Relakkes 2024-09-25 10:42:00 +08:00
  • 0fac56ce50 docs: add google analytics Relakkes 2024-09-20 19:18:48 +08:00
  • 52fe281f2f docs: update README.md Relakkes 2024-09-20 15:57:06 +08:00
  • c6a0960f54 docs: 在线文档添加捐赠者名单 && 添加新增捐赠者 Relakkes 2024-09-19 17:25:41 +08:00
  • fa4238e794 docs: fix vitepress config Relakkes 2024-09-19 17:06:50 +08:00