专栏名称: Dance with GenAI
关于生成式人工智能AIGC的一切
今天看啥  ›  专栏  ›  Dance with GenAI

AI网络爬虫:用GraphQL查询爬取动态网页数据

Dance with GenAI  · 公众号  ·  · 2024-06-05 07:04
    

文章预览

任务:爬取网站www.skillshare.com搜索结果页面数据: 查看网站的请求信息: 请求网址: https://www. skillshare.com/api/grap hql 请求方法: POST 状态代码: 200 OK 远程地址: 127.0.0.1:10809 引荐来源网址政策: strict-origin-when-cross-origin 请求载荷: {"operationName":"GetClassesQuery","variables":{"query":"ChatGPT","where":{"level":["ALL_LEVELS","BEGINNER","INTERMEDIATE","ADVANCED"]},"analyticsTags":["src:browser","src:browser:search","disc_cls_idx_mig","user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36"],"after":"191","first":24},"query":"fragment ClassFields on Class {\n badges {\n type\n __typename\n }\n durationInSeconds\n id\n publishTime\n largeCoverUrl\n sku\n sourceLanguage\n studentCount\n teacher {\n id\n name\n username\n vanityUsername\n __typename\n }\n title\n url\n viewer {\n hasSavedClass\n __typename\n }\n __typename\n}\n\nquery GetClassesQuery($query: S ………………………………

原文地址:访问原文地址
快照地址: 访问文章快照
总结与预览地址:访问总结与预览