V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
icy20088
V2EX  ›  Python

Python requests 爬虫结果不正确

  •  
  •   icy20088 · 2016-09-05 18:27:15 +08:00 · 2146 次点击
    这是一个创建于 3008 天前的主题,其中的信息可能已经有所发展或是发生改变。
    还是之前那个问题哈, 我改成了 POST, 结果还是报错, 麻烦大虾们帮忙看看.

    import requests
    from bs4 import BeautifulSoup

    myPost={"version":"1.1","method":"PmData.getDataSet","params":[[{"operation":"fetchpm","filter_starttime":"","filter_endtime":"","filter_extendtime":"","base_time":"","select_counters":"auto","filter_string":"nodename='BB1_C050E_C00323OD_BALOK'","prefix_class":0,"select_time":"total","base_object":"object","select_object":"nodename"},{"operation":"evalformula","formulasource":"all","formulafile":"etijian/etijian_New_BB","formulas":["C01_Data_CSSR","C02_RRC_Setup_Success_Rate","C03_RRC_Connection_Setup_SR_Service","C04_ERAB_Call_Setup_Success_Rate","C05_CSFB_Setup_Success_Rate","C06_eRAB_Drop_Call_Rate","C07_DL_Packet_Loss_Rate","C08_UL_Packet_Loss_Rate","C09_Intra_LTE_Handover_Success_Rate","C10_Network_Availability_Cell_Level","C11_IRATHO_Success_Rate","C12_VolTe_CSSR"]},{"operation":"output","fields":["C01_Data_CSSR","C02_RRC_Setup_Success_Rate","C03_RRC_Connection_Setup_SR_Service","C04_ERAB_Call_Setup_Success_Rate","C05_CSFB_Setup_Success_Rate","C06_eRAB_Drop_Call_Rate","C07_DL_Packet_Loss_Rate","C08_UL_Packet_Loss_Rate","C09_Intra_LTE_Handover_Success_Rate","C10_Network_Availability_Cell_Level","C11_IRATHO_Success_Rate","C12_VolTe_CSSR"],"top":"","filter":"","sortexpr":"time,object"}],{"repeat_merge":"0","repeat_timeprefix":["TimeSer"],"repeat_timefilter":[{"time_display":"From 2016-08-29 00:00 to 2016-09-05 00:00","time_start":"7d","time_end":"","time_unit":"d","time_extend_filter":""}],"repeat_objectprefix":[],"repeat_objectfilter":[],"repeat_objectexpand":"0"}],"itkdebug":{"view":"/itk/web/views/itk_standard/pm_dynaview.xml","client":"itk-web-client-3.0.5-1"}}

    myHder1={
    "Host":"4g.coldice.me",
    "User-Agent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0",
    "Accept":"application/json",
    "Accept-Language": "en-US,en;q=0.5",
    "Accept-Encoding":"gzip, deflate",
    "Content-Type":"application/json; charset=UTF-8",
    "Cookie":"itksession=5b4lj0Ai",
    "Referer":"https://4g.coldice.me/itk/web/",
    "Connection":"keep-alive",
    "Pragma":"no-cache",
    "Cache-Control":"no-cache"
    }
    myRep=requests.post("https://4g.coldice.me/itk/rpc",headers=myHder1,verify=False,data=myPost)

    print(myRep.content)

    回复:
    b'{"jsonrpc":"2.0","error":{"message":"Error parsing JSON: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before \\"method=PmData.getDat...\\") at /opt/ericsson/itk/lib/Itk/WebService/JsonRpcHandler.pm line 271.\\n\\nView name: No view information available\\nClient name: No client information available","code":"-32700"},"id":null}'

    可能是什么原因呢? requst的信息都是按照httpfox 抓到post请求写的.
    8 条回复    2016-09-05 22:39:19 +08:00
    paulw54jrn
        1
    paulw54jrn  
       2016-09-05 18:36:07 +08:00
    这个缩进让别人很难做啊..

    网页也打不开, 被逆向墙了?
    binux
        2
    binux  
       2016-09-05 18:48:21 +08:00
    myRep=requests.post("https://4g.coldice.me/itk/rpc",headers=myHder1,verify=False,json=myPost)
    aeshfawre
        3
    aeshfawre  
       2016-09-05 18:50:11 +08:00
    楼上正解。
    matrix67
        4
    matrix67  
       2016-09-05 18:53:53 +08:00 via Android
    json 没 dump
    icy20088
        5
    icy20088  
    OP
       2016-09-05 19:14:10 +08:00
    @binux 哈哈,多谢了! 终于出来了, 你是咋一样看出 data 是 json 格式的呀!
    binux
        6
    binux  
       2016-09-05 19:17:55 +08:00
    @icy20088
    Error parsing JSON: malformed JSON string
    at character offset 0 (before \\"method=PmData.getDat...\\")
    Chyroc
        7
    Chyroc  
       2016-09-05 22:11:29 +08:00
    @binux 请问 data 和 json 有什么区别。
    我做的话,都是构造 post 的字典 data ,然后 requests.post(url, data),这是你说的哪一种情况
    aeshfawre
        8
    aeshfawre  
       2016-09-05 22:39:19 +08:00
    @Chyroc 区别在于你 post 给服务器的数据格式不同,
    data 是这样 user=aaa&pass=ffff
    json 是这样{"user":"aaa","pass":"fff"}
    服务器那边解析是按他写前端的时候约定好的,后端按哪种格式解析你 post 过来的数据,你提交的数据如果和约定不同,那会解析失败的。
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   2585 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 31ms · UTC 10:12 · PVG 18:12 · LAX 02:12 · JFK 05:12
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.