解决requests的logging日志提示问题

解决requests的logging日志提示问题

python下哪个httpclient最好用,我想大家首选都是requets 。 这次使用requests调用nlp,分词http服务的时候,会提示大量的requests建立连接的日志… 

后来逐步排除发现当你配置logging的时候,requests才会发出各种各样的日志.  

2016-03-25 10:10:03,014 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,029 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,087 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,103 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,158 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,174 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,233 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,250 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,290 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,305 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,364 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,382 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,437 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,453 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,506 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,521 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,573 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc
2016-03-25 10:10:03,587 - 29659 - INFO:    Starting new HTTP connection (1): xiaorui.cc

该文章写的有些乱,欢迎来喷 ! 另外文章后续不断更新中,请到原文地址查看更新。http://xiaorui.cc/?p=3047

解决的方法:

最一开始使用的方法,把logging.basicConfig的level配置为WARNING,这样requests info级别的日志就打不出来了,但这样带来的问题是,你的业务逻辑logging也需要改级别.

后来看了requests的代码,发现其实可以在logging.getLogger配置requests的日志等级.

import logging

logging.basicConfig(filename = os.path.join(os.getcwd(), 'rui.log'),\
    level = logging.INFO, filemode = 'a', format = '%(asctime)s - %(process)s - %(levelname)s:\
    %(message)s')
logging.getLogger("requests").setLevel(logging.WARNING)

下面是requests官方文档中说明的各种阶段的日志等级.

INFO: New connections (HTTP or HTTPS)
INFO: Dropped connections
INFO: Redirects
WARN: Connection pool full (if this happens often increase the connection pool size)
WARN: Retrying the connection
DEBUG: Connection details: method, path, HTTP version, status code and response length

我们从requests源码里过滤一些日志级别. 大家可以对比参考下.

requests/packages/urllib3/connectionpool.py:206:        log.info("Starting new HTTP connection (%d): %s" %
requests/packages/urllib3/connectionpool.py:242:            log.info("Resetting dropped connection: %s" % self.host)
requests/packages/urllib3/connectionpool.py:274:            log.warning(
requests/packages/urllib3/connectionpool.py:385:        log.debug("\"%s %s %s\" %s %s" % (method, url, http_version,
requests/packages/urllib3/connectionpool.py:392:            log.warning(
requests/packages/urllib3/connectionpool.py:624:            log.warning("Retrying (%r) after connection "
requests/packages/urllib3/connectionpool.py:647:            log.info("Redirecting %s -> %s" % (url, redirect_location))
requests/packages/urllib3/connectionpool.py:659:            log.info("Forced retry: %s" % url)
requests/packages/urllib3/connectionpool.py:757:        log.info("Starting new HTTPS connection (%d): %s"
requests/packages/urllib3/contrib/appengine.py:147:            log.info("Forced retry: %s" % url)
requests/packages/urllib3/contrib/ntlmpool.py:46:        log.debug('Starting NTLM HTTPS connection no. %d: https://%s%s' %
requests/packages/urllib3/contrib/ntlmpool.py:59:        log.debug('Request headers: %s' % headers)
requests/packages/urllib3/contrib/ntlmpool.py:63:        log.debug('Response status: %s %s' % (res.status, res.reason))
requests/packages/urllib3/contrib/ntlmpool.py:64:        log.debug('Response headers: %s' % reshdr)
requests/packages/urllib3/contrib/ntlmpool.py:65:        log.debug('Response data: %s [...]' % res.read(100))
requests/packages/urllib3/contrib/ntlmpool.py:90:        log.debug('Request headers: %s' % headers)
requests/packages/urllib3/contrib/ntlmpool.py:93:        log.debug('Response status: %s %s' % (res.status, res.reason))
requests/packages/urllib3/contrib/ntlmpool.py:94:        log.debug('Response headers: %s' % dict(res.getheaders()))
requests/packages/urllib3/contrib/ntlmpool.py:95:        log.debug('Response data: %s [...]' % res.read()[:100])
requests/packages/urllib3/contrib/ntlmpool.py:104:        log.debug('Connection established')
requests/packages/urllib3/exceptions.py:198:    "Raised by assert_header_parsing, but we convert it to a log.warning statement."
requests/packages/urllib3/poolmanager.py:189:        log.info("Redirecting %s -> %s" % (url, redirect_location))
requests/packages/urllib3/util/retry.py:156:        log.debug("Converted retries value: %r -> %r" % (retries, new_retries))
requests/packages/urllib3/util/retry.py:275:        log.debug("Incremented Retry for (url='%s'): %r" % (url, new_retry))

通过上面的方法学到了一招,很多时候我们python项目中模块是自己可控的,创立一个日志对象,然后所有的模块都import引入。  但是如果开源的项目怎么搞? 
可以使用 logger = logging.getLogger(__name__) ,对于调用方可以使用getLogger获取和配置对象的日志.  比如, logging.getLogger(“test”).setLevel(logging.DEBUG)

~/github/requests/requests/packages/urllib3/connectionpool.py

log = logging.getLogger(__name__)

END.


大家觉得文章对你有些作用! 如果想赏钱,可以用微信扫描下面的二维码,感谢!
另外再次标注博客原地址  xiaorui.cc

发表评论

邮箱地址不会被公开。 必填项已用*标注