解决requests的logging日志提示问题
python下哪个httpclient最好用，我想大家首选都是requets 。 这次使用requests调用nlp，分词http服务的时候，会提示大量的requests建立连接的日志…
后来逐步排除发现当你配置logging的时候，requests才会发出各种各样的日志.
2016-03-25 10:10:03,014 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,029 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,087 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,103 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,158 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,174 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,233 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,250 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,290 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,305 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,364 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,382 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,437 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,453 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,506 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,521 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,573 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc 2016-03-25 10:10:03,587 - 29659 - INFO: Starting new HTTP connection (1): xiaorui.cc
该文章写的有些乱，欢迎来喷 ! 另外文章后续不断更新中，请到原文地址查看更新。http://xiaorui.cc/?p=3047
解决的方法:
最一开始使用的方法，把logging.basicConfig的level配置为WARNING,这样requests info级别的日志就打不出来了，但这样带来的问题是，你的业务逻辑logging也需要改级别.
后来看了requests的代码，发现其实可以在logging.getLogger配置requests的日志等级.
import logging
logging.basicConfig(filename = os.path.join(os.getcwd(), 'rui.log'),\
level = logging.INFO, filemode = 'a', format = '%(asctime)s - %(process)s - %(levelname)s:\
%(message)s')
logging.getLogger("requests").setLevel(logging.WARNING)
下面是requests官方文档中说明的各种阶段的日志等级.
INFO: New connections (HTTP or HTTPS) INFO: Dropped connections INFO: Redirects WARN: Connection pool full (if this happens often increase the connection pool size) WARN: Retrying the connection DEBUG: Connection details: method, path, HTTP version, status code and response length
我们从requests源码里过滤一些日志级别. 大家可以对比参考下.
requests/packages/urllib3/connectionpool.py:206: log.info("Starting new HTTP connection (%d): %s" %
requests/packages/urllib3/connectionpool.py:242: log.info("Resetting dropped connection: %s" % self.host)
requests/packages/urllib3/connectionpool.py:274: log.warning(
requests/packages/urllib3/connectionpool.py:385: log.debug("\"%s %s %s\" %s %s" % (method, url, http_version,
requests/packages/urllib3/connectionpool.py:392: log.warning(
requests/packages/urllib3/connectionpool.py:624: log.warning("Retrying (%r) after connection "
requests/packages/urllib3/connectionpool.py:647: log.info("Redirecting %s -> %s" % (url, redirect_location))
requests/packages/urllib3/connectionpool.py:659: log.info("Forced retry: %s" % url)
requests/packages/urllib3/connectionpool.py:757: log.info("Starting new HTTPS connection (%d): %s"
requests/packages/urllib3/contrib/appengine.py:147: log.info("Forced retry: %s" % url)
requests/packages/urllib3/contrib/ntlmpool.py:46: log.debug('Starting NTLM HTTPS connection no. %d: https://%s%s' %
requests/packages/urllib3/contrib/ntlmpool.py:59: log.debug('Request headers: %s' % headers)
requests/packages/urllib3/contrib/ntlmpool.py:63: log.debug('Response status: %s %s' % (res.status, res.reason))
requests/packages/urllib3/contrib/ntlmpool.py:64: log.debug('Response headers: %s' % reshdr)
requests/packages/urllib3/contrib/ntlmpool.py:65: log.debug('Response data: %s [...]' % res.read(100))
requests/packages/urllib3/contrib/ntlmpool.py:90: log.debug('Request headers: %s' % headers)
requests/packages/urllib3/contrib/ntlmpool.py:93: log.debug('Response status: %s %s' % (res.status, res.reason))
requests/packages/urllib3/contrib/ntlmpool.py:94: log.debug('Response headers: %s' % dict(res.getheaders()))
requests/packages/urllib3/contrib/ntlmpool.py:95: log.debug('Response data: %s [...]' % res.read()[:100])
requests/packages/urllib3/contrib/ntlmpool.py:104: log.debug('Connection established')
requests/packages/urllib3/exceptions.py:198: "Raised by assert_header_parsing, but we convert it to a log.warning statement."
requests/packages/urllib3/poolmanager.py:189: log.info("Redirecting %s -> %s" % (url, redirect_location))
requests/packages/urllib3/util/retry.py:156: log.debug("Converted retries value: %r -> %r" % (retries, new_retries))
requests/packages/urllib3/util/retry.py:275: log.debug("Incremented Retry for (url='%s'): %r" % (url, new_retry))
通过上面的方法学到了一招，很多时候我们python项目中模块是自己可控的，创立一个日志对象，然后所有的模块都import引入。 但是如果开源的项目怎么搞?
可以使用 logger = logging.getLogger(__name__) ,对于调用方可以使用getLogger获取和配置对象的日志. 比如, logging.getLogger(“test”).setLevel(logging.DEBUG)
~/github/requests/requests/packages/urllib3/connectionpool.py
log = logging.getLogger(__name__)
END.
