Python 协程笔记0 | LSABLOG

首页 » Program » Python » 正文

Python 协程笔记0

0x00 啥是协程(coroutine)

用户态,共享堆,不共享栈,单线程,不需要锁机制。上下文切换比线程效率高,多用于IO密集型。IO耗时就切换到其他协程,极大提高效率,互相协作让控制权。异步IO,高并发。拥有自己的寄存器和栈,可保留状态。需要多进程配合才能运行在多核CPU上。线程越多,协程优势越明显。

 

0x01 为什么用协程

  1. 占用资源相对少(轻量级)
  2. 效率高(子程序切换)
  3. 不需要锁(单线程)
  4. 高性能,高并发

 

0x02 python中怎么用协程

  1. Greenlet

手动切换

  1. Gevent

自动调度,猴子补丁,greenlet模式,基于libev,底层epoll。

 

0x03 gevent简单用例

同步vs异步:

#coding:utf-8

import gevent
import random

def task(number):
    gevent.sleep(random.randint(0,5)*0.001)   #交出控制权
    print('number %s done' % number)


def synchronous():   #同步
    for i in xrange(5):
        task(i)


def asynchronous():   #异步
    c = [gevent.spawn(task, i) for i in xrange(5)]   #spawn创建greenlet协程对象
    gevent.joinall(c)   #等待所有greenlet协程结束才退出


print('Synchronous:')
synchronous()

print '---------------------------'

print('Asynchronous:')
asynchronous()

更有用的例子:

#coding:utf-8

from gevent import monkey; monkey.patch_all()   #猴子补丁,把所有python原生库打上补丁变协作式
import gevent
import urllib2

def getData(url):
    print('GET: %s' % url)
    rsp = urllib2.urlopen(url)
    data = rsp.read()
    print('%d bytes received from %s.' % (len(data), url))

gevent.joinall([
        gevent.spawn(getData, 'https://www.python.org/'),
        gevent.spawn(getData, 'https://www.baidu.com/'),
        gevent.spawn(getData, 'https://github.com/'),
])

最好是需要协作的库才打补丁,避免影响其他功能,如from gevent import socket。

多线程和协程效率对比:
多线程版:

#coding:utf-8


import socket
import time
import Queue
import urllib2
import threading

 
urls = ['http://www.sina.com', 'http://www.gevent.org', 'https://www.python.org','https://www.baidu.com','https://github.com','http://www.cnblogs.cn','http://blog.csdn.net']

q = Queue.Queue()

threads = 3
threadlist = []
tNumber = 0

for url in urls:
    
    q.put(url)

def getData():
    while not q.empty():
        theUrl = q.get()
        print('GET: %s' % theUrl)
            rsp = urllib2.urlopen(theUrl)
            data = rsp.read()
            print('%d bytes received from %s.' % (len(data), theUrl))

start = time.time()
for thread in range(threads):
    
    t = threading.Thread(target=getData)
    t.start()
        
    print '\n********线程' + str(tNumber) + '启动了**********\n'

    tNumber = tNumber + 1
        threadlist.append(t)
for thr in threadlist:
    thr.join()
end = time.time()

print end-start

协程版:

 

#coding:utf-8



from gevent import monkey; monkey.patch_all()
import gevent
import socket
import time
import Queue
import urllib2
 
urls = ['http://www.sina.com', 'http://www.gevent.org', 'https://www.python.org','https://www.baidu.com','https://github.com','http://www.cnblogs.cn','http://blog.csdn.net']

q = Queue.Queue()

for url in urls:
    
    q.put(url)

def getData():
    while not q.empty():
        theUrl = q.get()
        print('GET: %s' % theUrl)
            rsp = urllib2.urlopen(theUrl)
            data = rsp.read()
            print('%d bytes received from %s.' % (len(data), theUrl))

start = time.time()
jobs = [gevent.spawn(getData) for i in range(1,4)]
gevent.joinall(jobs)
end = time.time()

print end-start

经测试,3个协程总比3个线程快1~2秒,而且只是7个url,相信url越多,协程越多,就会比多线程有更加明显的优势。

 

0x03 更多

greenlets

状态,超时,事件,队列,池,子进程

回调

通讯,信号量

遇到有需要的情况再说吧。

 

0x04 参考资料

https://softlns.github.io/2015/11/28/python-gevent/

https://www.liaoxuefeng.com/wiki/001374738125095c955c1e6d8bb493182103fac9270762a000/001407503089986d175822da68d4d6685fbe849a0e0ca35000

python.jobbole.com/84301/

https://liuliqiang.info/gevent-introduction/

www.bjhee.com/gevent.html

www.gevent.org/contents.html

sdiehl.github.io/gevent-tutorial/

brieflyx.me/2015/python-module/python-lib-gevent/

 

 

本文共 1 个回复

  • Kevi 2017/10/30 06:16

    :!: 厉害

Comment