Python 3.5에서 코 루틴과 미래 / 작업의 차이점은 무엇입니까?

Python 3.5에서 코 루틴과 미래 / 작업의 차이점은 무엇입니까?

---

더미 함수가 있다고 가정 해 보겠습니다.

async def foo(arg):
    result = await some_remote_call(arg)
    return result.upper()

차이점은 무엇입니까?

coros = []
for i in range(5):
    coros.append(foo(i))

loop = get_event_loop()
loop.run_until_complete(wait(coros))

과:

from asyncio import ensure_future

futures = []
for i in range(5):
    futures.append(ensure_future(foo(i)))

loop = get_event_loop()
loop.run_until_complete(wait(futures))

참고 :이 예제는 결과를 반환하지만 질문의 초점이 아닙니다. 반환 값 문제는, 사용하는 경우 gather()대신 wait().

반환 값에 관계없이 ensure_future(). wait(coros)그리고 wait(futures)모두는 코 루틴을 실행, 그래서 언제, 왜 코 루틴을 감싸한다 ensure_future?

기본적으로 Python 3.5를 사용하여 비 차단 작업을 실행하는 올바른 방법 (tm)은 async무엇입니까?

추가 크레딧을 위해 통화를 일괄 처리하려면 어떻게해야합니까? 예를 들어 some_remote_call(...)1000 번 전화를 걸어야하지만 1000 개의 동시 연결로 웹 서버 / 데이터베이스 등을 부수고 싶지는 않습니다. 이것은 스레드 또는 프로세스 풀을 사용하여 수행 할 수 있지만이 작업을 수행하는 방법이 asyncio있습니까?

---

코 루틴은 값을 산출하고 외부에서 값을받을 수있는 생성기 함수입니다. 코 루틴 사용의 이점은 함수 실행을 일시 중지하고 나중에 다시 시작할 수 있다는 것입니다. 네트워크 작업의 경우 응답을 기다리는 동안 함수 실행을 일시 중지하는 것이 좋습니다. 시간을 사용하여 다른 기능을 실행할 수 있습니다.

미래는 Promise자바 스크립트 의 객체 와 같습니다 . 미래에 구체화 될 가치에 대한 자리 표시 자입니다. 위에서 언급 한 경우 네트워크 I / O를 기다리는 동안 함수는 컨테이너를 제공하여 작업이 완료되면 컨테이너를 값으로 채울 것임을 약속합니다. 우리는 미래의 객체를 붙잡고 그것이 충족되면 실제 결과를 검색하기 위해 그것에 대한 메소드를 호출 할 수 있습니다.

직접 답변 : 당신은 필요하지 않습니다 ensure_future당신은 결과가 필요하지 않은 경우. 결과가 필요하거나 예외가 발생한 경우 유용합니다.

추가 크레딧 : 최대 작업자 수를 제어하기 run_in_executor위해 Executor인스턴스를 선택 하고 전달합니다 .

설명 및 샘플 코드

첫 번째 예에서는 코 루틴을 사용하고 있습니다. 이 wait함수는 많은 코 루틴을 가져 와서 함께 결합합니다. 따라서 wait()모든 코 루틴이 소진되면 완료됩니다 (모든 값을 반환하는 완료 / 완료 됨).

loop = get_event_loop() # 
loop.run_until_complete(wait(coros))

이 run_until_complete메서드는 실행이 완료 될 때까지 루프가 살아 있는지 확인합니다. 이 경우 비동기 실행의 결과를 얻지 못하는 것을 주목하십시오.

두 번째 예제에서는 ensure_future함수를 사용하여 코 루틴을 래핑 Task하고 일종의 Future. 코 루틴은를 호출 할 때 메인 이벤트 루프에서 실행되도록 예약되어 있습니다 ensure_future. 반환 된 미래 / 작업 객체에는 아직 값이 없지만 시간이 지남에 따라 네트워크 작업이 완료되면 미래 객체가 작업 결과를 보유합니다.

from asyncio import ensure_future

futures = []
for i in range(5):
    futures.append(ensure_future(foo(i)))

loop = get_event_loop()
loop.run_until_complete(wait(futures))

So in this example, we're doing the same thing except we're using futures instead of just using coroutines.

Let's look at an example of how to use asyncio/coroutines/futures:

import asyncio


async def slow_operation():
    await asyncio.sleep(1)
    return 'Future is done!'


def got_result(future):
    print(future.result())

    # We have result, so let's stop
    loop.stop()


loop = asyncio.get_event_loop()
task = loop.create_task(slow_operation())
task.add_done_callback(got_result)

# We run forever
loop.run_forever()

Here, we have used the create_task method on the loop object. ensure_future would schedule the task in the main event loop. This method enables us to schedule a coroutine on a loop we choose.

We also see the concept of adding a callback using the add_done_callback method on the task object.

A Task is done when the coroutine returns a value, raises an exception or gets canceled. There are methods to check these incidents.

I have written some blog posts on these topics which might help:

• http://masnun.com/2015/11/13/python-generators-coroutines-native-coroutines-and-async-await.html
• http://masnun.com/2015/11/20/python-asyncio-future-task-and-the-event-loop.html
• http://masnun.com/2015/12/07/python-3-using-blocking-functions-or-codes-with-asyncio.html

Of course, you can find more details on the official manual: https://docs.python.org/3/library/asyncio.html

---

Simple answer

• Invoking a coroutine function(async def) does NOT run it. It returns a coroutine objects, like generator function returns generator objects.
• await retrieves values from coroutines, i.e. "calls" the coroutine
• eusure_future/create_task schedule the coroutine to run on the event loop on next iteration(although not waiting them to finish, like a daemon thread).

Some code examples

Let's first clear some terms:

• coroutine function, the one you async defs;
• coroutine object, what you got when you "call" a coroutine function;
• task, a object wrapped around a coroutine object to run on the event loop.

Case 1, await on a coroutine

We create two coroutines, await one, and use create_task to run the other one.

import asyncio
import time

# coroutine function
async def p(word):
    print(f'{time.time()} - {word}')


async def main():
    loop = asyncio.get_event_loop()
    coro = p('await')  # coroutine
    task2 = loop.create_task(p('create_task'))  # <- runs in next iteration
    await coro  # <-- run directly
    await task2

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

you will get result:

1539486251.7055213 - await
1539486251.7055705 - create_task

Explain:

task1 was executed directly, and task2 was executed in the following iteration.

Case 2, yielding control to event loop

If we replace the main function, we can see a different result:

async def main():
    loop = asyncio.get_event_loop()
    coro = p('await')
    task2 = loop.create_task(p('create_task'))  # scheduled to next iteration
    await asyncio.sleep(1)  # loop got control, and runs task2
    await coro  # run coro
    await task2

you will get result:

-> % python coro.py
1539486378.5244057 - create_task
1539486379.5252144 - await  # note the delay

Explain:

When calling asyncio.sleep(1), the control was yielded back to the event loop, and the loop checks for tasks to run, then it runs the task created by create_task.

Note that, we first invoke the coroutine function, but not await it, so we just created a single coroutine, and not make it running. Then, we call the coroutine function again, and wrap it in a create_task call, creat_task will actually schedule the coroutine to run on next iteration. So, in the result, create task is executed before await.

Actually, the point here is to give back control to the loop, you could use asyncio.sleep(0) to see the same result.

Under the hood

loop.create_task actually calls asyncio.tasks.Task(), which will call loop.call_soon. And loop.call_soon will put the task in loop._ready. During each iteration of the loop, it checks for every callbacks in loop._ready and runs it.

asyncio.wait, asyncio.ensure_future and asyncio.gather actually call loop.create_task directly or indirectly.

Also note in the docs:

Callbacks are called in the order in which they are registered. Each callback will be called exactly once.

---

A comment by Vincent linked to https://github.com/python/asyncio/blob/master/asyncio/tasks.py#L346, which shows that wait() wraps the coroutines in ensure_future() for you!

In other words, we do need a future, and coroutines will be silently transformed into them.

I'll update this answer when I find a definitive explanation of how to batch coroutines/futures.

---

From the BDFL [2013]

Tasks

• It's a coroutine wrapped in a Future
• class Task is a subclass of class Future
• So it works with await too!

---

• How does it differ from a bare coroutine?
• It can make progress without waiting for it
  • As long as you wait for something else, i.e.
    • await [something_else]

With this in mind, ensure_future makes sense as a name for creating a Task since the Future's result will be computed whether or not you await it (as long as you await something). This allows the event loop to complete your Task while you're waiting on other things. Note that in Python 3.7 create_task is the preferred way ensure a future.

Note: I changed "yield from" in Guido's slides to "await" here for modernity.

참고URL : https://stackoverflow.com/questions/34753401/difference-between-coroutine-and-future-task-in-python-3-5