How to Run Python Functions in Parallel Using Ray API

There are many ways to run functions in parallel, but there might be many caveats to deal with. We can overcome all those caveats by using Ray is, a simple, universal API for building distributed applications.

What Ray can do for you:

• Providing simple primitives for building and running distributed applications.
• Enabling end-users to parallelize single machine code with little to zero code changes.
• Including a large ecosystem of applications, libraries, and tools on top of the core Ray to enable complex applications.

How to install Ray:sudo pip3 install Ray

A basic Ray example:

• Line 2 imports ray into our project
• @ray.remote decorator is used to define which functions will run in parallel
• Line 31 ray.init() initializes Ray
• Line 32 ray.get([func1.remote(),func2.remote()]) the list of functions to run in parallel

Ray provides a very nice output of the executing functions./test1.py
(func2 pid=1596) func 2 Start Time = 11:51:48
(func1 pid=1591) func 1 Start Time = 11:51:48
(func2 pid=1596) func 2 End Time = 11:51:50
(func1 pid=1591) func 1 End Time = 11:51:50

This is a very basic example of Ray that executes functions in parallel, those functions do not return anything, let's see a more advanced example of parallel execution with functions that accept parameters and return results

• Line 14 —u sing ray.get we execute the functions in parallel with parameters, and they return their results in variables ret1 and ret2.

ray.get function accepts functions as parameters in a list form, also returns results as a list, this means that we can do the following which can be very useful when the number of functions to be executed is varied each time.

I hope you found the article useful!