Python generator with yield for better memory management

Generators are simple way of creating iterator in python. They are similar to functions but yield is used in place or return.

def my_generator():
    yield 'a'
    yield 'b'
    yield 'c'

obj=my_generator()
print(next(obj))  # output a
print(next(obj))  # output b

Difference between function and generator

Generator must be an iterator. We can apply next() to the object of a generator.

Generator stops execution after reaching yield and store all the data of the variables. The execution resumes from the next line in the subsequent call to generator (with stored values of the variables ). In case of function the full code is executed.

Generators are memory efficient as they stop and resume execution in sequential calls

When we use yield in place of return, the function became a generator.

def my_generator():
    for i in range(0,5):
        yield  i      # Stops here and returns value of i 
        print('hi')   # resumes from here in next call 
    
obj=my_generator()
print(next(obj))   # Output 0 
#print(next(obj))  # Output hi 1

In the code above the first call to the generator will print 0 and will not print the string hi ( Why ? ). If you uncomment the 2nd call , it will first print the string hi and then it will print 1 .
Read more on range()

Using for loop

def my_generator():
    for i in range(0,5):
        yield i

for j in my_generator():
    print(j)

Output is here

Fibonacci series using generator

def my_febonacci(n):
    i=j=1
    for x in range(0,n):
        i,j=j,i+j
        yield j
    
obj=my_febonacci(5)
print(next(obj))   # Output 2
print(next(obj))   # Output 3
print(next(obj))   # Output 5
print(next(obj))   # Output 8
print(next(obj))   # Output 13
print(next(obj))   # Output Error StopIteration

Using for loop Each call returns one value and it stops execution once all the values are over.
Here For loop stops execution once it came across StopIteration exception. Here is the code.

def my_febonacci(n):
    i=j=1
    for x in range(0,n):
        i,j=j,i+j
        yield j
    
for a in my_febonacci(5):
 print(a)

Let us try the same code using a function.

def my_febonacci(n):
    my_list=[]
    i=j=1
    for x in range(0,n):
        i,j=j,i+j
        my_list.append(j)
    return my_list
    
print(my_febonacci(5))

Output is here

[2, 3, 5, 8, 13]

In case of function we need to store all the outputs in memory and requirement increases for a big series. In case of generator it is not that high and the output is posted back and requirement of memory is minimal. If we are required to work on Big Data where we need to act upon one line only instead of keeping all the data in memory, generators came handy.

Reading data from a file

By using generator we can read part of the data of a big file. This will be more efficient than reading the complete data using more memory. Here is the code to read chunk of data at a time from a file.

def read_in_chunks(fob, data_size=1024):
    
    while True:
        data = fob.read(data_size)
        if not data:
            break
        yield data


fob = open('plus2net_demo.sql') 
print(next(read_in_chunks(fob)))
print("\n\n### End of one chunk of data ###\n\n")
print(next(read_in_chunks(fob)))
print("\n\n### End of one chunk of data ###\n\n")
print(next(read_in_chunks(fob)))

All String methods list Python- Tutorials

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author Watch on YouTube

Connect:

Subscribe to our YouTube Channel here