Higher-Order Functions in Python: Concepts, Examples and Uses

Introduction to Higher-Order Functions in Python

Sometimes, a function needs to perform the same task using different logic. For example, a function may need to process a number using different operations depending on the situation. Writing a separate function for each operation can lead to repeated code.

To overcome this situation in a more flexible and reusable way, Python provides higher-order functions.

What are Higher-Order Functions in Python?

Higher-Order Functions in Python are functions that accept another function as an argument or return a function.

For example, a higher-order function can receive a function and use it to perform part of its task. The function that is passed can be changed without changing the higher-order function itself.

A higher-order function can also return another function. This allows it to create and return behavior that can be used later.

Note: Higher-order functions are possible because Python treats functions as values. To understand how this works, let’s first look at first-class functions.

First-Class Functions in Python

First, let’s understand what first-class functions are and how they are used in higher-order functions.

Definition: First-class functions are functions that Python treats like values. You can assign them to variables, pass them as arguments, return them from other functions, and store them in collections.

Think of a function like a piece of information that can be stored, passed around or returned whenever needed. Python allows functions to be handled in these ways just like other values.

Let’s understand how Python treats functions as values through the following scenarios:

  1. Treating Functions Like Values
  2. Assigning a Function to a Variable
  3. Storing Functions in Collections
  4. Passing Functions as Arguments
  5. Returning Functions From Functions

1. Treating Functions Like Values

When you use a function name without parentheses, Python refers to the function itself rather than calling it.

def greet():
    print("Hello")

print(greet)


# Output:
<function greet at 0x...>

Explanation: Here, greet refers to the function itself. Python does not run the function because there are no parentheses after greet.

↑ Move to Section Top

2. Assigning a Function to a Variable

You can assign a function to a variable just as you assign other values to a variable.

def greet():
    print("Hello")

message = greet

message()


# Output:
Hello

Explanation: Here, message refers to the same function as greet. Calling message() runs the greet() function.

Note: Notice that greet is assigned without parentheses. Writing greet() would call the function instead of assigning the function itself.

↑ Move to Section Top

3. Storing Functions in Collections

Since functions can be treated like values, you can also store them in collections such as lists.

def add(a, b):
    return a + b

def multiply(a, b):
    return a * b

operations = [add, multiply]

print(operations[0](5, 3))
print(operations[1](5, 3))


# Output:
8
15

Explanation: Here, the list stores the add and multiply functions. The functions are not called when they are added to the list.

The functions are called later using their positions in the list. operations[0](5, 3) calls add(), while operations[1](5, 3) calls multiply().

↑ Move to Section Top

4. Passing Functions as Arguments

Python allows you to pass a function to another function as an argument. This is useful when a function needs to use another function to perform a task.

  1. Passing a Function Without Calling It
  2. Calling the Received Function
  3. Building a Function That Accepts Another Function

1. Passing a Function Without Calling It

When you pass a function as an argument, write the function name without parentheses.

def greet():
    print("Hello")

def process(func):
    print(func)

process(greet)

Explanation: Here, greet refers to the function itself, so process() receives the function as an argument. Python does not call greet() when you pass it this way.

If you write process(greet()), Python calls greet() first and passes its return value to process() instead.

2. Calling the Received Function

A function received as an argument can be called inside the receiving function.

def greet():
    return "Hello"

def process(func):
    return func()

result = process(greet)

print(result)


# Output:
Hello

Explanation: Here, process() receives greet through the func parameter. The expression func() calls the received function.

3. Building a Function That Accepts Another Function

You can build a function that accepts another function and uses it to process a value.

def double(number):
    return number * 2

def apply_operation(value, operation):
    return operation(value)

result = apply_operation(5, double)

print(result)


# Output:
10

Explanation: Here, apply_operation() receives double as the operation argument and calls it with value. The function can therefore use the function passed to it.

You can pass a different function to apply_operation() without changing its code.

def triple(number):
    return number * 3

result = apply_operation(5, triple)

print(result)


# Output:
15

Explanation: Here, apply_operation() works with triple instead of double. The receiving function stays the same while the function passed to it changes.

↑ Move to Section Top

5. Returning Functions From Functions

A function can return another function instead of returning its result. This allows the returned function to be stored and used later.

  1. Returning and Calling a Function
  2. Creating Functions Dynamically

1. Returning and Calling a Function

A function can return another function by using the function name with the return statement.

def greet():
    return "Hello"

def get_function():
    return greet

message = get_function()

print(message())


# Output:
Hello

Explanation: Here, get_function() returns the greet function. The returned function is stored in message, and message() calls it.

Note: Notice that return greet does not use parentheses. Writing return greet() would call the function and return its result instead.

2. Creating Functions Dynamically

A function can create another function inside its body and return it. This allows the outer function to create different functions based on the values passed to it.

def create_multiplier(number):
    def multiply(value):
        return value * number

    return multiply

double = create_multiplier(2)
triple = create_multiplier(3)

print(double(5))
print(triple(5))


# Output:
10
15

Explanation: Here, create_multiplier() creates a multiply() function using the value passed to number. It returns that function, which is stored in double and triple.

The two returned functions use different values. double(5) multiplies by 2, while triple(5) multiplies by 3.

↑ Move to Section Top

Using Callable for Function Type Hints

When a function accepts another function as an argument, the parameter does not clearly show that it expects a function. Python provides Callable to make this expectation clear through a type hint.

Let’s first understand what Callable is and how it helps with function type hints.

1. What Is Callable?

Definition: Callable is a type hint used to describe an object that can be called like a function.

This is useful when building reusable functions that accept other functions as inputs, such as functions for sorting, filtering, validating or processing data.

Let’s understand Callable through an example.

Consider a function that accepts another function as an argument:

from collections.abc import Callable

def process(func: Callable):
    return func()

Explanation: Here, func is expected to be a callable object. A Callable type hint can make this expectation clear.

Note: You can import Callable from collections.abc:

Let’s look at two common ways to use Callable in function type hints.

  1. Type Hint for a Function Parameter
  2. Type Hint for a Function That Returns a Function

1. Type Hint for a Function Parameter

You can use Callable to show that a function parameter is expected to receive another function.

from collections.abc import Callable

def greet(name: str) -> str:
    return "Hello " + name

def process(func: Callable, name: str) -> str:
    return func(name)

result = process(greet, "Ravi")

print(result)


# Output:
Hello Ravi

Explanation: Here, func: Callable indicates that func is expected to be a callable object. The process() function then calls it with name.

Specifying Parameter and Return Types

You can make the type hint more specific by describing the parameter and return types of the function.

from collections.abc import Callable

def greet(name: str) -> str:
    return "Hello " + name

def process(func: Callable[[str], str], name: str) -> str:
    return func(name)

result = process(greet, "Ravi")

print(result)


# Output:
Hello Ravi

Explanation: Here, Callable[[str], str] indicates that func should accept a string and return a string.

The first str describes the function’s parameter type, while the second str describes its return type.

↑ Move to Section Top

2. Type Hint for a Function That Returns a Function

You can also use Callable when a function returns another function.

from collections.abc import Callable

def create_greeting() -> Callable[[str], str]:
    def greet(name: str) -> str:
        return "Hello " + name

    return greet

greeting = create_greeting()

print(greeting("Ravi"))

# Output:
Hello Ravi

Explanation: Here, Callable[[str], str] indicates that create_greeting() returns a function that accepts a string and returns a string.

The type hint describes the function being returned, not the value returned when that function is called.

↑ Move to Section Top

Practical Uses of Higher-Order Functions

Here are some practical uses of Higher-Order Functions in Python:

  1. Reusing Common Function Logic
  2. Creating Function-Based Utilities

1. Reusing Common Function Logic

You can use a higher-order function when several tasks share the same logic but need different operations.

def calculate(a, b, operation):
    return operation(a, b)

def add(a, b):
    return a + b

def multiply(a, b):
    return a * b

print(calculate(10, 5, add))
print(calculate(10, 5, multiply))


# Output:
15
50

Explanation: Here, calculate() contains the common logic for receiving two values and applying an operation. The add() and multiply() functions provide the different operations.

This lets you reuse calculate() instead of writing separate functions with the same structure.

↑ Move to Section Top

2. Creating Function-Based Utilities

You can create a utility function that accepts another function and applies it to each item in a collection.

def process_items(items, operation):
    results = []

    for item in items:
        results.append(operation(item))

    return results

def double(number):
    return number * 2

numbers = [1, 2, 3, 4]

result = process_items(numbers, double)

print(result)


# Output:
[2, 4, 6, 8]

Explanation: Here, process_items() handles the common task of processing each item and storing the results. The double() function determines what operation to apply to each item.

You can pass a different function when you need a different operation without changing process_items().

def square(number):
    return number * number

result = process_items(numbers, square)

print(result)


# Output:
[1, 4, 9, 16]

Explanation: Here, the same utility function processes the numbers using square() instead of double().

↑ Move to Section Top

Higher-Order Functions With Built-in Python Functions

Python includes built-in functions that support Higher-Order Functions in Python by accepting another function as an argument. Functions such as map(), filter() and sorted() use this approach to process or organize data. Let’s see how they work:

  1. map() With a Function
  2. filter() With a Function
  3. sorted() With a Key Function

1. map() With a Function

The map() function applies a function to each item in an iterable and returns the results.

def double(number):
    return number * 2

numbers = [1, 2, 3, 4]

result = map(double, numbers)

print(list(result))


# Output:
[2, 4, 6, 8]

Explanation: Here, double is passed to map() without parentheses. The map() function calls double() for each item in numbers.

The result returned by map() is a map object, so list() is used to display the results as a list.

↑ Move to Section Top

2. filter() With a Function

The filter() function uses a function to select items from an iterable. It keeps the items for which the function returns True.

def is_even(number):
    return number % 2 == 0

numbers = [1, 2, 3, 4, 5, 6]

result = filter(is_even, numbers)

print(list(result))


# Output:
[2, 4, 6]

Explanation: Here, is_even is passed to filter(). The function checks each number and returns True for even numbers.

The filter() function keeps the numbers for which is_even() returns True.

↑ Move to Section Top

3. sorted() With a Key Function

The sorted() function can accept a function through its key parameter. Python uses the returned values to determine the sorting order.

def get_length(word):
    return len(word)

words = ["Python", "AI", "Functions", "Code"]

result = sorted(words, key=get_length)

print(result)


# Output:
['AI', 'Code', 'Python', 'Functions']

Explanation: Here, get_length is passed to the key parameter without parentheses. The sorted() function calls it for each word and uses the returned length to sort the words.

The words are sorted from shortest to longest based on the values returned by get_length().

↑ Move to Section Top

Higher-order functions in Python are closely related to first-class functions, nested functions, and closures. However, these concepts describe different features of Python functions.

  1. Higher-Order Functions vs Nested Functions
  2. Higher-Order Functions vs Closures

1. Higher-Order Functions vs Nested Functions

A nested function is a function defined inside another function. A higher-order function is a function that accepts another function or returns a function.

A nested function does not automatically make the outer function a higher-order function.

The two concepts can therefore appear together, but they are not the same. Nested functions describe where a function is defined, while higher-order functions describe how functions are passed or returned.

ComparisonHigher-Order FunctionsNested Functions
MeaningA function accepts another function or returns a function.A function is defined inside another function.
Main ideaDescribes how functions are passed or returned.Describes where a function is defined.
Required relationshipAnother function must be accepted or returned.An inner function must be defined inside an outer function.
Exampleprocess(func)def outer():
    def inner(): ...

↑ Move to Section Top

2. Higher-Order Functions vs Closures

A closure is a function that remembers values from its enclosing scope even after the outer function has finished running. A higher-order function does not have to remember values from another scope.

def create_multiplier(number):
    def multiply(value):
        return value * number

    return multiply

double = create_multiplier(2)

print(double(5))


# Output:
10

Explanation: Here, create_multiplier() returns the multiply function. The returned function remembers the value of number, which is 2. This makes multiply a closure.

The create_multiplier() function is also a higher-order function because it returns another function.

A higher-order function can return a function without creating a closure. A closure specifically involves a function that retains access to values from its enclosing scope.

ComparisonHigher-Order FunctionsClosures
MeaningA function accepts another function as an argument or returns a function.A function remembers values from its enclosing scope after the outer function finishes.
Main ideaDescribes how functions work with other functions.Describes how a function retains access to enclosing variables.
Requires retained values?No.Yes. The inner function retains access to values from its enclosing scope.
Requires a nested function?No.Yes, a closure involves an inner function that retains access to an enclosing scope.
Exampleapply_operation(func, value)create_multiplier(2) returning a function that remembers 2.

↑ Move to Section Top

Common Mistakes With Higher-Order Functions

The following are some common mistakes to avoid when working with Higher-Order Functions in Python:

  1. Calling a Function Instead of Passing It
  2. Returning a Function’s Result Instead of the Function
  3. Using an Incorrect Callable Type Hint

1. Calling a Function Instead of Passing It

When you pass a function to a higher-order function, pass the function name without parentheses. Adding parentheses calls the function immediately.

Error: Calling the Function Instead of Passing It

def greet():
    return "Hello"

def process(func):
    return func()

result = process(greet())

print(result)


# Output:
TypeError

Explanation: Here, greet() is called before process() receives it. The return value of greet() is passed to process() instead of the function itself.

Correct: Passing the Function

def greet():
    return "Hello"

def process(func):
    return func()

result = process(greet)

print(result)


# Output:
Hello

Explanation: Here, greet passes the function itself to process(). The process() function then calls it using func().

↑ Move to Section Top

2. Returning a Function’s Result Instead of the Function

When a function should return another function, return the function name without parentheses. Using parentheses calls the function and returns its result instead.

Error: Returning the Function’s Result

def greet():
    return "Hello"

def get_function():
    return greet()

message = get_function()

print(message())


# Output:
TypeError: 'str' object is not callable

Explanation: Here, greet() is called immediately, so get_function() returns the string "Hello" instead of returning the greet function. Therefore, message() cannot be used to call it.

Correct: Returning the Function

def greet():
    return "Hello"

def get_function():
    return greet

message = get_function()

print(message())


# Output:
Hello

Explanation: Here, get_function() returns the greet function. The returned function is stored in message, and message() calls it.

↑ Move to Section Top

3. Using an Incorrect Callable Type Hint

When a higher-order function accepts another function, the Callable type hint should describe how the received function is used.

Error: Using a Mismatched Callable Type Hint

from collections.abc import Callable

def greet(name: str) -> str:
    return "Hello " + name

def process(func: Callable[[int], int], name: str) -> str:
    return func(name)

result = process(greet, "Ravi")

print(result)

Explanation: Here, Callable[[int], int] says that func should accept an integer and return an integer. However, greet() accepts a string and returns a string. The type hint therefore does not match the function passed to process().

Note: This mismatch is detected by static type checkers such as mypy or Pyright. Python does not enforce this Callable type hint at runtime, so the code can still run and produce an output.

Correct: Matching the Callable Type Hint

from collections.abc import Callable

def greet(name: str) -> str:
    return "Hello " + name

def process(func: Callable[[str], str], name: str) -> str:
    return func(name)

result = process(greet, "Ravi")

print(result)


# Output:
Hello Ravi

Explanation: Here, Callable[[str], str] correctly indicates that func accepts a string and returns a string. The type hint now matches how process() uses the function.

↑ Move to Section Top

Key Takeaways: Higher-Order Functions

These are the key concepts to remember about First-class and higher-order functions in Python:

  • Higher-order functions accept another function as an argument or return a function.
  • First-class functions allow Python to treat functions like values.
  • Use a function name without parentheses when you want to refer to or pass the function itself.
  • A function can be passed as an argument and called inside another function.
  • A function can return another function, which can be stored and used later.
  • Callable provides a type hint for parameters or return values that expect callable objects.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top