Static Duck Typing With Python’s Protocols

In our ongoing series at The Turing Taco Tales, we've been unraveling the intricacies of Python's type system. Previously, we've delved into the realm of type hints and type variables, unveiling how these features enhance code readability and maintainability. Today, we embark on a new chapter: the advent of Protocols in Python 3.8.

To appreciate the significance of Protocols, we must first understand two contrasting typing paradigms in programming languages: nominal and structural. Nominal typing, predominant in languages like Java, relies on explicit declarations and their names. Two objects have the same type if their types have the same name, and a type is a subtype of another if there is an explicitly declared either through the implements or extend keywords.

In contrast, structural typing, also popularly known as duck typing in dynamically typed languages, is guided by a simple principle: "If it looks like a duck and quacks like a duck, then it probably is a duck."

This phrase, from James Whitcomb Riley, captures the essence of structural typing. It's not about a type's name or explicit lineage but about the structure and behavior. If an object fulfills all the requirements of a structure (like methods and properties), it's considered an instance of that type.

The term Duck Typing has been primarily used in dynamically typed languages; when combined with compile time verification, it is usually named Structural Typing. Some languages, like Go or TypeScript, adopt this approach over the nominal one, while others, like Scala, use it as a complement.

Python 3.8 adopted PEP 544, introducing statically checked duck typing under Protocols. Type checkers can verify whether an object conforms to a particular protocol at development time, not just at runtime. It's a significant leap from Python's runtime type checking, offering a more robust, error-proof approach to managing types in your code.

Understanding Protocols in Python

Before Python 3.8, type hinting was largely nominal, focusing on explicit type declarations. While this method was straightforward, it sometimes restricted the dynamic flexibility that Python developers cherished. The introduction of Protocols, as detailed in PEP 544, offered a compelling alternative.

What Exactly are Protocols?

A Protocol defines the "minimum requirements" for a type, an object that complies with all the requirements accepted by type checkers anywhere an object of that Protocol is expected. If the object implements some but not all of the requirements, it won't be considered as implementing the Protocol, and type checkers will report a type error.

Unlike classes, which dictate what an object is, Protocols focus on how an object behaves, and we do not need to explicitly mark our objects as implementing the Protocol. 

This distinction is crucial. A Protocol doesn't care about the specific type of an object; it cares about whether the object meets specific criteria and whether it implements all the methods or attributes.

If an object fulfills the requirements of a Protocol, it's considered as an instance of that Protocol, regardless of its actual class.

The concept of duck typing is not new to Python, but Protocols have changed the ability to verify duck typing statically, at development time rather than dynamically at runtime. 

With the introduction of Protocols, Type checkers can ensure that an object adheres to a specific Protocol, making the code flexible and more robust by preventing type errors at runtime.

Illustrating Protocols With an Example

Imagine a function that needs to write data to a file-like object. Instead of restricting this function only to accept file objects, we can use a Protocol to allow any object that has a write() method:

from typing import Protocol

class Writeable(Protocol):
    def write(self, data: str) -> None:
        ...

class Logger:
    def write(self, data: str):
        print(f"Logging data: {data}")

class File:
    def write(self, data: str):
        with open("output.txt", "a") as f:
            f.write(data)

def save_data(storage: Writeable, data: str):
    storage.write(data)

# Both Logger and File instances are acceptable
save_data(Logger(), "Test log entry")
save_data(File(), "Test file content")
        

Traditionally, this would require the objects to inherit from a specific class or interface. However, with Protocols, we can define a set of methods that any object must have to be considered a specific type in the context of that function.
Let's illustrate this with a Writeable Protocol. This Protocol specifies a single method, write(data: str), which any object adhering to the Protocol should implement. The beauty of Protocols lies in their flexibility; they allow objects to be recognized by their behavior rather than their class hierarchy.
In our scenario, we have two distinct classes: Logger and File. Neither of these classes inherit from Writeable, yet they implement a write method that aligns with the Protocol's requirements.
We then have a function, save_data, which takes an object of type Writeable.

Thanks to Protocols, we can pass object instances from the Logger or File class to this function because they fulfill the structural requirements laid out by the Writeable Protocol without inheriting it directly.

Retrofitting the Standard Library for Type Checking

Python's introduction of Protocols in version 3.8 did more than add new capabilities; it provided a structured way to retrofit existing functionalities within the standard library, making them compatible with static type checking. This enhancement is significant because it bridges Python's dynamic nature with the rigor of static type analysis, thus fostering a more robust coding environment.

Reframing Iterable and Iterator

Take, for instance, the  Iterable and Iterator Protocols. Since its inception, these concepts have been integral to Python, facilitating the language's powerful looping and iteration mechanisms. By formally defining these behaviors as Protocols, Python has not introduced new functionality but has instead provided a framework for type checkers to understand and validate these patterns.

• The Iterable Protocol encapsulates the behavior of objects that Python can loop over. While this functionality existed through the __iter__() method, the Protocol formulates it so that type checkers can recognize and validate it.
• Similarly, the Iterator Protocol, requiring both __iter__() and __next__() methods, formalizes the iteration process. Static type analysis tools can ensure that objects used as iterators conform to the expected behavior.

from typing import Iterable, Iterator

class Fibonacci(Iterable[int]):
    def __init__(self, max_number: int):
        self.max_number = max_number
        self.first, self.second = 0, 1

    def __iter__(self) -> Iterator[int]:
        return self

    def __next__(self) -> int:
        if self.first > self.max_number:
            raise StopIteration
        current = self.first
        self.first, self.second = self.second, self.first + self.second
        return current

# Using the Fibonacci class
fib_sequence = Fibonacci(10)
for number in fib_sequence:
    print(number)
        

ContextManager: Standardizing Context Management

The ContextManager Protocol is another example. Context managers have always been a part of Python, primarily used for efficient resource management. Introducing the ContextManager Protocol doesn't change their behavior but provides a type-safe structure. Now, objects used within a statement can be type-checked to ensure they correctly implement the __enter__() and __exit__() methods.

Callable and Container Protocols

The Callable Protocol is a formalization of the concept of "callability". This Protocol allows type checkers to verify that objects expected to behave like functions (those implementing the __call__() method) meet this criterion at the code's static analysis phase.

Similarly, Protocols like Container, Collection, and Sequence do not introduce new functionality for container objects. Instead, they define the existing behaviors, like membership tests or sequence operations, in a way that can be statically checked.

Enhancing Python's Ecosystem

This retrofitting of existing functionalities with Protocols is crucial. It maintains Python's dynamic and flexibility while enhancing the language's ability to perform static type checking. The vast ecosystem of Python's data structures and functions can now be more rigorously verified for type consistency, reducing the likelihood of runtime errors and increasing code maintainability.

Through Protocols, Python has effectively merged its traditionally dynamic typing with the benefits of static type analysis, allowing developers to write more precise, more predictable code without sacrificing the language's inherent expressiveness and versatility.

Conclusion

Protocols introduce a flexible way of defining interfaces based on behavior rather than type, enhancing Python's dynamic nature with static type checking's robustness.

We've explored how Protocols retrofit existing functionalities in the standard library and the creation of custom Protocols. This feature significantly improves code readability and maintainability.

Some exciting topics await us in the following article: default implementations and the inheritance rules of Protocols. Inheritance offers us extensive possibilities for extending and combining them in innovative ways. Default implementations make protocols similar to Traits from languages like Scala or Rust.

Stay tuned for our next article, where we'll delve into the nuances of Protocol inheritance.

For more insights into Python's evolving landscape, consider subscribing to The Turing Taco Tales.

Addendum: A Special Note for Our Readers

I decided to delay the introduction of subscriptions, you can read the full story here.

In the meantime, I decided to accept donations.

If you can afford it, please consider donating:

Every donation helps me offset the running costs of the site and an unexpected tax bill. Any amount is greatly appreciated.

Also, if you are looking to buy some Swag, please visit I invite you to visit the TuringTacoTales Store on Redbubble.

Take a look, maybe you can find something you like: