The Promise of LiteralString in Python 3.11

Overview

Python has evolved quickly from purely dynamic typing into a language with string static typing credentials. Adding PEP 675 in Python 3.11 is taking these capabilities even further. This proposal is about making Python code safer and easier to understand.

In our previous article, we reviewed the introduction of Type Literals, which we use to ensure variables in Python can take only a limited set of values. We also ensure those values are explicitly written into the code instead of being received as input from the user.

PEP 675 generalizes the checking of literals for strings, removing the limitation to specific values and allowing us to accept any string as long as that string is an explicit literal in a Python program.

This might seem a small change, but it's a big deal. Why? Because in coding, the devil's often in the details. Let's say we're writing a function and must be clear about what kind of string it should work with. PEP 675 is here to help with that, making your code easier to read and potentially ironclad against some common security headaches, like those nasty SQL injection attacks.

The article we're diving into takes a close look at PEP 675. More precise code means fewer bugs and better security, which every Python developer can get excited about.

So, let's dig into how PEP 675 could be Python's next giant leap forward, keeping the language on top of its game.

A Brief History of Type Hints

Python's journey with type annotations is a fascinating story of how the language evolves to become even more powerful and user-friendly.

It started with PEP 484, which was like a new chapter for Python. Known for being easy-going with its dynamic typing, Python introduced type hints, and it was like giving developers a superpower. Suddenly, you could be clear about what type of data your code was dealing with, whether it was variables, what you're passing into functions, or what you're getting out of them. And this wasn't just about making code look neat – it made life easier with better tool support and made it simpler to catch bugs.

Then came PEP 586, and things got even more enjoyable. Imagine we're setting up a function, and you only want it to accept concrete things, like "ascending" or "descending". That's where literal types come into play. They allow us to control our code's behavior more precisely. It's like having a bouncer at the door of your function, only letting in the VIPs.

But there was a catch. What if you needed to know all the possible VIPs ahead of time? What if the list of who's allowed in your function could change while your program runs? This was a real head-scratcher, especially in areas like working with databases, managing configurations, or building user interfaces.

Here's where PEP 675 enters the scene. It's like a plot twist in Python's story. It proposes a way to say, "Hey, this string can be anything from a specific set, but I might not know what that set is right now." This is huge because it helps Python bridge the gap between being super flexible and having the ability to do strict checks on your code. It's like having the best of both worlds – the freedom of Python's dynamic nature and the safety net of static type checking. This could be a big deal for ensuring our code is doing exactly what we want it to, significantly when the rules of the game can change on the fly.

The Proposal of PEP 675

In our previous article, we discussed using Literal Types to improve the security of functions that perform SQL queries. Unfortunately, Literal Types require us to know all the possible values ahead of time, making them unsuitable as types for variables for which we need to know the possible values our library users might pass.

If Literal Types were our only option, we would be forced to use a general string type, opening our code to some vectors of attack like SQL Injection:

import sqlite3
from typing_extensions import LiteralString

# Create a SQLite database and a 'products' table with sample data
conn = sqlite3.connect(':memory:')
cursor = conn.cursor()
cursor.execute("CREATE TABLE products (id INTEGER PRIMARY KEY, name TEXT)")
cursor.execute("INSERT INTO products (name) VALUES ('Widget')")
cursor.execute("INSERT INTO products (name) VALUES ('Gadget')")
cursor.execute("INSERT INTO products (name) VALUES ('Thingamajig')")
conn.commit()


# Function with potential SQL injection vulnerability
def unsafe_query(db_conn, product_name: str) -> None:
    query = f"SELECT * FROM products WHERE name = ?"
    cursor = db_conn.cursor()
    cursor.execute(query, (product_name,))
    for row in cursor.fetchall():
        print(row)


# Example usage
print("Unsafe Query:")
product_name = input()
# Possible SQL Injection! E.g: "products; DROP TABLE products;"
unsafe_query(conn, product_name)  
        

But with the addition of LiteralString, we can use it to limit the inputs to our function to strings explicitly written in the Python program, forbidding the use of strings not directly typed in a Python program.

# Function using LiteralString to prevent SQL injection
def safe_query(db_conn, product_name: LiteralString) -> None:
    query = f"SELECT * FROM products WHERE name = ?"
    cursor = db_conn.cursor()
    cursor.execute(query, (product_name,))
    for row in cursor.fetchall():
        print(row)

# Example usage
print("Safe Query:")
# Only allows literal strings
safe_query(conn, "Widget")  
        

Once Type Checkers completely implement support for LiteralStrings the following code should be reported as an error:

product_name = input()
print("Safe Query:")
# Only allows literal strings
safe_query(conn, product_name)  
        

But it would allow to pass calculated strings for which it can proof all the strings used in the calculation were LiteralString, which means the following code is allowed by type checkers:

query = f"SELECT * FROM products WHERE name = "
value = "Widget"          

# This will be allowed 
# because it is a concatenation of two literal strings
safe_query(conn, query + value)  
        

But this will be reported as an error:

query = f"SELECT * FROM products WHERE name = "
value = input()          

# This will be allowed 
# because it is a concatenation of two literal strings
safe_query(conn, query + value)  
        

Type checkers have not yet completed support for PEP 675; type errors like the above won’t be caught, but they will also not report inexisting errors. This allows us to start using the LiteralString type immediately, even if we won’t receive its benefits until type checkers complete the adoption of the PEP 675.

Conclusion

We look forward to type checkers fully implementing PEP 675, allowing us to realize the benefits of LiteralString. 

Meanwhile, We can start writing code with LiteralString in mind. Type checkers have begun introducing minimum compatibility for this feature, allowing developers to write code without encountering false errors. This proactive approach ensures that when full support is available, the transition will be seamless and the benefits immediate.

Addendum: A Special Note for Our Readers

I decided to delay the introduction of subscriptions. You can read the full story here.

If you find our content helpful, there are several ways you can support us:

• The easiest way is to share our articles and links page on social media; it is free and helps us greatly.
• If you want a great experience during the Chinese New Year, I am renting my timeshare in Phuket. A five-night stay in this resort in Phuket costs 11,582 € on Expedia. I am offering it in USD at an over 40% discount compared to that price. I received the Year of the Snake in style.

Anantara Vacation Club Phuket Mai Khao $$1,390/night

Phuket, Thailand / Posting R1239106

RedWeek

• If your finances permit it, we are happy over any received donation. It helps us offset the site's running costs and an unexpected tax bill. Any amount is greatly appreciated:

• Finally, some articles have links to relevant goods and services; buying through them will not cost you more. And if you like programming swag, please visit the TuringTacoTales Store on Redbubble. Take a look. Maybe you can find something you like: