Python 3.9 Features

Date: 01 December 2020

Category: Releases

Tag: Python

Hot on the heels of my post about the new features in Python 3.8, here’s a roundup of the new features in Python 3.9.

This information is primarily from Real Python and the Python documentation.

zoneinfo Library

By default, datetime objects have no timezone information. Timestamps without timezone information are called naive.

It was possible to pass the tz parameter into methods such as timezone.now() to associate the returned timestamp with a timezone, however prior to python 3.9 the only timezone that was provided out-of-the-box was UTC (others had to be implemented manually, or imported from third-party libraries).

Python 3.9 introduces the zoneinfo module, which contains a ZoneInfo class that accesses the computer’s time zone database and makes that information available in the code. Timezones can be specified by name, and the module handles zone name changes during daylight savings time.

from datetime import datetime
from zoneinfo import ZoneInfo

datetime.now(tz=ZoneInfo("Europe/London"))
# datetime.datetime(2020, 12, 2, 11, 30, 0, 717862, tzinfo=zoneinfo.ZoneInfo(key='Europe/London'))

# Convert times between timezones
_.astimezone(ZoneInfo("Europe/Moscow"))  # Using underscore to access previous expression result
# datetime.datetime(2020, 12, 2, 14, 30, 0, 717862, tzinfo=zoneinfo.ZoneInfo(key='Europe/Moscow'))

Note: Remember that the information is coming from a database on the computer. Therefore, the available timezones will change from operating system to operating system. Windows does not have a suitable database at all!

Therefore, need to make sure the tdzata version of the database provided as a Python library (Git repo) is installed:

pip install tzdata

Real Python makes the following recommendations for best practices with timezones:

  • Civil times should have a specified timezone
    • These are events that occur in a physical location - like meetings, train times, concerts, etc.
    • Thus, datetime objects should be associated with the timezone appropriate to the location
  • Timestamps should be naive and based on UTC
    • Things like server logs, etc.
    • UTC is always monotonically increasing - you don’t want to have to adjust the order of events based on timezone changes (daylight savings, etc.)

New Dictionary Operators

Two new operators are introduced:

  • Use the | operator to create a new dictionary based on the merging of two dictionaries, without effecting the originals
  • Update a dictionary based on another using the |= operator
# An original method of merging two dictionaries, without impacting the original 
pycon = {2016: "Portland", 2018: "Cleveland"}
europython = {2017: "Rimini", 2018: "Edinburgh", 2019: "Basel"}
merged = {**pycon, **europython}
# {2016: 'Portland', 2018: 'Edinburgh', 2017: 'Rimini', 2019: 'Basel'}

# Could also get the same results with the following, which updates the original dictionary in place (hence the copy)
merged = pycon.copy()
merged.update(europython)  # Note that this returns None

# Possible to use the walrus operator introduced in Python 3.8 to reduce this to a single line
(merged := pycon.copy()).update(europython)

# The same can be achieved using the new pipe operator
merged = pycon | europython

# And the original pycon dictionary can be updated in place using the |= operator
pycon |= europython  # Returns none, but pycon has been updated

Note also that:

One advantage of using | is that it works on different dictionary-like types and keeps the type through the merge

For example, if a defaultdict was being used, | will preserve the type, whereas {**dict1, **dict2} will not.

Decorators

Prior to Python 3.9 a decorator had to be a named, callable object (usually a function or a class). Now, decorators can be any callable expression. Real Python comments that this is a bit of a niche improvement, given that not many will need this flexibility.

Example from Real Python:

# normal, shout and whisper are functions
DECORATORS = {"normal": normal, "shout": shout, "whisper": whisper}

voice = input(f"Choose your voice ({', '.join(DECORATORS)}): ")

# Function is being accessed as an item from the DECORATORS dictionary
@DECORATORS[voice]
def get_story():
    pass

In earlier versions of Python, it would have been necessary to assign the dictionary item to a variable and use this to decorate:

selected_decorator = DECORATORS[voice]

@selected_decorator
def get_story():
    pass

Annotated Type Hints

Introduction of typing.Annotated class, whcih can be used to combine type hints with other information (e.g. documentation).

The following is an example from Real Python, where the Annotated class has been used to specify the expected units of each argument:

from typing import Annotated

def speed(distance: Annotated[float, "feet"], time: Annotated[float, "seconds"]) -> Annotated[float, "miles per hour"]:
    ...

To reduce the verbosity of code, type aliases can be employed. These are variables representing the annotated types:

from typing import Annotated

Feet = Annotated[float, "feet"]
Seconds = Annotated[float, "seconds"]
MilesPerHour = Annotated[float, "miles per hour"]

def speed(distance: Feet, time: Seconds) -> MilesPerHour:
    ...

Generic Type Hints

In previous Python versions it was necessary to import classes from the typing module for generics like list and dict. This is no longer necessary in Python 3.9.

# Earlier versions of Python
from typing import List
numbers: List[float]

# From Python 3.9, list can be used directly as a type
numbers: list[float]

String Methods for prefix and suffix removal

Most easily demonstrated using the Real Python example:

"three cool features in Python".removesuffix(" Python")
# 'three cool features in'

"three cool features in Python".removeprefix("three ")
# 'cool features in Python'

# Nothing is removed here as the prefix to be removed does not appear at the start of the string
"three cool features in Python".removeprefix("Something else")
# 'three cool features in Python'

Introduction of graphlib Module

New core python module encompassing functionality to operate with graph-like structures. Python docs. Seems a little immature at the moment, probably better off continuing to use networkx for now.

A dictionary is used to describe the graph, where values are an iterable. Note: remember that strings are iterables, therefore single strings need to be wrapped in some kind of container.

dependencies = {
    "realpython-reader": {"feedparser", "html2text"},
    "feedparser": {"sgmllib3k"},
}

The graphlib.TopologicalSorter class can be used to determine the order of the graph; the result will not necessary be unique (i.e. there could be multiple valid possible orders).

From Real Python:

TopologicalSorter has an extensive API that allows you to add nodes and edges incrementally using .add(). You can also consume the graph iteratively, which is especially useful when scheduling tasks that can be done in parallel.

The module documentation gives an example:

topological_sorter = TopologicalSorter()

# Add nodes to 'topological_sorter'...

topological_sorter.prepare()
while topological_sorter.is_active():
    for node in topological_sorter.get_ready():
        # Worker threads or processes take nodes to work on off the
        # 'task_queue' queue.
        task_queue.put(node)

    # When the work for a node is done, workers put the node in
    # 'finalized_tasks_queue' so we can get more nodes to work on.
    # The definition of 'is_active()' guarantees that, at this point, at
    # least one node has been placed on 'task_queue' that hasn't yet
    # been passed to 'done()', so this blocking 'get()' must (eventually)
    # succeed.  After calling 'done()', we loop back to call 'get_ready()'
    # again, so put newly freed nodes on 'task_queue' as soon as
    # logically possible.
    node = finalized_tasks_queue.get()
    topological_sorter.done(node)

A very simple example would be:

import time
from graphlib import TopologicalSorter

topological_sorter = TopologicalSorter()

topological_sorter.add("A")
topological_sorter.add("B", "A")
topological_sorter.add("C", "A")
topological_sorter.add("D", "B", "C")

topological_sorter.prepare()

while topological_sorter.is_active():
    for node in topological_sorter.get_ready():
        print(f"Processing node: {node}")
        time.sleep(1)
        topological_sorter.done(node)

Which outputs the following, with a gap between each printed statement:

Processing node: A
Processing node: B
Processing node: C
Processing node: D

Note: The method static_order marks the nodes as done, and so it can’t be used for debugging before processing without the re-defining the nodes.