Ctrl+K

Comprehensions

30. Comprehensions#

Python has a bit of syntactic sugar to help create lists, sets, and dictionaries.

While technically not needed, comprehensions allow programmers to create these data structures from other sequences cleanly and concisely.

30.1. List Comprehension#

The typical example to show list comprehensions is first building a list utilizing a for-loop. For example, if we want to produce a list from 1 to 10, we can use the following code:

1numbers = []
2for x in range(1,11):
3    numbers.append(x)
4print(numbers)

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

With list comprehensions, we can concisely put this into a single line of code:

1numbers = [ number for number in range(1,11)]
2print(numbers)

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

List comprehensions follow this initial pattern:

[expression for item in iterable]

which is equivalent to following for loop

values = []
for item in iterable:
    values.append(expression)

We can also provide filtering capability by adding a condition to the comprehension. These list comprehensions have this pattern:

[expression for item in iterable if condition]

For example, if we only wanted the square of even numbers between 1 and 10:

1numbers = [ number**2 for number in range(1,11) if number % 2 == 0]
2print(numbers)

[4, 16, 36, 64, 100]

The corresponding for loop for the filter would have this format:

values = []
for item in iterable:
    if condition:
        values.append(expression)

1values = []
2for x in range(1,11):
3    if x % 2 == 0:
4        values.append(x**2)
5print(values)

[4, 16, 36, 64, 100]

You can also create lists of lists by using a list comprehension as the expression (i.e., a nested comprehension.)

1[[ number * y for number in range(0,9)] for y in range(1,4)]

[[0, 1, 2, 3, 4, 5, 6, 7, 8],
 [0, 2, 4, 6, 8, 10, 12, 14, 16],
 [0, 3, 6, 9, 12, 15, 18, 21, 24]]

30.2. Set Comprehensions#

Set comprehensions function in the same manner as list comprehensiona, except that we use curly brackets at the start and the end.

The two formats are -

{expression for item in iterable}
{expression for item in iterable if condition}

1modular_numbers = { number % 5 for number in range(1,100)}
2print(modular_numbers)

{0, 1, 2, 3, 4}

30.3. Dictionary Comprehensions#

Dictionary comprehensions function the same as list and set comprehensions; they just have a slightly more complex syntax as we need to store both the key and the value. Dictionary comprehensions follow one of these two formats:

{key_expression : value_expression for item in iterable}
{key_expression : value_expression for item in iterable if condition}

For example to count all of the different characters in a string:

1s = "the quick brown fox jumps over the lazy dog"
2counts = { letter: s.count(letter) for letter in set(s)} 
3print(counts)

{'f': 1, 'e': 3, 'l': 1, 'b': 1, 'o': 4, 'r': 2, 'q': 1, 'x': 1, 'a': 1, 'v': 1, 'w': 1, 'y': 1, 'k': 1, 'i': 1, 'h': 2, 'n': 1, 'j': 1, 't': 2, 'm': 1, 'd': 1, 'z': 1, 'u': 2, 'g': 1, ' ': 8, 'p': 1, 's': 1, 'c': 1}

The set(s) was used so that each character count was only computed once. As the string is an iterable/sequence, set(s) converts it into a set of the distinct characters in the string s. What’s the length of set(s) versus list(s). Write some code to determine the answer.

1# print out the contents of set(s) and list(s)
2
3# print and compare the sizes of set(s) and list(s)

As another example, this comprehension swaps the keys and values in a dictionary. As many values are not unique, we will have fewer entries in the new dictionary.

1flipped = { value : key for key, value in counts.items()}
2print(flipped)

{1: 'c', 3: 'e', 4: 'o', 2: 'u', 8: ' '}

30.4. Maps and Filters#

While not directly used by name, comprehensions use two key capabilities found in functional programming: map and filter.

Mapping transforms a single value into another value by applying an expression or calling a function with the original value. In the second example, x**2 serves as a map. Applying a map will result in a list with the same number of elements as before, although the resulting values do not need to have the same type as the original. For example, a map can convert a list of numbers to a list of strings.

1nlist = [5,6,7.023,3.14,]
2alist = [str(n) for n in nlist]
3print(alist)

['5', '6', '7.023', '3.14']

Filtering selects only certain elements that match a condition. Comprehensions perform this with their if condition clauses. Typically, the resulting list will have fewer elements than the original iterable. In the second code example, if number % 2 == 0 serves as a filter.

30.5. Notes#

  1. Only mutable types (lists, sets, and dictionaries) have comprehensions.

  2. While comprehensions are a powerful tool, it does become easier to write code that’s difficult to read - especially with nested comprehension. A fundamental tenet in programming is to produce maintainable code - this requires that the code be readable and understood both by yourself and by others. Using for loops to create nested collections will likely be the easiest to understand.

  3. If the logic within the comprehension is complex, consider moving it to a separate function. This enhances readability and reusability.

  4. From a performance standpoint, list comprehensions are roughly 20% faster than equivalent for loops with append() statements. Stack Overflow Discussion. From a practicality standpoint, choose which version is more understandable and readable. And, if you are really worried about speed, choose another programming language.

  5. Comprehensions are the “pythonic” way as compared to for loops.

30.6. Suggested LLM Prompts#

  • Write a detailed explanation of list comprehensions in Python, including their syntax, use cases, and how they differ from traditional for loops. Provide examples to illustrate different scenarios where list comprehensions can be used effectively.

  • Given a set of simple and complex for loops, rewrite them using list comprehensions. Explain the thought process behind each conversion and highlight the benefits of using list comprehensions over the original for loops.

  • Explore the concept of nested list comprehensions in Python. Provide examples of scenarios where nested list comprehensions can be useful, such as flattening lists or creating matrices. Explain the syntax and how it differs from regular list comprehensions. However, also discuss drawbacks of nested comprehensions.

  • Demonstrate the use of conditional statements within list comprehensions. Provide examples of how to filter or modify elements based on specific conditions. Explain the syntax and potential use cases for conditional list comprehensions.

  • Introduce dictionary comprehensions in Python, which work similarly to list comprehensions but create dictionaries instead of lists. Provide examples of how to create dictionaries from lists or other data structures using dictionary comprehensions.

  • Explain set comprehensions in Python, which are used to create sets from other iterables. Provide examples of how set comprehensions can be used to remove duplicates or perform set operations efficiently.

  • Compare the performance of list comprehensions with traditional for loops and other alternatives, such as generator expressions or built-in functions like map() and filter(). Provide benchmarks and analysis to help programmers understand the trade-offs between different approaches.

  • Discuss best practices for using comprehensions in Python, including guidelines for when to use them and when to stick with traditional for loops or other constructs. Address readability concerns and provide tips for writing comprehensions that are easy to understand and maintain.

30.7. Review Questions#

  1. How does a list comprehension differ from a traditional for loop?

  2. What is the syntax for a list comprehension in Python?

  3. What is the purpose of the conditional expression in a list comprehension?

  4. How would you use a dictionary comprehension to create a dictionary from two lists?

  5. What are some best practices for writing readable and maintainable comprehensions in Python?

answers

30.8. Exercises#

  1. Write a list comprehension to create a list of tuples. Each tuple’s first element is the numbers 10,20,30,40,50,60,70,80,90,100. The second value of the tuple is the first number squared

  2. Write a list comprehension that creates a list of numbers from 1 to 30 that are multiples of 3 and divisible by 2

  3. Create an experiment that tests the performance of list comprehensions versus for loops.
    Look at performance testing for a function decorator that times the execution time of a function.

  4. Given a list of dictionaries representing stock data, where each dictionary contains keys for the stock symbol, price, and volume, create a new list containing only the dictionaries for stocks with a price above a certain threshold (e.g., $50). Use a list comprehension with a conditional statement.

    Example input: [{'symbol': 'AAPL', 'price': 120.50, 'volume': 10000}, {'symbol': 'GOOG', 'price': 2500.75, 'volume': 500}, {'symbol': 'META', 'price': 180.25, 'volume': 2000}, {'symbol': 'MSFT', 'price': 280.00, 'volume': 5000}]

    Example output: [{'symbol': 'AAPL', 'price': 120.50, 'volume': 10000}, {'symbol': 'GOOG', 'price': 2500.75, 'volume': 500}, {'symbol': 'MSFT', 'price': 280.00, 'volume': 5000}]

  5. Given a list of currency pairs (e.g., ‘USD/EUR’, ‘GBP/JPY’, ‘EUR/USD’), create a set containing the unique currencies present in the list using a set comprehension.

    Example input: ['USD/EUR', 'GBP/JPY', 'EUR/USD', 'AUD/NZD', 'EUR/GBP']

    Example output: {'USD', 'EUR', 'GBP', 'JPY', 'AUD', 'NZD'}

  6. Given a list of tuples representing loan data, where each tuple contains the loan amount, the interest rate (as a decimal), and the term in years, calculate the total interest owed for each loan using a list comprehension.

    Example input: [(10000, 0.05, 5), (25000, 0.07, 10), (50000, 0.035, 15)]

    Example output: [2500.0, 17500.0, 26250.0]

  7. Given a list of dictionaries representing budget items, where each dictionary contains keys for the item description, amount, and category, create a new dictionary where the keys are the categories, and the values are the total amount budgeted for that category. Use a dictionary comprehension.

    Example input: [{'description': 'Rent', 'amount': 1200, 'category': 'Housing'}, {'description': 'Groceries', 'amount': 500, 'category': 'Food'}, {'description': 'Gas', 'amount': 150, 'category': 'Transportation'}, {'description': 'Utilities', 'amount': 300, 'category': 'Housing'}]

    Example output: {'Housing': 1500, 'Food': 500, 'Transportation': 150}

Contents