Tuples, dicts and the toolkit

The last lesson worked on one string at a time. Real questions bring several values at once: a day and its order count, a word and how often it appears, a list that must come out sorted. In SQL you would reach for a join, a GROUP BY with COUNT(*), and an ORDER BY.

Python's answers are tuples, dicts and sorted, plus one-line versions of loops you already write: sum, any, all and list comprehensions. They make interview code short and readable.

Every tool here is a loop you could write yourself, so we show the loop first. That way you always know what a tool does at the edges and what it costs.

#Tuples: several values held as one

A tuple is a fixed group of values, written with commas: pair = (3, 8). You read it like a list (pair[0] is 3), but like a string it cannot be changed. Tuples matter in everyday Python because of one move, unpacking: put several names on the left of = and Python hands them the items in order. low, high = (3, 8) makes low 3 and high 8.

The trace shows the three places you will use this: a function that returns two values, unpacking them, and swapping two names. Guess what the last line prints.

▶1def ends(word):
2    return word[0], word[-1]
3pair = ends("data")
4first, last = pair
5a, b = 3, 8
6a, b = b, a
7print(first, last, a, b)

(nothing printed yet)

a, b = b, a is the Python swap. The right side is finished before any name changes, so you need no spare name like the first that kept the old item safe in lesson 1's swap. Writing it as two separate lines does not swap. Check that you see why:

a = 3
b = 8
a = b
b = a
print(a, b)

A function returns two answers by returning one tuple. return low, high packs them, and the caller unpacks them: low, high = min_max(nums). Write that function now: two notebook values, one walk.

So far: commas make a tuple, and unpacking takes it apart, one name per item. That one move gives you the swap and the two return values, and next, the loops over pairs.

Pairs in a loop: enumerate and zip

Unpacking also works in a for line, and that is where you will use it most. enumerate(nums) hands out (position, item) tuples, so for i, x in enumerate(nums): unpacks each one into i and x. zip(a, b) walks two lists side by side: first the pair of first items, then the pair of second items, stopping at the end of the shorter list. Guess the first line printed by the second loop.

▶1days = ["mon", "tue", "wed"]
2orders = [12, 30, 7]
3for day, count in zip(days, orders):
4    print(day, count)
5for i, day in enumerate(days):
6    print(i, day)

(nothing printed yet)

zip also pairs each item with the next one. Zip a list with itself minus its first item, and the shorter copy decides when the walk stops. Predict it:

orders = [12, 30, 7, 25]
for prev, cur in zip(orders, orders[1:]):
    print(prev, cur, cur - prev)

#A dict is a lookup table

How do you count how often each word appears, when you do not know the words in advance? The last lesson counted letters in a list of 26, because each letter has a fixed position from 0 to 25. Words have no positions, so the notebook needs to be found by the word itself.

A dict (short for dictionary) maps keys to values: give it a key, and it hands back the value stored with it. {} is an empty dict, and {"cat": 2, "dog": 1} is a dict with two keys. Each key appears once and has one value. These five moves cover most interview code, shown on d = {"cat": 2}:

get is the safe read. Its second value is what to hand back for a missing key, and that is exactly what counting needs: a word seen for the first time has a count of 0 so far. Try the moves before the counting loop.

stock = {"apple": 3}
stock["pear"] = 5
stock["apple"] = stock["apple"] - 1
print(stock)
print("kiwi" in stock, stock.get("kiwi", 0), len(stock))

Now the counting walk. The trace counts the words in ["cat", "dog", "cat"], with a dict named counts as the notebook. counts.items() hands out each key with its value, as (key, value) tuples, so the second loop unpacks them the way enumerate did. Guess what the two loops print before you step through.

▶1words = ["cat", "dog", "cat"]
2counts = {}
3for w in words:
4    counts[w] = counts.get(w, 0) + 1
5for w, n in counts.items():
6    print(w, n)

(nothing printed yet)

counts[w] = counts.get(w, 0) + 1 is the counting line. It reads the count so far, using 0 for a word never seen, adds 1, and stores the result back under the same key. The shorter counts[w] += 1 fails on the very first word with a KeyError: it has to read counts["cat"] before it can add 1, and there is nothing there yet.

A dict lookup does not search. "cat" in counts takes about one step on average, however many keys there are, while "cat" in a_list checks the items one by one. Module 3 opens the box: how a dict finds a key without searching, and two tools, Counter and defaultdict, that count in fewer lines.

Now combine the counting walk with lesson 2's running maximum.

#Sorting: a new list, or the same list changed

Python has two ways to sort, and mixing them up is a classic bug. sorted(nums) returns a new sorted list and leaves nums alone. nums.sort() sorts nums itself and returns None, Python's value for "no value", the same None that a function with no return hands back. It is the append trap from lesson 1 again: append and sort both change the list in place and hand back None.

The trace also sorts by a key: a function Python calls on each item, sorting by its results instead of by the items. Before you step through, guess what result and longest hold at the end.

▶1nums = [3, 1, 2]
2ordered = sorted(nums)
3result = nums.sort()
4words = ["join", "a", "etl"]
5by_length = sorted(words, key=len)
6longest = max(words, key=len)

(nothing printed yet)

Pass the key function itself, without parentheses: key=len, not key=len(). Python calls it once per item. Add reverse=True for largest first: sorted(words, key=len, reverse=True) is ["join", "etl", "a"]. min and max take the same key= and return the item. On a tie, sorted keeps the tied items in their original order, even with reverse=True, and min and max return the first tied item they meet.

Predict a sort with a tie in it, in both directions:

words = ["etl", "sql", "a", "join"]
print(sorted(words, key=len))
print(sorted(words, key=len, reverse=True))
print(min(words, key=len), max(words, key=len))

Then the classic bug from the start of this section, in three lines:

nums = [5, 2, 9]
nums = nums.sort()
print(nums)

Sorting is not free: in general it costs more than one pass over the list, and module 2 puts a number on it. Now say the toolkit and the walk side by side, as you would in the room.

#The toolkit: sum, any, all and comprehensions

Each tool in this section is a loop you already know, written in one line. sum(nums) is the running total from lesson 2. The others need a closer look.

A list comprehension is a loop that builds a list, written as one expression. Read [x * x for x in nums if x % 2 == 0] from the middle: for each x in nums, if x is even, keep x * x. The trace runs the loop first, then the comprehension on the same list. Guess what the last line prints.

▶1nums = [3, 4, 1, 6]
2squares = []
3for x in nums:
4    if x % 2 == 0:
5        squares.append(x * x)
6short = [x * x for x in nums if x % 2 == 0]
7print(squares == short, sum(short))

(nothing printed yet)

any and all are the early-return search from lesson 3. Here is the loop behind any:

def has_negative(nums):
    for x in nums:
        if x < 0:
            return True    # one match is enough: stop here
    return False           # walked everything, found none

any(...) is True as soon as one value is true; all(...) is False as soon as one is false. all is the mirror image, the all_positive you put in order in lesson 3: False at the first number that breaks the rule, True only after the loop has checked them all. Both stop early, as the loop does.

Inside a call, a comprehension can drop its square brackets, so has_negative(nums) is any(x < 0 for x in nums), and sum(x * x for x in nums) adds the squares.

Use the one-liner when it reads well at a glance, and the loop when the body needs more than one step. A comprehension with two fors and two ifs is harder to check than the loop it replaces.

nums = [3, -1, 4, -2, 5]
print([x * 2 for x in nums if x > 0])
print(sum(x for x in nums if x < 0))
print(any(x > 4 for x in nums), all(x > 0 for x in nums))