Pop quiz: how many uses are there in Python for the in keyword?

  • Checking membership (x in s)
  • Iteration (for x in s: ...)
  • Comprehension ([x for x in s])

Let’s see, is that all of them? It doesn’t matter, because it was a TRICK QUESTION. Python doesn’t have one in keyword, it has four. Check out the CPython Grammar file:

for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]

comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'

sync_comp_for: 'for' exprlist 'in' or_test [comp_iter]

in is used separately as a helper keyword in for statements and list comprehensions, and there is the in binary operator, and the not in binary operator, which is distinct from in. Thus x not in s and not x in s, though equivalent semantically, are not the same expression: the former is the application of the binary operator not in to x and s, while not x in s in the application of the unary operator not to the result of applying the binary operator in to x and s.

It might seem like an academic question whether two identical keywords are really the same or not, but as a practical consequence it means one can be changed without affecting the other. Suppose, for example, that we wanted to use belongingto as the helper keyword for iteration, but memberof for comprehension (I don’t know why we would want that, but stick with me here). To do this, we merely need to modify the grammar file entries separately, keeping the old in helpers for backwards compatibility:

modified   Grammar/Grammar
@@ -74 +74 @@ while_stmt: 'while' test ':' suite ['else' ':' suite]
-for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
+for_stmt: 'for' exprlist ('in' | 'belongingto') testlist ':' suite ['else' ':' suite]
@@ -142 +142 @@ comp_iter: comp_for | comp_if
-sync_comp_for: 'for' exprlist 'in' or_test [comp_iter]
+sync_comp_for: 'for' exprlist ('in' | 'memberof') or_test [comp_iter]

To build a Python interpreter with these new keywords1, run make regen-grammar && make -j. The first command regenerates a file called Python/graminit.c based on the modifed grammar file, and the second builds the whole thing. Then run the interpreter and witness magic before your eyes.

$ ~/cpython/python
Python 3.8.0a0 (heads/belongingto-memberof-dirty:062433f341, Aug  4 2018, 16:36:27)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> s = [1, 2, 3]
>>> for x in s: print(x**2)
...
1
4
9
>>> for x belongingto s: print(x**2)
...
1
4
9
>>> for x memberof s: print(x**2)
  File "<stdin>", line 1
    for x memberof s: print(x**2)
                 ^
SyntaxError: invalid syntax
>>> [x**2 for x in s]
[1, 4, 9]
>>> [x**2 for x memberof s]
[1, 4, 9]
>>> [x**2 for x belongingto s]
  File "<stdin>", line 1
    [x**2 for x belongingto s]
                          ^
SyntaxError: invalid syntax

Footnotes

1 Building the interpreter requires that you have a copy of the CPython repo. You can use the official one or my fork of the CPython repo with these changes.