Advanced Python: Comprehensions - 03/02/2023

Powerful one-liners for generators in python3

Site under construction! Migrated content might be displayed ugly!

medium-to-markdown@0.0.3 convert node index.js https://medium.com/@birnadin/advanced-python-comprehensions-c6914520323a

Advanced Python: Comprehension to save code

[ Enigma Bits

](https://medium.com/@birnadin?source=post_page-----c6914520323a--------------------------------)

Enigma Bits

6 min read·Feb 2, 2023

—

Listen

by end, what you should be able to

Ever got annoyed when you want a list of items from another sequence but had to write a for/while loop? Did you know you could have done it in one line? I am not just taking about list, you can do this trick with any standard sequence like dictionary, set and any custom generators.

Executable Codesandbox…

Comprehension, you say?

Comprehension in python is just a syntactic sugar for a loop and .append() statement.

Let’s say we have database query of blog posts with attribute of publish_at, by design this is a way this system supports scheduled blog post and if current datetime is latest than this, then response will contain filtered blog post items.

from datetime.datetime import now  
  
blogs\_filtered = \[  
  (blog.title(), blog.category()) # what to save  
  for blog in blogs               # what to iterate  
  if blog.publish\_at <= now()     # any filtering condiitonal  
\]

As you can see, we iterate over query results, blogs and if publish_at < now() i.e. if publish_at is latest than now() we save the query’s title & category. If we didn’t use comprehensions, then the snippet would be like,

from datetime.datetime import now  
  
blogs\_filtered = \[\]  
for blog in blogs:                    # what to iterate  
  if blog.publish\_at <= now():      # any filtering condiitonal  
    blogs\_filtered.append(  
     (blog.title(), blog.category())  # what to save  
    )

See? Comprehensions saves you lines, which means less byte-code to process thus comparatively faster execution; less mental power to comprehend what the snippet does and also can be written in one line 😎.

Not just list but _dict_s also. E.g., we have 2 lists one should be key and other should be its corresponding value, given that the position of these items are already in place.

keys = \[...\] # some keys  
vals = \[...\] # some values  
  
assert(len(keys) == len(vals)) # just in case  
  
\# dict comprehension  
dict\_needed = {k:v for k,v in zip(keys,vals)}

If not for comprehensions, then you would have gone like this:

keys = \[...\] # some keys  
vals = \[...\] # some values  
  
assert(len(keys) == len(vals)) # just in case  
  
\# already seen for loop, thus let's use while loop  
i = len(keys)  
dict\_needed = {}  
while i:  
  dict\_needed\[keys\[i\]\] = vals\[i\]  
  i -= 1

Which one is concise? Let me know in the comments. Let’s examine the syntax.

The Syntax

List Comprehension

list\_out = \[element for element in sequence if condition\]

Here, sequence can be any generator or iterator and element is each element in the sequence; and it is committed to the output is condition evaluates to True.

src/list-comp.py from the codesandbox.

Dictionary Comprehension

dict\_out = {k:v for k,v in dict if condition}

Same as list comprehension but using tuple unpacking, we can separate the key and value of each record and operate on them; conditional block operates the same though.

src/dict-comp.py from the codesandbox.

Set Comprehension

set\_out = {i for i in sequence if condition}

Looks like combination of list and dictionary, eh? Functionality is the same.

So

Comprehension in a nutshell being: -

operation on el,
an iterator lexemes, for el in els,
a conditonal, if condition, that filters out commition of el based on returned boolean (optional),
surrounded by either [] or {} to indicate what datatype to produce.

Complex Patterns

Creating hashmap of titles and their category

Assume, we have list of Blogs whose title have to be indexed according to corresponding category. If Blog is defined as…

class Blog:  
  title: str  
  category: str  
  # other attributes  
      
  def \_\_init\_\_(self, t, c):  
    self.title = t  
    self.category = c  
      
  def \_\_repr\_\_(self):  
    return self.title.lower()

And let’s create a helper function to generate n no. of blogs.

def \_gen\_blogs(n):  
    from random import randint  
    return \[Blog('blog title'+str(i), str(randint(0,100))) for i in range(1,n+1)\]

You can clearly tell that list comprehension is used. This is a fairly good example when a syntactic sugar being more of a pain in the finger. When I am faced with such a gibberish I used to format it in 3+ lines based of the abstract lexemes, see code-block-1.

def \_gen\_blogs(n):  
  from random import randint  
  return \[  
    Blog(  
      'blog title'+str(i), str( randint(0,100) )  
    )                       # what to commit  
    for i in range(1,n+1)   # what to iterate  
                            # no conditional  
  \]

Right away a bad practice of defining import statement in the local function. Then readability being shot down by comprehensions.

If we disect the return statement, we are iterating to arithmetic progression of common difference 1, i.e. 1,2,3,...,n-2,n-1,n. Each iteration i takes each value. Then for each iteration, a Blog instance is created with title of blog title {i}, e.g. for 3rd iteration title will be blog title 3; And each instance will have the category of a pseudo-random integer generated in between 0 and 100 (exclusive).

Then generation of map will be…

def map\_title\_to\_category():  
    blogs = \_gen\_blogs(4) # generate 4 blogs, play with 4 if you want 😁  
    out = {blog.title:blog.category for blog in blogs} # mapping  
    map\_result(blogs, out)

Try to understand the block,

out = {blog.title:blog.category for blog in blogs}

Hope you can, it ain’t much. If you are stuck, use the comments, I will reply.

You can actually chain them.

def alpha\_pos():  
    m = {a:p for a,p in zip(\[chr(i) for i in range(97,123)\], \[p for p in range(1, 27)\]) if a not in "a.e.i.o.u".split(".")}  
    map\_result(\[\[chr(i) for i in range(97,123)\], \[p for p in range(1, 27)\]\], m) # just for pretty output
```src/chain-comp.py

It is the chain that makes it hard to read, not the logic. Try to interpret on your own, and if you need help, I am just a comment away. _Hint: make each comprehensions a separate variable and analyze_.

With a pinch of salt & a punch of lime
--------------------------------------

To be efficient, let’s save user configuration in _bitmask_ instead of _Json_ etc.

> If you think _JSON_ is better way to save data, the watch this [**short video**](https://www.youtube.com/watch?v=MuCK81q1edU).

By design we are saving 4 toggles (but you can go wild) with 4 bits, each being,

dark-mode data-saver-on auto-play auto-update
1|0 1|0 1|0 1|0


Altogether there are 16 permutations. If we are doing analysis, where we need to decide whether we keep data-saver algorithm, you are tasked with finding no of users, you can go about doing this as,

users = [] # our specimen
num = len([u for u in users if u.conf | 0b0100]) # extremly simplified for brevity


In another scenario, say we are changing our database design and decided to not let users have period `.` in the end of the _username_. To save the day and show dominance let’s use a one-liner even though we could have gone with readable script of poly-liner 😎.

# Lazy loading the billion users
def get_users():
i = 0
while i < 3 * 10**12:
# this is mapped in the data presentation layer,
# not that we are indexing off of a list
yield User[i]
i += 1

for user in get\_users():  
  user.uname = user.uname if user.uname\[-1\] != '.' else user.uname\[:-1\]  
  user.commit() # or whatever
```

Can be…

```
\_ = map(lambda u: u.commit(), \[lambda u: u.uname\[:-1\] if u.uname\[-1\] == '.' else u.uname for user in (User\[i\] for i in range(3 \* 10\*\*12 + 1))\])
```

…like this.

now show dominance 😎.

Epilogue
========

That’s all I can think of. If you are intrigued and interested, make sure you follow me on [Medium](https://www.medium.com/@birnadin) or [Twitter](https://twitter.com/birnadin) for follow up.

See you in another one, till then **_it’s me the BE,_** signing off 👋.

Cover background by [**Jan Karan**](https://www.pexels.com/@jan-karan/)  & Code to PNG using  [**_Ray.so_**](https://ray.so)