Python Filtering

1. Python Filter Function

It takes a function and an iterable as arguments. filter() will invoke the function on each element of the iterable, and return a new iterable composed of only those elements for which the function returned True.

The return type depends on the type of the iterable passed in. If the iterable is either a string or a tuple, the return type will reflect the input type. Otherwise, the filter function will always return a list.

2. Python Filter with Number

Lets look at a couple of examples. We’ll work in the Python intrepreter.

First let’s make a list of numbers:

>>> numbers = [1, 6, 3, 8, 4, 9]

Next, we’ll define a function to act as our criteria to filter on:

>>> def lessThanFive(element):
... return element < 5
...

Notice that in order to work properly, our criteria function must take a single argument (filter() will call it on each element of the iterable, one at a time). Using our newly defined lessThanFive() criteria function, we expect filter() to return a list of only the elements with a value of less than 5. and that’s exactly what we get.

>>> filter(lessThanFive, numbers)
[1, 3, 4]

3. Python Filter with String

Let’s look at another example. This time, we’ll make a tuple of names:

>>> names = ('Jack', 'Jill', 'Steve', '')

Notice that the last name in the tuple is the empty string. If None is handed as the first argument to filter(), it falls back to use the identity function (a function that simply returns the element).

Since Python evaluates empty strings, 0’s and None’s as False in a boolean context, this default behaviour can be useful to remove the ‘empty’ elements of a literable:

4. Python Filter with a Function

I check the length of the name first to ensure that there is in fact a first character.

We should expect to get back a tuple containing only ‘Jack’ and ‘Jill’ when we use this as our criteria function:

>>> filter(startsWithJ, names)
('Jack', 'Jill')

Again, notice that the return type is a tuple, and not a list.

For more information about the filter function, type help(filter) in a python interpreter to see the man page, or browse the online python filter docs.

List Comprehension

Another way to approach this idiom lists is to use a list comprehension. Essentially, a list comprehension is a compact for-loop that builds lists. Each iteration, an element can be appended to list being built. The syntax is:

5. Basic List Comprehension Usage

[ <output value> for <element> in <list> <optional criteria> ]

This looks like a lot so lets start off with a simple example. Remember the ‘numbers’ list we defined earlier?

>>> numbers
[1, 6, 3, 8, 4, 9]

we can use a list comprehension to simply ‘echo’ the list:

>>> [ num for num in numbers ]
[1, 6, 3, 8, 4, 9]

6. Process List as For Loop

Let’s break this expression down:

The middle part of the comprehension, ‘for num in numbers’, looks exactly like a for loop. It tells us that we are iterating over the ‘numbers’ list, and binds the name ‘num’ to the current element each iteration.

The leading ‘num’ tells us what to append to the list we are building. We could have just as easily said: to build a list with each element of ‘numbers’ doubled.

>>> [ num * 2 for num in numbers ]
[2, 12, 6, 16, 8, 18]

7. If Condition in Python List

Let’s add a criteria to recreate what we did with the built-in filter function earlier:

>>> [ num for num in numbers if num < 5 ]
[1, 3, 4]

This time, we’re still building our new list from the original values, but we only append an original value to the new list if it satisfies the criteria (‘if num < 5’). I hope you can start to see the similarities between comprehensions and filtering.

8. Python List like an Array

Our intentions don’t always have to be to create a new list when using list comprehensions though. We can make use of side-effects, carrying out behaviour on a subset of a list all in one step. Let’s look a slightly less trivial example. Lets say we have a list of customers:

this function is only a stub. If we were doing this for real, the logic to send an email would go here, but in our example, we simply print out to show that the function was called for a given customer. Notice that emailUpdate does not return a customer, but instead carries out behaviour using one.

9. Function, If condition and For loop in Python List

Now, we have everything we need to use a list comprehension to send out our notifications:

There are similar comprehension syntaxes for Dictionaries, tuples, and even sets. To read more about list comprehensions, visit the offical python lists docs.

I hope you’re getting familiar with the list comprehension syntax, but let’s break this one down:

The middle of the comprehension, ‘for customer in customers’, tells us that we are iterating over the list ‘customers’. It also tells us that the current list element for each iteration will be bound to the variable ‘customer’.

The start of the comprehension, ’emailUpdate(customer)’, tells us that we will invoke the emailUpdate function on each customer (this is where the side-effect behaviour is important), and then append its return value to the list we are building.

The end of the comprehension, ‘if subscribesForUpdates(customer)’, says to only execute the emailUpdate function if the condition is true (i.e. if the customer subscribed for emails).

From the output, we see that the emailUpdate function was indeed only invoked on Jack (as Jill does not subscribe for emails).

We also see that the return value from the comprehension was [None]. Why is this?

Remember that list comprehensions build lists. This time, we are building a list using the return of emailUpdates().

Since emailUpdates() doesn’t explicitly return a value, Python will make it implicitly return None. Since only Jack subscribed for email updates, the emailUpdates() function was only called once, and only 1 None was appended to the initially empty list that the comprehension built for us.

In the little python, I’ve written, I have not used filter() or list comprehension. I’ll add that to my bag of tricks. I found it useful to follow along with your examples using idle. That way I could play with variations of the examples. Good article – thanks.

Hi, good article. But I have a few recommendation for you.
First of all, read PEP 8, especially this .
The second one, you should prefer list comprehension over ‘filter’ and ‘map’ functions.
Another thing, use ‘startswith’ method of str instead of your own function.

sorry but my copy and pastes above do not show the entire things that I pasted in.
not sure why but many lines were not pasted in, resulting in blank lines where there is suppoessed to be output frok IDLE. Anyways it just show in those areas… not sure why it prints the object and not the results as in your post.

Bill, the filter() function works differently in Python 3.x (the author used Python 2.x).
In Python 2, filter() returns a list, tuple or string depending on what you used as argument, but in Python 3 it constructs an iterator.
That means you can use a ‘for loop’ to print out all the values in the filter object
or you can call the list() or tuple() function to create a list or tuple: list(filter_object). Note that the filter object is empty after you do that. You could also use a generator comprehension (just like a list comprehension but with parentheses () ) instead of filter().

About The Geek Stuff

My name is Ramesh Natarajan. I will be posting instruction guides, how-to, troubleshooting tips and tricks on Linux, database, hardware, security and web. My focus is to write articles that will either teach you or help you resolve a problem. Read more about Ramesh Natarajan and the blog.

Contact Us

Email Me :
Use this Contact Form to get in touch me with your comments, questions or suggestions about this site. You can also simply drop me a line to say hello!.