How to Remove Duplicates from a List in Python
In this article, we will learn how to remove duplicates from a list in Python. We have a list of names that contains duplicate entries, and our goal is to remove these additional names efficiently. While one approach could be to iterate through the list multiple times and check the frequency of each name, this method can be quite inefficient. Instead, we can use the characteristics of a set to achieve our objective.
First, let’s understand what a set is. Sets in Python are denoted by curly braces. However, be careful not to confuse them with dictionaries, which also use curly braces but have key-value pairs. Sets, on the other hand, only contain individual elements separated by commas.
my_set = {10, 10, 10, 20, 20}
print(my_set)
Output:
{10, 20}
As you can see, even though we initially had three 10s and two 20s in the set, when we printed it, the duplicates were automatically removed. This is one of the main characteristics of sets – they do not allow duplicates.
To confirm that we are working with a set, we can use the type() function:
print(type(my_set)) Output: <class 'set'>
Now, let’s move on to the actual removal of duplicates from our list. We will perform all the necessary steps in the middle section, while using the “before” and “after” sections to illustrate the changes made to the list.
# Before: ['Alice', 'John', 'Alice', 'John', 'Alice'] # After: ['Alice', 'John'] names = ['Alice', 'John', 'Alice', 'John', 'Alice'] # Convert the list to a set to remove duplicates names = set(names) # Convert the set back to a list names = list(names)
By converting the list to a set, we effectively remove the duplicate entries. Then, by converting the set back to a list, we obtain a list without any duplicates.
Let’s run the code and observe the changes:
print(names) Output: ['Alice', 'John']
As you can see, the original list had five elements, but after converting it to a set and then back to a list, we are left with only two unique elements.
Alternatively, we can achieve the same result in a more concise and Pythonic way by combining the set and list conversion into a single line:
names = list(set(names))
Additionally, if you want the resulting list to be sorted, you can use the sorted() function:
names = sorted(list(set(names)))
This will return a sorted list without any duplicates. However, keep in mind that sorting a large list can be computationally expensive.
Conclusion
In conclusion, the easiest way to remove duplicates from a list in Python is by converting the list to a set and then back to a list. This leverages the inherent property of sets to remove duplicates. By following this approach, you can efficiently eliminate duplicate entries from your list.