Python Program To Remove Duplicate Elements in a List

To remove duplicate elements in the Python list, we created several programs that remove the duplicate elements and produce a list of unique elements. Elements that appear more than once in the list are called duplicate elements.

Since Python list allows to store duplicate elements and in some cases we don't want to store duplicate elements then we need to create a list of unique elements.

In this post, we have certain examples in which we used our own logic to remove duplicate and some built-in features of Python language.

Python Program To Get List of Non Duplicate (unique)  Elements After Removing Duplicate Elements

Here we are getting list of unique elements after removing duplicate elements. In the below example, we created a new list to store unique elements by using append() method of Python list. 

# Take a list
list = [2,3,6,8,11,2,5,6,8,4,3]
# Take an empty list to store non duplicate (unique) elements
newList = [];
# Traverse the list elements
for element in list:
		# Check for the element
		if element not in newList:
			# Add element
			newList.append(element)
# Print list of unique elements
print(newList)

Output:

[2, 3, 6, 8, 11, 5, 4]
 

Python Program: Remove Duplicate Using Set

Set is a data structure which is used to store unique elements. We can use the set to make a list unique. In this example, we are storing list elements to the set and then convert the set to list, so that we can get all the unique elements in the list. This is another way by which we can remove duplicate elements from the python list.

# Take a list
listItems = [2,3,6,8,11,2,5,6,8,4,3]
# Print the list
print("List of duplicate items:\n",listItems)
# Take an empty set to store non duplicate (unique) elements
setA = set(); # Set does not store duplicate elements
# Store list elements into set
set1 = set(listItems)
# Convert set to list
listItems = list(set1)
# Print list of unique elements
print("List of unique items:")
print(listItems)

Output:

List of duplicate items:
[2, 3, 6, 8, 11, 2, 5, 6, 8, 4, 3]
List of unique items:
[2, 3, 4, 5, 6, 8, 11]
 

Python Code: Remove Duplicate Using Dictionary

This is another way by which we can remove duplicate elements of the list. Since dictionary stores unique keys then we can use dictionary to store list elements as keys and then get beck all the keys elements to the list. See the below python program in which we used fromkeys() method of dictionary that will create unique keys.

# Take a list
listItems = [2,3,6,8,11,2,5,6,8,4,3]
# Print the list
print("List of duplicate items:\n",listItems)
# Take an empty dictionary to store non duplicate (unique) elements
dictA = dict(); # Dictionary does not store duplicate keys
# Store list elements into dictionary as keys
newDict = dictA.fromkeys(listItems)
# Convert dictionary to list
listItems = list(newDict)
# Print list of unique elements
print("List of unique items:")
print(listItems)

Output:

List of duplicate items:
 [2, 3, 6, 8, 11, 2, 5, 6, 8, 4, 3]
List of unique items:
[2, 3, 6, 8, 11, 5, 4]

Conclusion

Well, in this topic, we learnt to write programs to remove duplicate elements from the list and get list of unique elements. We used set and dictionary to remove duplicate elements and store back the unique elements to the list.