Find duplicate items in a Python list

This post will discuss how to find duplicate items in a list in Python.

1. Using `index()` function

A simple solution is to get iterate through the list with indices using list comprehension and check for another occurrence of each encountered element using the index() function. The time complexity of this solution would be quadratic, and the code does not handle repeated elements in output.

if __name__ == '__main__':

nums = [1, 5, 2, 1, 4, 5, 1]

dup = [x for i, x in enumerate(nums) if i != nums.index(x)]

print(dup) # [1, 5, 1]

Download Run Code

2. Using In operator

Alternatively, you can use slicing with the in operator to search in the already visited portion of the list. The time complexity of the solution remains quadratic and allows repeated elements in the output.

if __name__ == '__main__':

nums = [1, 5, 2, 1, 4, 5, 1]

dup = [x for i, x in enumerate(nums) if x in nums[:i]]

print(dup) # [1, 5, 1]

Download Run Code

3. Using Set (Efficient)

To improve performance and get the work done in linear time, you can use the set data structure.

if __name__ == '__main__':

nums = [1, 5, 2, 1, 4, 5, 1]

visited = set()

dup = [x for x in nums if x in visited or (visited.add(x) or False)]

print(dup) # [1, 5, 1]

Download Run Code

To get each duplicate only once, you can use the set comprehension, as shown below:

if __name__ == '__main__':

nums = [1, 5, 2, 1, 4, 5, 1]

visited = set()

dup = {x for x in nums if x in visited or (visited.add(x) or False)}

print(dup) # {1, 5}

Download Run Code

4. Using `count()` function

Here’s an alternate solution using the count() function, which provides a simple, clean way to identify duplicates in a list. This is not recommended for large lists as the time complexity is quadratic.

if __name__ == '__main__':

nums = [1, 5, 2, 1, 4, 5, 1]

dup = {x for x in nums if nums.count(x) > 1}

print(dup) # {1, 5}

Download Run Code

5. Using `iteration_utilities` module

Finally, the iteration_utilities module offers the duplicates function, which yields duplicate elements. You can use this as:

from iteration_utilities import duplicates

if __name__ == '__main__':

nums = [1, 5, 2, 1, 4, 5, 1]

dup = list(duplicates(nums))

print(dup) # [1, 5, 1]

To get each duplicate only once, combine it with unique_everseen():

from iteration_utilities import unique_everseen

if __name__ == '__main__':

nums = [1, 5, 2, 1, 4, 5, 1]

dup = unique_everseen(duplicates(nums))

print(dup) # [1, 5]

That’s all about finding the duplicate items in a list in Python.

Also See:

Remove duplicate values from a Python list

Find duplicate items in a Python list

1. Using index() function

2. Using In operator

3. Using Set (Efficient)

4. Using count() function

5. Using iteration_utilities module

Share this:

1. Using `index()` function

4. Using `count()` function

5. Using `iteration_utilities` module