How to remove duplicate items in an array

If you would like to remove duplicate items in an array you have a few options.

For an array of primitives, you can use a Set.

const arr = [1, 2, 3, 1, 2, 3];
const unique = [...new Set(arr)]; // [1, 2, 3]

A Set lets you store unique values of any types, including primitives and objects which share the same reference. Here we use a Set to get the unique values from the array, and then spread it back into a new array.

A limitation of a Set is that it will only unique objects which are the same in reference but not necessarily by reference. Say we have 2 objects which are identical by their properties, but are assigned to different variables. If we log their equality, our environment will return false.

const a = { a: '1', b: '2' };
const b = { a: '1', b: '2' };
console.log(a === b); // false

Unfortunately a Set will not be able to unique objects like this.

const arr = [{ a: '1' }, { a: '1' }, { a: '1' }, { a: '1' }];
const unique = [...new Set(arr)];
console.log(unique); // [ { a: '1' }, { a: '1' }, { a: '1' }, { a: '1' } ]

So what can we do? Well, one option is to use a Map or an object literal. Both Maps and objects literals require key value pairs, with each key being unique. This allows us to effectively build up an object with unique keys value pairs and we can then return the values in an array. When I tested the performance of these 2, the difference was negligible.

Another option we have is to use a Set. As discussed earlier, we can use a Set to unique items in array as long as they are either primitives, or are the same by reference. However, with object literals, we can turn them into a string primitive using JSON.stringify and then add them to our Set. Our Set will automatically remove any duplicate entries. The only issue we have now is that the values of the Set will be string representations of the object literals we passed in. We can fix this by mapping over the values and calling JSON.parse to get the object literal back out.

So there you have it. A few different ways to remove duplicate items in an array.