Complete Guide to Filtering Duplicate Results with AngularJS ng-repeat

Problem Background and Challenges

In AngularJS application development, the ng-repeat directive is a core tool for handling list data. However, when the data source contains duplicate items, directly using ng-repeat results in redundant element output. For example, when processing JSON data containing category information, each category may correspond to multiple data items, but only unique category options need to be displayed in the interface.

The original code example demonstrates this issue:

<select ng-model="orderProp">
  <option ng-repeat="place in places" value="{{place.category}}">{{place.category}}</option>
</select>

This code generates an <option> element for each place object, even when multiple objects belong to the same category. For a dataset containing 100 objects but only 6 unique categories, this produces 94 redundant options, significantly impacting user experience and performance.

AngularUI Unique Filter Solution

The AngularUI project provides a specialized unique filter, which represents the best practice for handling duplicate data filtering. This filter identifies and removes duplicates by comparing values of specified properties.

Implementation steps:

1. First, import the AngularUI library or individually import the unique filter module
2. Apply deduplication logic using filter syntax in templates

Improved code example:

<select ng-model="orderProp" ng-options="place.category for place in places | unique:'category'">
  <option value="0">Default Option</option>
</select>

Here, unique:'category' specifies deduplication based on the category property. The combination of ng-options directive with the filter automatically generates a list of unique category options.

Custom Filter Implementation

When AngularUI is not available, custom filters can be created. Using Lodash library's _.uniq function simplifies implementation:

app.filter('unique', function() {
  return function (arr, field) {
    return _.uniq(arr, function(a) { 
      return a[field]; 
    });
  };
});

This custom filter accepts an array and field name as parameters, returning a new array with duplicates removed based on the specified field. Usage in templates is identical to the AngularUI filter.

Third-party Module Solutions

The angular-filter module provides more powerful deduplication capabilities, supporting nested property filtering:

<tr ng-repeat="order in orders | unique: 'customer.id'">
  <td>{{ order.customer.name }} , {{ order.customer.id }}</td>
</tr>

This solution is particularly suitable for handling complex object structures where duplicate detection needs to be based on nested property values.

Performance Analysis and Best Practices

When dealing with large datasets, filter performance is crucial:

• AngularUI filters are optimized for most application scenarios
• Custom filters can employ more efficient algorithms, such as using Set data structures for deduplication
• For static data, consider preprocessing data in controllers to avoid repeated calculations in each digest cycle

Preprocessing example:

$scope.uniquePlaces = _.uniq($scope.places, 'category');

Compatibility and Considerations

Although official AngularJS support has ended, these solutions remain effective in existing projects. When migrating to newer Angular versions, similar filtering functionality can be implemented through Pipes. During development, attention should be paid to:

• Ensuring filter modules are correctly loaded
• Handling potential data changes using $watch to monitor data updates
• Considering track by usage to optimize ng-repeat performance