Solution · Marketplace Deduplication

Marketplace Image Duplicate Detection and Catalog Deduplication

MediaLayer detects copied seller images, duplicate product listings, and visually similar catalog assets — using fingerprint-based matching that survives re-encoding, watermarks, and minor photo edits.

Check Catalog Duplicates Explore Media Search

The problem

Where this hurts in production

Copied seller images
Sellers copy product photos from brand catalogs, competing sellers, or manufacturer libraries. Re-encoding, cropping, and watermarks break pixel hash checks but not perceptual fingerprinting.
Duplicate product listings
The same product appears under multiple listings with different titles, prices, and seller accounts. Visual media similarity is a more reliable deduplication signal than title or attribute matching.
Visual spam and fraudulent listings
Low-quality sellers flood categories with near-duplicate listings using the same image assets. Content-based matching detects these clusters that metadata-based filters miss.
Catalog quality at scale
Marketplaces with millions of SKUs cannot manually review media duplicates. Automated fingerprint matching groups near-duplicate listings for cleanup or consolidation.

How MediaLayer fits

How MediaLayer fits this workflow

MediaLayer fingerprints product images using dHash and LSH band indexing. Two product photos of the same item — regardless of re-encoding, brightness adjustment, or logo overlay — produce near-identical fingerprints. A similarity search against your indexed catalog returns all matching listings in milliseconds.

Video product demos and audio content are matched using the same pipeline. Frame-level video fingerprinting detects duplicate demo videos across seller accounts. Audio fingerprinting identifies reused voice-over content or brand audio across listings.

The workflow runs on ingest: index your catalog as assets are submitted, then search each new image against the index. Flag high-similarity matches for manual review, consolidate duplicate listings, or auto-reject fraudulent submissions.

Workflow example

From media in to match decision out

1
Index your product catalog
Submit product images, videos, and audio via URL or bulk CSV. MediaLayer fingerprints each asset.
2
Search on ingest
As new listings are submitted, query their media against your indexed catalog using POST /v1/search/{media_type}/url.
3
Get ranked matches
Receive similarity scores, confidence labels, and matched asset IDs for review or automated action.
4
Take catalog action
Consolidate duplicates, flag fraudulent listings, or group near-similar SKUs for catalog cleanup.

POST /V1/SEARCH/IMAGE/URL

# Request
POST /v1/search/image/url
X-API-Key: your-api-key

{
  "url": "https://cdn.example.com/seller-product-photo.jpg",
  "top_k": 5
}

# Response
{
  "query_media_id": "c2d3e4f5-...",
  "results": [
    {
      "media_id": "a9b8c7d6-...",
      "name": "brand-catalog-sku-10042.jpg",
      "similarity_score": 0.98,
      "confidence": "high"
    },
    {
      "media_id": "e5f6g7h8-...",
      "name": "seller-listing-4521.jpg",
      "similarity_score": 0.81,
      "confidence": "medium"
    }
  ]
}

Relevant API endpoints

Drop these into your pipeline

POST /v1/index/image/url

Index product images

Learn more →

POST /v1/index/video/url

Index product videos

Learn more →

POST /v1/search/image/url

Search image against catalog

Learn more →

POST /v1/search/video/url

Search video against catalog

Learn more →

Real-world examples

Patterns we see in this space

Seller image copy detection
Flag new seller listings that reuse product images from brand catalogs or competing sellers.
Duplicate listing consolidation
Group near-duplicate product listings for catalog cleanup and SKU merging.
Visual spam filtering
Detect clusters of listings using the same or near-identical image assets across accounts.
Brand catalog protection
Monitor for unauthorized use of brand-owned product media across third-party seller listings.