logoalt Hacker News

SkiFire13 • today at 10:49 AM • 1 reply • view on HN

I wonder why you didn't opt to perform a Top-K aggregation in each partition in parallel. That would give you both a reduction of elements in each partition and also the correct maximum of each partition, which can be used for further pruning.

Replies

malisper • today at 3:13 PM

The query is:

  SELECT item_id, COUNT(*) AS c
  FROM orders
  GROUP BY item_id
  ORDER BY c DESC
  LIMIT 10;
They are getting the top 10 most frequent values of item_id. It's possible for one item_id to be the 11th most popular item across every partition and not the 10th. Then when you sum the counts across all partitions, it becomes the 10th most common.