I'll Be Backend
Search lessons / @illbebackend
Principles · Ruby

Imperative to declarative: loops into pipelines

Say what you want from a list, and let the pipeline do the walking.

12 steps4 chapters~5 min at 1.25×#declarative code#collection pipeline#enumerable#loops
Language
JavaKotlinC#C++PythonRubyPHPTypeScriptGoRustSwift+1 soon
The lesson plays right here. Pause anywhere, step back, or jump to a step below. Watch the Ruby video on YouTube Code on GitHub

Watch the narrated video

Every step

Problem

Steps 1–4

Refactor

Steps 5–8

Payoff

Steps 9–10

Recap

Steps 11–12

The code: before and after

Before

Order = Data.define(:customer, :status, :total)

class SalesReport
  def shipped_revenue(orders)
    revenue = 0
    orders.each do |order|
      revenue += order.total if order.status == "SHIPPED"
    end
    revenue
  end

  def top_customers(orders)
    shipped = []
    orders.each do |order|
      shipped << order if order.status == "SHIPPED"
    end
    shipped.sort_by! { |order| -order.total }
    names = []
    shipped.each do |order|
      break if names.size == 3
      next if names.include?(order.customer)
      names << order.customer
    end
    names
  end
end

class DailyDigest
  def self.run(report)
    orders = [
      Order.new("ana@shop.com", "SHIPPED", 120),
      Order.new("qa@test.shop", "SHIPPED", 999),
      Order.new("bo@shop.com", "PENDING", 80),
      Order.new("cy@shop.com", "SHIPPED", 45),
      Order.new("ana@shop.com", "SHIPPED", 60)
    ]
    p report.shipped_revenue(orders)
    p report.top_customers(orders)
  end
end

After

Order = Data.define(:customer, :status, :total)

class SalesReport
  def shipped_revenue(orders)
    orders
      .select { |order| counts_as_sale?(order) }
      .sum(&:total)
  end

  def top_customers(orders)
    orders
      .select { |order| counts_as_sale?(order) }
      .sort_by { |order| -order.total }
      .map(&:customer)
      .uniq
      .first(3)
  end

  private

  def counts_as_sale?(order)
    order.status == "SHIPPED" &&
      !order.customer.end_with?("@test.shop")
  end
end

class DailyDigest
  def self.run(report)
    orders = [
      Order.new("ana@shop.com", "SHIPPED", 120),
      Order.new("qa@test.shop", "SHIPPED", 999),
      Order.new("bo@shop.com", "PENDING", 80),
      Order.new("cy@shop.com", "SHIPPED", 45),
      Order.new("ana@shop.com", "SHIPPED", 60)
    ]
    p report.shipped_revenue(orders)
    p report.top_customers(orders)
  end
end

Transcript, step by step

Problem

Step 1: A sales report in loops Before

This sales report works on a list of orders, and each order has a customer email, a status and a total. Shipped revenue walks the list and adds the total of every shipped order to a running total. Top customers copies the shipped orders into an array, sorts them by total, and walks them again to collect three different customers. The daily digest builds today's orders and prints both numbers. One of those orders comes from a test account at test dot shop, and it's the biggest order of the day.

Step 2: Leave out test orders Code smell

Finance says test orders, from customers at test dot shop, must not count in either number. So the check goes into the condition in shipped revenue, and into the condition in the first loop of top customers. This code is imperative, because it spells out how to walk the list step by step, so you have to replay the loop in your head to learn what it's for. You can see it in the running total here, the temporary shipped array here, and the break and the include check here.

Step 3: The rule, twice Code smell

The rule for what counts as a sale is now written twice, so the next rule has to be found and added twice too. And in top customers, the filter, the ranking, the repeat check and the stop at three are tangled across two loops. So you replay both loops line by line to see which one decides what counts as a sale, and where a new condition belongs.

Step 4: Back it out, then pipelines Plan

Let's back that change out. Declarative code says what you want, so we'll turn each loop into a collection pipeline, a chain of operations where each one has one job, and add the rule after that.

Refactor

Step 5: Shipped revenue as one chain Replace Loop with Pipeline

Shipped revenue is now one pipeline on orders, a select followed by a sum. Select keeps the shipped orders, and sum takes each order's total and adds them up. The running total is gone, because sum keeps that total for you, and the digest still prints twelve twenty-four.

Step 6: Filter and sort in one chain Replace Loop with Pipeline

In top customers, the first loop and the sort become one chain on orders. Select keeps the shipped orders, and sort by ranks them by total, with the minus sign putting the biggest first. Nobody pushes onto a temporary array by hand anymore, and the second loop still reads from shipped, so the output stays the same.

Step 7: The second loop joins Replace Loop with Pipeline

Now the second loop joins the same pipeline. Map takes each order's customer, uniq drops the repeats, and first with a three keeps the top three names. The output doesn't change, because uniq keeps each name where it first appears, as the include check did, and first keeps three names, as the break did.

Step 8: Name the sale rule Extract Method

The status test now sits in two select blocks, so we extract it into one private method called counts as sale. Both pipelines call it by name, first in shipped revenue and then in top customers. Now the diagram shows both methods asking one place what a sale is.

Payoff

Step 9: Test orders, one edit Payoff

Now Finance's rule goes in again, and this time it's one edit inside counts as sale. Neither pipeline changed, because both of them ask counts as sale and don't care what it checks. So the digest now prints two twenty-five, with Ana and Cy as the top customers.

Step 10: Why it was easy Payoff

The running total became sum here, the temporary array became select and sort by here, and the break and the include check became uniq and first here. Top customers now gives each job its own line, so you can see at a glance that a new sale rule belongs in the select. Giving that rule one home for both pipelines came from Extract Method, and the pipelines made the spot easy to find.

Recap

Step 11: Where we started, where we ended Before / after

On the left, shipped revenue keeps a running total, and top customers fills a temporary array, checks for repeats, and breaks at three. On the right, each method is one pipeline, and each line in it does one job. And counts as sale holds the rule for both of them, so Finance's next rule is one edit too.

Step 12: When to reach for it Takeaways

Reach for a pipeline when a loop filters, transforms or totals a collection, and you want each job readable on its own line. Skip it when each step depends on the one before, like a running balance that stops at the first overdraft, because the loop says that more plainly than a reduce. You'll spot it by a temporary array filled in a loop, a running total, or a break or flag that stops early. Ruby's any and all methods already work this way, because they take a block and stop at the first answer, the way a flag and a break do.