Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Share anonymised sales data on DFC API with authorised users #12831

Merged
merged 5 commits into from
Sep 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# frozen_string_literal: true

module DfcProvider
# Aggregates anonymised sales data for a research project.
class AffiliateSalesDataController < DfcProvider::ApplicationController
rescue_from Date::Error, with: -> { head :bad_request }

def show
person = AffiliateSalesDataBuilder.person(current_user, filter_params)

render json: DfcIo.export(person)
end

private

def filter_params
{
start_date: parse_date(params[:startDate]),
end_date: parse_date(params[:endDate]),
}
end

def parse_date(string)
return if string.blank?

Date.parse(string)
end
end
end
17 changes: 17 additions & 0 deletions engines/dfc_provider/app/services/affiliate_sales_data_builder.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# frozen_string_literal: true

class AffiliateSalesDataBuilder < DfcBuilder
class << self
def person(user, filters = {})
data = AffiliateSalesQuery.data(user.affiliate_enterprises, **filters)
suppliers = data.map do |row|
AffiliateSalesDataRowBuilder.new(row).build_supplier
end

DataFoodConsortium::Connector::Person.new(
urls.affiliate_sales_data_url,
affiliatedOrganizations: suppliers,
)
end
end
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# frozen_string_literal: true

# Represents a single row of the aggregated sales data.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So glad you did this part, it probably would have taken me ages to figure out all the required relations.

class AffiliateSalesDataRowBuilder < DfcBuilder
attr_reader :item

def initialize(row)
super()
@item = AffiliateSalesQuery.label_row(row)
end

def build_supplier
DataFoodConsortium::Connector::Enterprise.new(
nil,
localizations: [build_address(item[:supplier_postcode])],
suppliedProducts: [build_product],
)
end

def build_distributor
DataFoodConsortium::Connector::Enterprise.new(
nil,
localizations: [build_address(item[:distributor_postcode])],
)
end

def build_product
DataFoodConsortium::Connector::SuppliedProduct.new(
nil,
name: item[:product_name],
quantity: build_product_quantity,
).tap do |product|
product.registerSemanticProperty("dfc-b:concernedBy") {
build_order_line
}
end
end

def build_order_line
DataFoodConsortium::Connector::OrderLine.new(
nil,
quantity: build_line_quantity,
price: build_price,
order: build_order,
)
end

def build_order
DataFoodConsortium::Connector::Order.new(
nil,
saleSession: build_sale_session,
)
end

def build_sale_session
DataFoodConsortium::Connector::SaleSession.new(
nil,
).tap do |session|
session.registerSemanticProperty("dfc-b:objectOf") {
build_coordination
}
end
end

def build_coordination
DfcProvider::Coordination.new(
nil,
coordinator: build_distributor,
)
end

def build_product_quantity
DataFoodConsortium::Connector::QuantitativeValue.new(
unit: QuantitativeValueBuilder.unit(item[:unit_type]),
value: item[:units]&.to_f,
)
end

def build_line_quantity
DataFoodConsortium::Connector::QuantitativeValue.new(
unit: DfcLoader.connector.MEASURES.PIECE,
value: item[:quantity_sold]&.to_f,
)
end

def build_price
DataFoodConsortium::Connector::QuantitativeValue.new(
value: item[:price]&.to_f,
)
end

def build_address(postcode)
DataFoodConsortium::Connector::Address.new(
nil,
postalCode: postcode,
)
end
end
90 changes: 90 additions & 0 deletions engines/dfc_provider/app/services/affiliate_sales_query.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# frozen_string_literal: true

class AffiliateSalesQuery
class << self
def data(enterprises, start_date: nil, end_date: nil)
end_date = end_date&.end_of_day # Include the whole end date.

Spree::LineItem
.joins(tables)
.where(
spree_orders: {
state: "complete", distributor_id: enterprises,
completed_at: [start_date..end_date],
},
)
.group(key_fields)
.pluck(fields)
end

# Create a hash with labels for an array of data points:
#
# { product_name: "Apple", ... }
def label_row(row)
labels.zip(row).to_h
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was worried about indiscriminately zipping two arrays together, so I was wondering if there's a way to map by the column names.
There's discussion here about avoiding instantiating AR records, which you've rightly done:
https://stackoverflow.com/questions/20794398/how-can-i-have-activerecords-pluck-also-return-the-column-name-for-rendering-in

I like that your solution creates a short-lived hash to label the columns on-demand.

One person suggests using ActiveRecord::Base.connection.select_all which provides a ActiveRecord::Result. I haven't looked at the implementation of that class, but I suspect it does the same thing, so is probably equally efficient.

But it's probably not worth exploring further, moving on...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't know select_all. I'll keep that in mind for another time. If it could do it in batches then I would probably prefer that. But my version occupies the least memory for grabbing all data at once, I think, because it's only keeping the values in an array and the keys are only added while processing one row, as you acknowledged above.

end

private

# We want to collect a lot of data from only a few columns.
# It's more efficient with `pluck`. But therefore we need well named
# tables and columns, especially because we are going to join some tables
# twice for different columns. For example the distributer postcode and
# the supplier postcode. That's why we need SQL here instead of nice Rails
# associations.
def tables
<<~SQL.squish
JOIN spree_variants ON spree_variants.id = spree_line_items.variant_id
JOIN spree_products ON spree_products.id = spree_variants.product_id
JOIN enterprises AS suppliers ON suppliers.id = spree_variants.supplier_id
JOIN spree_addresses AS supplier_addresses ON supplier_addresses.id = suppliers.address_id
JOIN spree_orders ON spree_orders.id = spree_line_items.order_id
JOIN enterprises AS distributors ON distributors.id = spree_orders.distributor_id
JOIN spree_addresses AS distributor_addresses ON distributor_addresses.id = distributors.address_id
SQL
end

def fields
<<~SQL.squish
spree_products.name AS product_name,
spree_variants.display_name AS unit_name,
spree_products.variant_unit AS unit_type,
spree_variants.unit_value AS units,
spree_variants.unit_presentation,
spree_line_items.price,
distributor_addresses.zipcode AS distributor_postcode,
supplier_addresses.zipcode AS supplier_postcode,

SUM(spree_line_items.quantity) AS quantity_sold
SQL
end

def key_fields
<<~SQL.squish
product_name,
unit_name,
unit_type,
units,
spree_variants.unit_presentation,
spree_line_items.price,
distributor_postcode,
supplier_postcode
SQL
end

# A list of column names as symbols to be used as hash keys.
def labels
%i[
product_name
unit_name
unit_type
units
unit_presentation
price
distributor_postcode
supplier_postcode
quantity_sold
]
end
end
end
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@
class QuantitativeValueBuilder < DfcBuilder
def self.quantity(variant)
DataFoodConsortium::Connector::QuantitativeValue.new(
unit: unit(variant),
unit: unit(variant.product.variant_unit),
value: variant.unit_value,
)
end

def self.unit(variant)
case variant.product.variant_unit
def self.unit(unit_name)
case unit_name
when "volume"
DfcLoader.connector.MEASURES.LITRE
when "weight"
Expand Down
2 changes: 2 additions & 0 deletions engines/dfc_provider/config/routes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,6 @@
resources :affiliated_by, only: [:create, :destroy], module: 'enterprise_groups'
end
resources :persons, only: [:show]

resource :affiliate_sales_data, only: [:show]
end
1 change: 1 addition & 0 deletions engines/dfc_provider/lib/dfc_provider.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
# Custom data types
require "dfc_provider/supplied_product"
require "dfc_provider/address"
require "dfc_provider/coordination"

module DfcProvider
DataFoodConsortium::Connector::Importer.register_type(SuppliedProduct)
Expand Down
28 changes: 28 additions & 0 deletions engines/dfc_provider/lib/dfc_provider/coordination.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# frozen_string_literal: true

if defined? DataFoodConsortium::Connector::Coordination
ActiveSupport::Deprecation.warn <<~TEXT
DataFoodConsortium::Connector::Coordination is now available.
Please replace your own implementation with the official class.
TEXT
end

module DfcProvider
class Coordination
include VirtualAssembly::Semantizer::SemanticObject

SEMANTIC_TYPE = "dfc-b:Coordination"

attr_accessor :coordinator

def initialize(semantic_id, coordinator: nil)
super(semantic_id)

self.semanticType = SEMANTIC_TYPE

@coordinator = coordinator
registerSemanticProperty("dfc-b:coordinatedBy", &method("coordinator"))
.valueSetter = method("coordinator=")
end
end
end
61 changes: 61 additions & 0 deletions engines/dfc_provider/spec/requests/affiliate_sales_data_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# frozen_string_literal: true

require_relative "../swagger_helper"

RSpec.describe "AffiliateSalesData", swagger_doc: "dfc.yaml", rswag_autodoc: true do
let(:user) { create(:oidc_user) }

before { login_as user }

path "/api/dfc/affiliate_sales_data" do
parameter name: :startDate, in: :query, type: :string
parameter name: :endDate, in: :query, type: :string

get "Show sales data of person's affiliate enterprises" do
produces "application/json"

response "200", "successful", feature: :affiliate_sales_data do
let(:startDate) { Date.yesterday }
let(:endDate) { Time.zone.today }

before do
order = create(:order_with_totals_and_distribution, :completed)
ConnectedApps::AffiliateSalesData.new(
enterprise: order.distributor
).connect({})
end

context "with date filters" do
let(:startDate) { Date.tomorrow }
let(:endDate) { Date.tomorrow }

run_test! do
expect(json_response).to include(
"@id" => "http://test.host/api/dfc/affiliate_sales_data",
"@type" => "dfc-b:Person",
)

expect(json_response["dfc-b:affiliates"]).to eq nil
end
end

context "not filtered" do
run_test! do
expect(json_response).to include(
"@id" => "http://test.host/api/dfc/affiliate_sales_data",
"@type" => "dfc-b:Person",
)
expect(json_response["dfc-b:affiliates"]).to be_present
end
end
end

response "400", "bad request" do
let(:startDate) { "yesterday" }
let(:endDate) { "tomorrow" }

run_test!
end
end
end
end
Loading
Loading