Objective

In a retail store, tracking customer transactions chronologically is crucial for maintaining inventory, generating sales insights, and building recommendation engines.

Task

Write a PySpark function that processes a transactions DataFrame to compute the following:

1. Add a new column called previous_product that contains the product_id of the previous transaction made by the same customer, in chronological order. If there is no previous transaction for a customer, this column should be null.
2. Add a new column called date_and_product that concatenates the transaction date and the previous_product, separated by a space. If the previous product is null, the string should literally say "None" (e.g., "2023-01-01 None").

Save your result as result_df. Ensure the output matches the exact schema order requested.

File Path

• Dataset: /home/interview/transactions.csv
• Starter script: /home/interview/purchase_history.py

Schema

transactions.csv

Expected Output Schema

Example

Given this sample input:

transactions

The output would be: