Objective

A manufacturing company is analyzing its production process and organizing its inventory. You are provided with two DataFrames: df1 contains the manufacturing logs and locations, while df2 contains the product catalog details.

Task

Write a PySpark function that joins both DataFrames on the product_id. After joining, create a new column named row_number which assigns a unique serial number to each row in ascending order of the manufacturing_date. If two dates are identical, use product_id (ascending) as a tie-breaker to ensure row numbers are assigned consistently.

The row_number should start from 1 and increment by 1 for every subsequent row. Save your resulting DataFrame as result_df. Ensure the output matches the exact schema order requested, and order the final output by the newly created row_number column.

File Path

• Manufacturing Logs: /home/interview/df1.csv
• Product Catalog: /home/interview/df2.csv
• Starter script: /home/interview/organize_parts.py

Schema

df1.csv

df2.csv

Expected Output Schema

Example

Given this sample input:

df1

df2

The expected output would be: