Objective

You are analyzing expedition logs. You have two DataFrames: mountain_info, containing details about various peaks, and mountain_climbers, which logs individual ascents.

Task

Write a PySpark script to find the most recent climber for each mountain. Your output should only contain mountains that have been climbed by at least one person.

The final DataFrame must be saved as result_df and include the mountain's name, the last climber's name, and the date and time of that most recent climb. Rename the output columns to match the expected schema below.

File Path

• Mountain Info Dataset: /home/interview/mountain_info.csv
• Mountain Climbers Dataset: /home/interview/mountain_climbers.csv
• Starter script: /home/interview/latest_climbers.py

Schema

mountain_info.csv

mountain_climbers.csv

Expected Output Schema

Example

Given this sample input:

mountain_info

mountain_climbers

The output would be:

Notice how Mount Fuji and Mont Blanc are excluded because they do not appear in the climbers log in this specific excerpt. For Everest, Jane's 2022 climb is kept over John's 2020 climb.