This is the project for the course named “Application of big data” by Alexandre NOUAR & Théo DURA

The environnement properties with all the dependencies needed for this project is stored in the conda.yml file.

Big Data Application Project documentation!

Contents:

• Dataset informations
  • Files
  • Importing the files
• Project organization
• Data exploration
  • Target feature analysis
  • Missing values overview
  • Unique values overview
  • Correlations
• Data processing
  • Data cleaning
  • Feature engineering
• Models building and training
  • XGBOOST Model
  • Random Forest Classifier
  • Gradient Boosting Model
• XGBoost Model Training with MLFLOW
  • Code
    • Model arguments
    • Model metrics
    • Model storing
  • MLFlow UI
• Predictions using our models
• Model explaination using SHAP
• Commands
  • Process the data
  • Models
    • Basic model training
    • Training with MLFlow
  • MLFlow ui
  • Model predictions