How can explanation data be inverse normalized?

[StijnBerendse]

Hi,
I am using LIME to explain an RNN based forecasting model, so I am using the RecurrentTabularExplainer. My data is continuous, and I am using regression mode as that matches my prediction type. Because my data has very different scales per feature, I must normalize it before training. When using this to generate descriptions, I also get normalized values in my explanation, which is incredibly unclear.

How can I invert this normalization? I cannot find anything about this, except for many other people asking this question on various forums. This seems like a strong requirement for an explanation technique. Another issue is that the min and max of the bar on the left seems to be completely random. Sometimes the range is less than 1 between min and max, and I cannot find what this bar truly is meant to be.

Kind regards,
Stijn

[ShahrinNakkhatra-optimizely]

@StijnBerendse Not sure if this is the accurate way as I also searched for it a lot and didn't find any answer. But I'm doing it this way:

• Save the state of your dataset before using scaling/ normalization
• Then write a function to be used as predict_fn which not just provides the prediction, but also does the necessary preprocessing steps including the scaling and the later steps.
• Pass that function as your predict_fn in LimeTabularExplainer instance
  This will provide you the explanations according to your unscaled values. I have removed some lines from my code for privacy, but you get the idea.

    def explain_pipe(self, selected_df=None, cols=None):
        if selected_df is None:
            selected_df = self.selected_df

        if cols is None:
            cols = self.cols

        temp_df = pd.DataFrame(selected_df, columns=cols)
        selected_df_ = temp_df.copy()
        dp = DataProcessingPrediction(selected_df_, self.local_directory, self.product)

        selected_df_ = dp.scale_df(
            scaler_path=os.path.join(self.local_directory, "scaler_objects.pkl"),
            col_names_path=os.path.join(self.local_directory, "scaled_col_names.pkl"),
        )

        selected_df_ = dp.clean_column_names()

        selected_df_ = dp.load_and_reorder(os.path.join(self.local_directory, "column_order.pkl"))

        output = self.model.predict_proba(selected_df_)  # [ :,1]
        return output

    def explain_row(self, X_train, X_pred, row_number: int):
        lime_explainer = lime_tabular.LimeTabularExplainer(
            training_data=np.array(X_train),
            training_labels=self.training_labels,
            feature_names=X_train.columns,
            class_names=[.....],
            mode="classification",
        )

        instance = X_pred.iloc[row_number]
        lime_exp = lime_explainer.explain_instance(data_row=instance, predict_fn=self.explain_pipe)
        logging.info(lime_exp.as_list())
        return lime_exp