Classification Models Fine-Tuning

In this example, we again use our sample PostgreSQL database.

First, we create and train the model using a subset of the customer_churn data, considering only female customers.

CREATE MODEL mindsdb.adjust_customer_churn_model
FROM example_db
    (SELECT *
    FROM demo_data.customer_churn
    WHERE gender = 'Female')
PREDICT churn;

On execution, we get:

Query successfully completed

We can check its status using this command:

SELECT *
FROM mindsdb.models
WHERE name = 'adjust_customer_churn_model';

Once the status is complete, we can query for predictions.

SELECT churn, churn_explain
FROM mindsdb.adjust_customer_churn_model
WHERE seniorcitizen = 0
AND partner = 'Yes'
AND dependents = 'No'
AND tenure = 1
AND phoneservice = 'No'
AND multiplelines = 'No phone service'
AND internetservice = 'DSL';

On execution, we get:

+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| churn  | churn_explain                                                                                                                                              |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| No     | {"predicted_value": "No", "confidence": 0.9887640449438202, "anomaly": null, "truth": null, "probability_class_No": 0.934, "probability_class_Yes": 0.066} |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+

Let’s adjust this model with more training data. Now we also consider male customers.

FINETUNE mindsdb.adjust_customer_churn_model
FROM example_db
    (SELECT *
    FROM demo_data.customer_churn
    WHERE gender = 'Male');

On execution, we get:

Query successfully completed

To check the status and versions of the model, run this command:

SELECT name, engine, project, active, version, status
FROM mindsdb.models_versions
WHERE name = 'adjust_customer_churn_model';

On execution, we get:

+-----------------------------+-----------+---------+--------+---------+----------+
| name                        | engine    | project | active | version | status   |
+-----------------------------+-----------+---------+--------+---------+----------+
| adjust_customer_churn_model | lightwood | mindsdb | false  | 1       | complete |
| adjust_customer_churn_model | lightwood | mindsdb | true   | 2       | complete |
+-----------------------------+-----------+---------+--------+---------+----------+

Let’s query for a prediction again.

SELECT churn, churn_explain
FROM mindsdb.adjust_customer_churn_model
WHERE seniorcitizen = 0
AND partner = 'Yes'
AND dependents = 'No'
AND tenure = 1
AND phoneservice = 'No'
AND multiplelines = 'No phone service'
AND internetservice = 'DSL';

On execution, we get:

+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| churn  | churn_explain                                                                                                                                                |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| No     | {"predicted_value": "No", "confidence": 0.9887640449438202, "anomaly": null, "truth": null, "probability_class_No": 0.9294, "probability_class_Yes": 0.0706} |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+

Here after adjusting the model, there are no significant changes to the predictions. However, the probability class for Yes and No values has been updated. The probability of a Yes value has increased slightly, while the probability of a No value has decreased.