Sample Code - Creating an Inference Service

1. Create the Inference Service class

This class should extend the xpresso.ai AbstractInferenceService class

import os
import pandas as pd
import tensorflow as tf

from xpresso.ai.core.data.inference.abstract_inference_service import \
    AbstractInferenceService


class DNNInference(AbstractInferenceService):

    def __init__(self):
        super().__init__()
        # any other initializations

2. Create an instance of this class

The main method is the entry point for your service. In this method, create an instance of the class, and call its load and run_api methods

if __name__ == "__main__":
    pred = DNNInference()
    pred.load()
    pred.run_api(port=5000)

3. Implement the get_credentials and load_model methods

These methods are called by the superclass load method. The get_credentials method should return the credentials for the model versioning system, and the load_model method should load the trained model from the path provided as a parameter.

def get_credentials(self):
    return {
        "uid": "my_user_id",
        "pwd": "my_password",
        "env": "qa"
    }


def load_model(self, model_path):
    self.model = tf.keras.models.load_model(os.path.join(model_path, "saved_model.h5"))

4. Implement the transform_input, predict and transform_output methods

These methods are called when a request is serviced. The transform_input and transform_output methods are optional, required only if the input or output need to be transformed. The predict method is mandatory, to generate a prediction from the input request.

def transform_input(self, input_request):
    data_to_list = [input_request[col] for col in self.all_cols]
    print(data_to_list)
    feature = pd.DataFrame(data=[data_to_list],
                           columns=self.all_cols)
    print([feature[col].values for col in self.all_cols])
    return feature

def transform_output(self, output_response):
    return output_response

def predict(self, input_request):
    tf_vector = [tf.convert_to_tensor(input_request[col].values) for col in
                 self.all_cols]
    value = self.model.predict(tf_vector)
    print(value)
    print(type(value[0]))
    return value[0].tolist()