Developing python components using Pycharm¶
Follow the steps below to develop and test python components using pycharm
Your solution will have a skeleton code structure already created in the code repository. Clone your solution using “git clone”
Develop code for your solution in pycharm.
When you build the code, xpresso.ai creates a Docker image for the code. Thus, the control flow for all components is via the Dockerfile. If you examine this file (located in the xpruilddocker folder of the component), you can see that the entry point of execution in the Docker image is the run.sh script, which invokes the main python code in the app.py file
Specifically,
For jobs and services, run.sh executes the “main” function of app.py. Populate this function
You can get more information about coding pipeline jobs from the Pipeline Developer Guide
6. You can test components individually by running them within pycharm - make sure you have set up the development environment for local testing. Specifically, make sure the environment variable enable_local_execution is set to True
7. You can test pipelines as well by writing a simple script that calls each component of the pipeline in turn. The first parameter to a pipeline is always the xpresso run name. This is normally passed to the pipeline by the xpresso.ai Controller. While testing locally, you can use any string as the run name, e.g., “sample_run”
8. Tip for testing file storage locally - create a folder by the same name as your mount path on your local system. Your components will read from and write files to this folder when run locally. For example, if you expect your shared drive to be mounted to “/data”, make sure you have a folder called “/data” on your development machine (on Windows, this would be “C:/data”, or “D:/data”, depending on the drive you’re running the program from). You should then use this folder in the same way that you expect the mounted drive to be used. For example, if you expect that a pipeline will pick up a source file from this folder, you can copy the data file to the appropriate local folder, and run the pipeline to test it. Similarly, files output by the pipeline should also be present on this folder
9. When your code has been tested successfully, you can commit and push the code into the code repository using appropriate Git commands
10. If you need specific python libraries to be installed, add them to the requirements.txt file of the component (located in the requirements folder). xpresso.ai will ensure that these libraries are included in the component’s Docker image during the build process.
What do you want to do next?