Exploring the shared file system for a solution


xpresso.ai provides shared disk space for all solutions. This can be used by solution components to share data.

Organization of the Shared Disk Space

Each solution gets its own folder in the shared disk space. This folder contains sub-folders called “components” and “pipelines”, which, in turn, contain folders for each component or pipeline in the solution.

  • When a solution is created, the root folder for the solution is mounted at “/project”. Thus, any files saved by a component or pipeline in the “/project” folder will be stored here. For example, if any component or pipeline saves a file called “/project/test.txt”, this will be stored as <solution_root>/test.txt.

  • During component deployment, the component root folder is mounted at “/component”. Thus, any files saved by the component in the “/component” folder will be stored here. For example, if a component named “comp1” saves a file called “/component/test.txt”, this will be stored in <solution_root>/components/comp1/test.txt.

  • During pipeline deployment, the pipeline root folder is mounted at “/pipeline”. Thus, any files saved by any component of the pipeline in the “/pipeline” folder will be stored here. For example, if a component of a pipeline named “pipeline1” saves a file called “/pipeline/test.txt”, this will be stored in <solution_root>/pipelines/pipeline1/test.txt.

  • When an experiment is rum, the run root folder is mounted at “/run”. Thus, any files saved by any component of the pipeline in the “/run” folder will be stored here. For example, if a component of a pipeline named “pipeline1” saves a file called “/run/test.txt” during the run of an experiment named, “run1”, this will be stored in <solution_root>/pipelines/pipeline1/run1/test.txt.

Tip

All underscores (“_”) in component or pipeline names are replaced by dashes (“-”) on the shared drive

Tip

Any files saved by a component or pipeline without referencing a parent folder will be stored in the Docker container and will not normally be available after the execution is complete. Hence, it is always advisable to use one of the mounted folders to save your files

The xpresso.ai Control Center enables developers to explore the shared disk space for solutions, and to upload and download files.

To explore the files of a solution, expand the “Data Ops” menu on the left-hand panel of the solution Overview Page by clicking on it, and then click “NFS Explorer” for exploring the NFS Drive, or “HDFS Explorer” for exploring the HDFS drive (see below)

image1

Note: The NFS File Explorer is described in detail below, but the UI for the HDFS Explorer is similar.

The NFS / HDFS Explorer screen is displayed (below):

image2

The dashboard on top describes some Key Performance Indicators (KPIs) of the file system.

The file system consists of folders and files. A folder can contain other folders or files.

The left-hand panel in the page initially displays the solution at the root, with different folders under it. Note the “components” and “pipelines” folders described earlier

The right-hand pane shows details of all folders in the solution.

  • Click the folder expander (“>”) next to a branch to see the sub-folders under the folder

  • Click a folder name on either pane to see folder details (see below)

image3

  • You can use the Download icon on the Action menu to download a file or folder.

  • You can use the Delete icon on the Action menu to delete a file or folder

The toolbar on the top (see below) enables some additional actions on the file explorer

image4

  • You can upload one or more files into the currently selected folder by using the “Upload Files” button on the top left

  • You can create a new sub-folder in the current folder by clicking the “+” button

  • You can use the “Left Arrow” link (placed to the left of the folder name on top of the screen) to navigate back to the parent folder

  • You can filter the list of files or folders displayed on the right-hand pane by using the “Search” box