Architecture

The User-friendly Deep Learning architecture is comprised of three components:

  • backend

  • worker node(s)

  • client(s)

../images/architecture.png

Backend

The backend provides the REST API for managing datasets, job templates, jobs (and there output) and in general the configuration of the system.

By default, the backend stores data files on disk. But thanks to abstraction, the storage system can be easily swapped out.

All other system related information (job templates, etc) is stored in a database. Currently, sqlite and MySQL are supported.

Worker Node(s)

Nodes can be dynamically added to the pool of available workers, by simply starting up the job-launcher. A worker node polls for available jobs using its hardware environment (GPU? Hardware generation? etc) to only attempt to execute jobs that will succeed.

Each job execution will result in a log file being stored back in the backend. This log can be used to determine potential error causes in case a job terminates unexpectedly.

Client(s)

At the time of writing, two low-level client libraries are available that fully encapsulate the REST API:

These client libraries can be used to build user-facing applications: