Originally described in #1058, we move the tracking of this feature in a separate issue as the implementation will span multiple PRs.

Storing and accessing courses and tasks are old issues in INGInious. Courses are currently simple directories containing a sub-directory for each task and configuration files.

First, almost every service of INGInious has to access such data, e.g., the frontend to write changes from the web GUI and agents to fetch the tasks files. Currently, the courses are served with a simple NFS storage.

Second, tasks are defined per course. This strongly limits tasks re-utilization, i.e., when a course administrator wants to create a new course by picking existing open-source tasks from other courses. The taskset feature has been introduced with #982 to solve that issue, but has been reverted with v0.8.9 due to operational issues.

Third, tasks and courses are not versioned. This is a limitation for the replay feature as well as for the tasks CI, #834 documents further the issue.

Fourth, authorship of tasks modification is lost when using the web GUI.

Fifth, the INGInious FS is exposed to teachers through a webdav which is not robust against concurrent modifications. In that case partial changes are effective. This common issue arise when multiple Teaching Assistants simultaneously update a same task.

An alternative solution we considered was using the MongoDB instance as a storage backend (see #883), but it didn't solve all above issues at once.

We now consider a new alternative which is storing tasks and courses as Git repositories.

1. Tasks would be stored in self-contained Git repositories. The $common directory could be replaced by a Git sub-module. This would avoid loading a full course on the agents when a single task is used. The versioning is built-in, as well as derived tasks. A task could be forked and the full history is still maintained.
2. Courses are also stored as Git repositories. They still contain the course.yaml configuration file but embed tasks as tagged sub-modules. Either the $common directory becomes a sub-module at the course level or at the tasks level. This point still should be discussed.
3. Access control to courses and tasks will be fine-grained with access tokens or SSH key. Each INGInious service will embed its own asymmetric key pair which will define specific RW permissions according to the resources that have to be reached.
4. Built-in concurrency control solving the fifth issue mentioned above.

Feature requests

Exposing versioning information to users through web GUI

See #1060.

Distributing courses and tasks through micro-services

See #1061.

Migrating LocalFSProvider based instances to GitFSProvider based ones.

See #1068.

Implementing tasksets

See #1069.

Push-based Agents

See #1070.

Leveraging versioning information in tasks replay / CI.

See #1071.