Another Virtual File System for Go

AVFS is a virtual file system abstraction, inspired mostly by Afero and Go standard library. It provides an abstraction layer to emulate the behavior of a file system that provides several features :

• a set of constants, interfaces and types for all file systems
• a test suite for all file systems (emulated or real)
• each file system has its own package
• a very basic identity manager allows testing of user related functions (Chown, Lchown) and file system permissions

Additionally, some file systems support :

• user file creation mode mask (Umask) (MemFS, OrefaFS)
• chroot (OSFS on Linux)
• hard links (MemFS, OrefaFS)
• symbolic links (MemFS)
• multiple users concurrently (MemFS)
• Linux and Windows emulation regardless of host operating system (MemFS, OrefaFS)

This package can be installed with the go install command :

go install github.com/avfs/avfs@latest

It is only tested with Go version >= 1.18

To make an existing code work with AVFS :

• replace all references of os, path/filepath with the variable used to initialize the file system (vfs in the following examples)
• import the packages of the file systems and, if necessary, the avfs package and initialize the file system variable.
• some file systems provide specific options available at initialization. For instance MemFS needs WithSystemDirs option to create /home, /root and /tmp directories.

The example below demonstrates the creation of a file, a symbolic link to this file, for a different file systems (depending on an environment variable). Error management has been omitted for the sake of simplicity :

package main

import (
	"bytes"
	"log"
	"os"
	
	"github.com/avfs/avfs"
	"github.com/avfs/avfs/vfs/memfs"
	"github.com/avfs/avfs/vfs/osfs"
)

func main() {
	var vfs avfs.VFS

	switch os.Getenv("ENV") {
	case "PROD": // The real file system for production.
		vfs = osfs.NewWithNoIdm()
	default: // in memory for tests.
		vfs = memfs.New()
	}

	// From this point all references of 'os', 'path/filepath'
	// should be replaced by 'vfs'
	rootDir, _ := vfs.MkdirTemp("", "avfs")
	defer vfs.RemoveAll(rootDir)

	aFilePath := vfs.Join(rootDir, "aFile.txt")

	content := []byte("randomContent")
	_ = vfs.WriteFile(aFilePath, content, 0o644)

	aFilePathSl := vfs.Join(rootDir, "aFileSymlink.txt")
	_ = vfs.Symlink(aFilePath, aFilePathSl)

	gotContentSl, _ := vfs.ReadFile(aFilePathSl)
	if !bytes.Equal(content, gotContentSl) {
		log.Fatalf("Symlink %s : want content to be %v, got %v",
			aFilePathSl, content, gotContentSl)
	}

	log.Printf("content from symbolic link %s : %s", aFilePathSl, gotContentSl)
}

The example below demonstrates the concurrent creation of subdirectories under a root directory by several users in different goroutines (works only with MemFS) :

package main

import (
	"fmt"
	"log"
	"sync"

	"github.com/avfs/avfs"
	"github.com/avfs/avfs/idm/memidm"
	"github.com/avfs/avfs/vfs/memfs"
)

func main() {
	const (
		maxUsers  = 100
		groupName = "test_users"
	)

	idm := memidm.New()
	vfs := memfs.New()

	rootDir, _ := vfs.MkdirTemp("", "avfs")
	vfs.Chmod(rootDir, 0o777)

	g, _ := idm.GroupAdd(groupName)

	var wg sync.WaitGroup
	wg.Add(maxUsers)

	for i := 0; i < maxUsers; i++ {
		go func(i int) {
			defer wg.Done()

			userName := fmt.Sprintf("user_%08d", i)
			idm.UserAdd(userName, g.Name())

			vfsU, _ := vfs.Sub("/")
			vfsU.SetUser(userName)

			path := vfsU.Join(rootDir, userName)
			vfsU.Mkdir(path, avfs.DefaultDirPerm)
		}(i)
	}

	wg.Wait()

	entries, _ := vfs.ReadDir(rootDir)

	log.Println("number of dirs :", len(entries))
	for _, entry := range entries {
		info, _ := entry.Info()
		sst := vfs.ToSysStat(info)

		u, _ := idm.LookupUserId(sst.Uid())

		log.Println("dir :", info.Name(),
			", mode :", info.Mode(),
			", owner :", u.Name())
	}
}

Almost ready for Windows.

All file systems implement at least avfs.FS and avfs.File interfaces. By default, each file system supported methods are the most commonly used from packages os and path/filepath. All methods have identical names as their functions counterparts. The following file systems are currently available :

Identity managers allow users and groups management. The ones implemented in avfs are just here to allow testing of functions related to users (Chown, Lchown) and access rights, so they just allow one default group per user.

All file systems supporting identity manager implement by default the identity manager DummyIdm where all functions returns avfs.ErrPermDenied.