Exploring Stowage

I originally came across Stowage on the blog aggregator Awesome .NET. Its github repo has a description which reads "Stowage is a bloat-free .NET cloud storage kit that supports at minimum THE major cloud providers." That caught my attention because I'd been contemplating how best to modernize and standardize file storage operations in an older .NET application for some time.

Onward

I created the StowageExplorer project to take Stowage for a spin. While adding the NuGet package I discovered Stowage is .NET 6 only. That ruled it out for my original intent but then I found it had ZERO dependencies. I absolutely loved that so I continued.

Stowage provides a simple abstraction around file/directory operations and built-in implementations of that abstraction for Local Disk, In-Memory, AWS S3, Azure Blob Storage, Google Cloud Storage and Databricks storage systems. Stowage's primary abstraction is the interface IFileStorage and implementations of that interface are available for all the previously mentioned storage systems. In general an instance of an IFileStorage implementation is obtained as follows:

using (IFileStorage fileStorage = Files.Of.{storage_provider}(string arg1,...))  
{
   await fileStorage.{action_method};
}

Factory methods are available through Files.Of which return instances of each of the IFileStorage implementation previously listed. For example the Azure Blob Storage implementation might be generated as follows:

var fileStorage = Files.Of.AzureBlobStorage(string "{account}", string "{key}");

An instance of IFileStorage exposes methods for all common read/write operations against the underlying file storage type and all the methods are async. With this abstraction tasks such as copying files from one place to another are standardized and the developer has no need to concern himself with the details of the underlying implementations. Following are some example usages:

// Get a list of all files in a store recursively
var files = await fileStorage.Ls(path, true, cancellationToken);

// Write a text file
await fileStorage.WriteText("ThisIsMyFileName.txt", "This is the body of the file.");

// Copy a file from one file store to another
await using var sourceStream = await fileStorage1.OpenRead(sourcePath, cancellationToken)  
await using var targetStream = await fileStorage2.OpenWrite(targetPath, writeMode, cancellationToken)  
{
    await sourceStream.CopyToAsync(targetStream, cancellationToken);
}

// Delete a file
await fileStorage.Rm("ThisIsMyFileName.txt");

The IFileStorage interface is a straightforward and elegant abstraction for working with files but ideally the instantiation of IFileStorage objects would be decoupled from their specific implementations in code. For example, let us say a developer (Bilbo) wishes to get a list of files in the root folder of an application's primary cloud storage (Cloud1). To do so he'd need to know the fact that the storage provider for Cloud1 is Azure Blob Storage so that he could call the method Files.Of.AzureBlobStorage. In this scenario Bilbo should not need to know which provider hosts Cloud1. He just needs a list of files and ideally he should only need to know the name (or key) of the provider he wishes to access. Specific details regarding the type of storage host and its required options should be resolved by configuration and dependency injection. As an example of how this might be accomplished with Stowage I created the class library SE.Domain.

The use case for SE.Domain is to supply a fictitious application with the three different IFileStorage providers it needs to function correctly: Cloud1, LocalStorage and TempStorage. Additionally, these providers must target two different underlying storage hosts, i.e. Azure Blog Storage and local disk.

We can satisfy this use case with reasonable simplicity by defining the interface.

public interface IStorageOptions  
{
    string Name { get; }
}

We then create implementations of IStorageOptions for each of our two required storage hosts.

public class AzureStorageOptions : IStorageOptions  
{
    public string AccountName { get; set; } = string.Empty;
    public string ContainerName { get; set; } = string.Empty;
    public string Key { get; set; } = string.Empty;
    public string Name { get; set; } = string.Empty;
}

public class LocalStorageOptions : IStorageOptions  
{
    public string Root { get; set; } = string.Empty;
    public string Name { get; set; } = string.Empty;
}

Each of these options classes can then be easily loaded by configuration during application start-up. An example of this can be seen in the SE.ConsoleApp application. The appsettings.json file in that project defines the required options as one AzureStorageOptions and two LocalStorageOptions:

"AzureStorageOptions": [
  {
    "Name": "Cloud1",
    "AccountName": "account1",
    "ContainerName": "container1",
    "Key": "key1"
  }
],
"LocalStorageOptions": [
  {
    "Name": "LocalStorage",
    "Root": "\\fileserver1\\files"
  },
  {
    "Name": "TempStorage",
    "Root": "c:\\temp"
  }
]

The last piece we need is something to instantiate the specific IFileStorage implementations based on the configuration options, i.e. call the correct Files.Of factory methods. This is the function of the SE.Domain.StorageManger class. StorageManager instantiates IFileStorage objects as defined by the configuration options and manages their lifetimes. It implements IDisposable and is intended to be scoped as singleton. Finally, it allows for access to those instances by name alone, e.g. Cloud1.

All of this is brought together during application start-up when configuring services (i.e. IServiceCollection).

services  
    .AddSingleton<StorageManager>()
    .AddSingleton(_ => {
        var options = new List<IStorageOptions>();
        options.AddRange(configuration.GetSection("AzureStorageOptions").Get<AzureStorageOptions[]>());
        options.AddRange(configuration.GetSection("LocalStorageOptions").Get<LocalStorageOptions[]>());
        return options;
    });

Now Bilbo can easily get his list of Cloud1 files provided the class he is working in has taken a dependency on StorageManager.

var files = storageManager  
                .GetFileStorage("Cloud1")
                .Ls(path: null, recurse: true)


Conclusion

Stowage is a beautiful and focused library which greatly simplifies working with multiple storage providers. With just a bit of extra effort abstractions can be created to further standardize such work in any application. I love the simplicity and uniformity throughout the library and, as previously noted, having no dependencies on other packages is tremendous. Stowage has found a home in my toolbox.

References