Using AI to Generate a File Cache in C#

Feel like sharing?

Like most, I’ve been playing around with the awesome capabilities of AI for developers. Recently we had to develop a small class library that would act as a specialized cache. Since I had a little extra bandwidth, I decided to have a little fun and see if I could get an AI engine to write the entire class library without the need for me to make any modification.

Warm Up Web Pages to a Local File Cache

The application is responsible for generating web content that can be fairly large and computationally taxing to create. Luckily, once a web page is created, it does not change frequently. In most cases, we would use a typical cache library, but there were a number of specialized abilities we needed to consider:

• Since each web page was fairly large, we did not like the performance we were seeing and error rates we were receiving from trying to store the generated pages within Redis.
  
• In-memory caching wasn’t an option since we need to cache upwards of 10,000 pages.

Our best option seemed to be a local file cache. Unfortunately, no cache libraries we found fit the basic needs we had regarding key expiration and dealing with application resets.

Most of our caching is currently done with a cache-aside pattern, but in this case since we wanted to pre-generate the web pages on each webserver, so we were going to need a background job to do the work. Some other project details driving our requirements:

• Each tenant (customer) can have potentially hundreds or even thousands of custom web pages which we need to warm-up on each web server.
  
• Each tenant account also creates a single event that invalidates all existing cached pages. When the event occurs, we want to integrate the timestamp of that event as part of our cache key. This will allow us to quickly delete batches of old entries.

Prompt Engineering

Prompt engineering is the next big thing. It is the modern-day equivalent of Name That Tune. We want to try and identity the minimum amount of text needed by our AI engines to get the exact output we want. We also want to get our results with the least number of iterations to our prompt. Learning to write these prompts is essentially natural language programming. The trick is trying to think like the AI system and pre-determining exactly what it needs to do its job. We are all going to get much better at requirement definitions because of this technology.

Amazingly, I was able to get exactly what I wanted in 2 prompts. Here’s my first attempt (formatted a little for clarity):

The AI Engine’s response:

using System;
using System.Collections.Concurrent;
using System.IO;
using System.Threading.Tasks;

public class FileCache
{
    private readonly string rootDirectory;
    private readonly ConcurrentDictionary<int, long> tenantTimestamps = new ConcurrentDictionary<int, long>();

    public FileCache(string rootDirectory)
    {
        if (!Directory.Exists(rootDirectory))
        {
            throw new ArgumentException("Root directory does not exist", nameof(rootDirectory));
        }

        this.rootDirectory = rootDirectory;
    }

    public async Task Store(int tenantId, long timestamp, string key, string value)
    {
        string filePath = GetFilePath(tenantId, timestamp, key);

        await Task.Run(() =>
        {
            lock (filePath)
            {
                Directory.CreateDirectory(Path.GetDirectoryName(filePath));
                File.WriteAllText(filePath, value);
                tenantTimestamps.AddOrUpdate(tenantId, timestamp, (_, oldValue) => Math.Max(oldValue, timestamp));
            }
        });

        await Task.Run(() => DeleteOldEntries(tenantId));
    }

    public async Task<string> Retrieve(int tenantId, long timestamp, string key)
    {
        string filePath = GetFilePath(tenantId, timestamp, key);

        try
        {
            string content = await Task.Run(() =>
            {
                lock (filePath)
                {
                    return File.Exists(filePath) ? File.ReadAllText(filePath) : null;
                }
            });

            await Task.Run(() => DeleteOldEntries(tenantId));

            return content;
        }
        catch (Exception)
        {
            return null;
        }
    }

    private string GetFilePath(int tenantId, long timestamp, string key)
    {
        string tenantDirectory = Path.Combine(rootDirectory, tenantId.ToString());
        return Path.Combine(tenantDirectory, timestamp.ToString(), key);
    }

    private void DeleteOldEntries(int tenantId)
    {
        if (!tenantTimestamps.TryGetValue(tenantId, out long currentTimestamp))
        {
            return;
        }

        string tenantDirectory = Path.Combine(rootDirectory, tenantId.ToString());
        string[] timestampDirectories = Directory.GetDirectories(tenantDirectory);

        foreach (string timestampDirectory in timestampDirectories)
        {
            long timestamp = long.Parse(Path.GetFileName(timestampDirectory));
            if (timestamp != currentTimestamp)
            {
                Directory.Delete(timestampDirectory, true);
            }
        }
    }
}

Not too shabby. In fact, it was almost perfect. The only thing I noticed was that on the creation of a new entry, our AI engine wanted to launch a new thread to both check for and delete old entries each time. This is a bit wasteful. It would make more sense to check for the change and only launch the thread to delete old files if needed.

I also wanted some better documentation.

So I changed my prompt to this:

Notice how I had to react to the AI’s desire to launch a thread. I introduced some natural language logic to prevent it from making inefficient thread creations.

AI Engine’s response:

using System;
using System.Collections.Generic;
using System.IO;
using System.Threading;

/// <summary>
/// Represents a mechanism to cache strings within a local file system.
/// </summary>
public class FileSystemCache
{
    private readonly string _rootDirectory;
    private readonly Dictionary<int, long> _lastTimestampByTenantId = new Dictionary<int, long>();
    private readonly ReaderWriterLockSlim _tenantLock = new ReaderWriterLockSlim();

    /// <summary>
    /// Initializes a new instance of the <see cref="FileSystemCache"/> class with the specified root directory.
    /// </summary>
    /// <param name="rootDirectory">The root directory for the cache.</param>
    public FileSystemCache(string rootDirectory)
    {
        _rootDirectory = rootDirectory;
        Directory.CreateDirectory(_rootDirectory);
    }

    /// <summary>
    /// Stores a string value in the cache for the specified tenant, timestamp, and key.
    /// </summary>
    /// <param name="tenantId">The identifier for the tenant associated with the cached item.</param>
    /// <param name="timestamp">The timestamp associated with the cached item.</param>
    /// <param name="key">The key for the cached item.</param>
    /// <param name="value">The value to cache.</param>
    public void Set(int tenantId, long timestamp, string key, string value)
    {
        try
        {
            var tenantDirectory = GetTenantDirectory(tenantId);
            var timestampDirectory = GetTimestampDirectory(tenantDirectory, timestamp);
            var filePath = Path.Combine(timestampDirectory, key);

            _tenantLock.EnterWriteLock();

            if (_lastTimestampByTenantId.TryGetValue(tenantId, out var lastTimestamp)
                && lastTimestamp != timestamp)
            {
                ThreadPool.QueueUserWorkItem(_ => DeleteOldTimestampDirectories(tenantDirectory, timestamp));
            }

            _lastTimestampByTenantId[tenantId] = timestamp;

            using (var fileStream = File.Open(filePath, FileMode.Create, FileAccess.Write))
            using (var writer = new StreamWriter(fileStream))
            {
                writer.Write(value);
            }
        }
        finally
        {
            _tenantLock.ExitWriteLock();
        }
    }

    /// <summary>
    /// Gets the cached string value for the specified tenant, timestamp, and key.
    /// </summary>
    /// <param name="tenantId">The identifier for the tenant associated with the cached item.</param>
    /// <param name="timestamp">The timestamp associated with the cached item.</param>
    /// <param name="key">The key for the cached item.</param>
    /// <returns>The cached value if it exists; otherwise, null.</returns>
    public string Get(int tenantId, long timestamp, string key)
    {
        try
        {
            var tenantDirectory = GetTenantDirectory(tenantId);
            var timestampDirectory = GetTimestampDirectory(tenantDirectory, timestamp);
            var filePath = Path.Combine(timestampDirectory, key);

            _tenantLock.EnterReadLock();

            if (!File.Exists(filePath))
            {
                return null;
            }

            using (var fileStream = File.Open(filePath, FileMode.Open, FileAccess.Read))
            using (var reader = new StreamReader(fileStream))
            {
                return reader.ReadToEnd();
            }
        }
        catch (Exception)
        {
            return null;
        }
        finally
        {
            _tenantLock.ExitReadLock();
        }
    }


    private string GetTenantDirectory(int tenantId)
    {
        var tenantDirectory = Path.Combine(_rootDirectory, tenantId.ToString());

        Directory.CreateDirectory(tenantDirectory);

        return tenantDirectory;
    }

    private string GetTimestampDirectory(string tenantDirectory, long timestamp)
    {
        var timestampDirectory = Path.Combine(tenantDirectory, timestamp.ToString());

        Directory.CreateDirectory(timestampDirectory);

        return timestampDirectory;
    }

    private void DeleteOldTimestampDirectories(string tenantDirectory, long currentTimestamp)
    {
        var tenantId = int.Parse(new DirectoryInfo(tenantDirectory).Name);

        _tenantLock.EnterReadLock();

        try
        {
            foreach (var directory in Directory.GetDirectories(tenantDirectory))
            {
                var timestampString = new DirectoryInfo(directory).Name;

                if (!long.TryParse(timestampString, out var timestamp) || timestamp >= currentTimestamp)
                {
                    continue;
                }

                Directory.Delete(directory, recursive: true);
            }

            _lastTimestampByTenantId[tenantId] = currentTimestamp;
        }
        finally
        {
            _tenantLock.ExitReadLock();
        }
    }
}

This class was exactly what I needed, and the icing on the cake is the little “documentation” that the AI engine provided after code generation, helping stupid humans understand what it just gave us:

The FileSystemCache class provides a thread-safe way to cache string values in a local file system. The class stores the cached files in subdirectories under the root directory, with each tenant having its own subdirectory. The class allows you to set a value for a specific tenant, timestamp, and key, and retrieve the cached value for the same set of parameters.

The class uses a Dictionary<int, long> to keep track of the last timestamp for each tenant. When a new value is set for a tenant, the class checks the timestamp for that tenant. If the timestamp has changed since the last set operation, the class launches a background thread to delete all the old directories for that tenant that have a different timestamp in the path.

The Set method takes four parameters: tenantId, timestamp, key, and value. The Get method takes three parameters: tenantId, timestamp, and key. If a cached value is found for the specified set of parameters, the method returns the cached value as a string. If no cached value is found or an exception occurs during the retrieval process, the method returns null.

To ensure thread-safety, the class uses a ReaderWriterLockSlim to allow multiple concurrent read accesses but only one write access at a time. The class creates directories as necessary when setting or getting cached values.

Conclusion

AI IS going to have an impact on developers. This quick example shows how I used it to save me HOURS of time. Any organization that can apply this framework at scale will ultimately need far less salaried developers on staff. If you don’t quickly learn to utilize AI to help yourself become a more productive developer you ultimately risk getting left behind.

PS Always say please and thank you to your future AI overlords. Like Santa, they will probably remember who was polite and who was rude to them while they were in the infancy.

Feel like sharing?