Creating a 'pooled' dependency injection lifetime

This post follows on from my previous post, in which I discussed some theoretical/experimental dependency-injection lifetimes, based on the discussion in an episode of The Breakpoint Show. In the previous post I provided an overview of the built-in Dependency Injection lifetimes, and described the additional proposed lifetimes: tenant, pooled, and drifter.

The previous post provided a overview of each of these proposed lifetimes, and an implementation of the drifter (time-based) lifetime. In this post I provide an example of the “pooled” lifetime.

Dependency injection lifetimes

This post is intended as a direct follow-on from the previous post in which I provide the full context, but I’ll provide a brief outline of the premise here. When you register services in the .NET Core DI container, you choose one of three different lifetimes.

The lifetime you specify controls how and when the DI container chooses to create a new instance of a given service, and when it instead returns an already-existing instance of the service:

For a full introduction to dependency injection in .NET Core, see the Microsoft documentation, or chapters 8 and 9 of my book.

In episode 36 of The Breakpoint Show, Khalid, Maarten, and Woody discuss the three lifetimes I described above, providing some examples of when you might choose each one, problems to watch out for, and other things to consider.

Throughout the show, they also discuss the desire for three “additional” types of services, which didn’t quite fit into the standard lifetimes:

You can read one way to implement tenant-scoped services in this blog series from Michael McKenna and I showed an implementation of the time-based/drifter lifetime in my previous post. In this post I look at one possible implementation of a pooled lifetime.

Pooled lifetime requirements

Woody mentioned pooling in the podcast episode as a way to reduce allocations and thereby improve performance. The pooled lifetime (inspired by EF Core’s DbContext pooling feature) would make this capability a general DI feature, allowing pooling of “arbitrary” services.

In general, I considered the requirements for this feature to be:

I considered those to be the main requirements, but to make my proof of concept implementation a bit easier, I added a few anti-requirements:

That covers the majority of the important behaviour points. In the next section, I’ll show an example implementation that meets all of these.

Note that I chose not to use ObjectPool<T> to explore this. Not for any principled reason, I was just hoping to have a slightly different API. You can see a similar implementation that uses ObjectPool<T> in the Microsoft docs.

Implementing a pooled lifetime service

There are 5 different moving parts in the implementation:

Now let’s look at each of these in turn

IResettableService

This is the only requirement on the pooled service itself, and there’s only one method to implement, Reset():

public interface IResettableService
{
void Reset();
}

When called, Reset() should reset the service to its “original” state, so that it can be reused when it’s removed from the pool. If we compare this to DbContext (the original inspiration for the pooling idea), this is where the change tracking would be reset, for example.

IPooledService<T>

The IPooledService<T> interface is how the application primarily retrieves instances of a pooled service T. It’s the type that you inject into your service constructors, and it’s how you access instances of T, similar to how IOptions<T> works for configuration:

public interface IPooledService<out T>
where T : IResettableService
{
T Value { get; }
}

As you can see from the above, the service T must implement IResettableService and can be accessed via the Value property. For example, you might use it in a dependent service like this:

public class DependentService
{
private readonly IMyService _myService;
public DependentService(IPooledService<IMyService> pooled)
{
// Extract the value from the IPooledService<T>
_myService = pooled.Value;
}
}

This level of indirection is a bit annoying from a practical point of view, but I found it to be a necessary evil to create a generalised pooling lifetime. All the alternatives I could come up with that wouldn’t require IPooledService<T> instead required the IResettableService to do a lot more work, which I wanted to avoid for this implementation.

PooledService<T>

PooledService<T> is the internal implementation of IPooledService<T>:

internal class PooledService<T> : IPooledService<T>, IDisposable
where T : IResettableService
{
private readonly DependencyPool<T> _pool;


<span class="token keyword">public</span> <span class="token function">PooledService</span><span class="token punctuation">(</span><span class="token class-name">DependencyPool<span class="token punctuation">&lt;</span>T<span class="token punctuation">&gt;</span></span> pool<span class="token punctuation">)</span>
<span class="token punctuation">{</span>
    _pool <span class="token operator">=</span> pool<span class="token punctuation">;</span>
    <span class="token comment">// Rent a service from the pool</span>
    Value <span class="token operator">=</span> _pool<span class="token punctuation">.</span><span class="token function">Rent</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

<span class="token keyword">public</span> <span class="token return-type class-name">T</span> Value <span class="token punctuation">{</span> <span class="token keyword">get</span><span class="token punctuation">;</span> <span class="token punctuation">}</span>

<span class="token return-type class-name"><span class="token keyword">void</span></span> IDisposable<span class="token punctuation">.</span><span class="token function">Dispose</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token punctuation">{</span>
    <span class="token comment">// When the PooledService&lt;T&gt; is disposed,</span>
    <span class="token comment">// the service is returned to the pool</span>
    _pool<span class="token punctuation">.</span><span class="token function">Return</span><span class="token punctuation">(</span>Value<span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

}

As you can see from the code above, the PooledService<T> implementation depends on the DependencyPool<T>, which we’ll look into in detail shortly. The service rents an instance of T from the pool in the constructor. The DI container will then automatically dispose the PooledService<T> when the service scope ends, which returns the service T to the pool.

DependencyPool<T>

The DependencyPool<T> instance is where the bulk of the work happens for the implementation. The DependencyPool<T> is responsible for:

internal class DependencyPool<T>(IServiceProvider provider) : IDisposable
where T : IResettableService
{
private int _count = 0; // The number of instances in the pool
private int _maxPoolSize = 3; // TODO: Set via options
private readonly ConcurrentQueue<T> _pool = new();
private readonly Func<T> _factory = () => ActivatorUtilities.CreateInstance<T>(provider);


<span class="token keyword">public</span> <span class="token return-type class-name">T</span> <span class="token function">Rent</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token punctuation">{</span>
    <span class="token comment">// Try to retrieve an item from the pool</span>
    <span class="token keyword">if</span> <span class="token punctuation">(</span>_pool<span class="token punctuation">.</span><span class="token function">TryDequeue</span><span class="token punctuation">(</span><span class="token keyword">out</span> <span class="token class-name"><span class="token keyword">var</span></span> service<span class="token punctuation">)</span><span class="token punctuation">)</span>
    <span class="token punctuation">{</span>
        <span class="token comment">// the </span>
        Interlocked<span class="token punctuation">.</span><span class="token function">Decrement</span><span class="token punctuation">(</span><span class="token keyword">ref</span> _count<span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token keyword">return</span> service<span class="token punctuation">;</span>
    <span class="token punctuation">}</span>

    <span class="token comment">// No services in the pool</span>
    <span class="token keyword">return</span> <span class="token function">_factory</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

<span class="token keyword">public</span> <span class="token return-type class-name"><span class="token keyword">void</span></span> <span class="token function">Return</span><span class="token punctuation">(</span><span class="token class-name">T</span> service<span class="token punctuation">)</span>
<span class="token punctuation">{</span>
    <span class="token keyword">if</span> <span class="token punctuation">(</span>Interlocked<span class="token punctuation">.</span><span class="token function">Increment</span><span class="token punctuation">(</span><span class="token keyword">ref</span> _count<span class="token punctuation">)</span> <span class="token operator">&lt;=</span> _maxPoolSize<span class="token punctuation">)</span>
    <span class="token punctuation">{</span>
        <span class="token comment">// there was space in the pool, so reset and return the service</span>
        service<span class="token punctuation">.</span><span class="token function">Reset</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        _pool<span class="token punctuation">.</span><span class="token function">Enqueue</span><span class="token punctuation">(</span>service<span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token punctuation">}</span>
    <span class="token keyword">else</span>
    <span class="token punctuation">{</span>
        <span class="token comment">// The maximum pool size has been exceeded</span>
        <span class="token comment">// We incremented when attempting to return, so reverse that </span>
        Interlocked<span class="token punctuation">.</span><span class="token function">Decrement</span><span class="token punctuation">(</span><span class="token keyword">ref</span> _count<span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token punctuation">(</span>service <span class="token keyword">as</span> <span class="token class-name">IDisposable</span><span class="token punctuation">)</span><span class="token punctuation">?.</span><span class="token function">Dispose</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token punctuation">}</span>
<span class="token punctuation">}</span>

<span class="token keyword">public</span> <span class="token return-type class-name"><span class="token keyword">void</span></span> <span class="token function">Dispose</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
<span class="token punctuation">{</span>
    <span class="token comment">// If the pool itself is disposed, dispose all the pooled services</span>
    _maxPoolSize <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>

    <span class="token keyword">while</span> <span class="token punctuation">(</span>_pool<span class="token punctuation">.</span><span class="token function">TryDequeue</span><span class="token punctuation">(</span><span class="token keyword">out</span> <span class="token class-name"><span class="token keyword">var</span></span> service<span class="token punctuation">)</span><span class="token punctuation">)</span>
    <span class="token punctuation">{</span>
        <span class="token punctuation">(</span>service <span class="token keyword">as</span> <span class="token class-name">IDisposable</span><span class="token punctuation">)</span><span class="token punctuation">?.</span><span class="token function">Dispose</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token punctuation">}</span>
<span class="token punctuation">}</span>

}

The implementation is relatively simple, but with a couple of interesting points:

PoolingExtensions

We’ve pretty much covered all the moving parts now, the one last step is to register everything in the DI container. We only have two services we need to register here, DependencyPool<T> and IPooledService<T>

public static class PoolingExtensions
{
public static IServiceCollection AddScopedPooling<T>(this IServiceCollection services)
where T : class, IResettableService
{
services.TryAddSingleton<DependencyPool<T>>();
services.TryAddScoped<IPooledService<T>, PooledService<T>>();


    <span class="token keyword">return</span> services<span class="token punctuation">;</span>
<span class="token punctuation">}</span>

}

Note that we don’t register T itself as a service that you can directly pull from the container; you always need to retrieve an IPooledService<T> and access the T by calling Value. You might think that you could handle this automatically in the container, doing something like the following:

services.AddScoped(s => s.GetRequiredService<IPooledService<T>>().Value);

But unfortunately, that doesn’t work. By having the DI container return the T directly, the DI container will automatically dispose the T when the scope ends. That’s not what we want for pooled services—the pooled service will be handed out again, and we don’t want it to be disposed, we rather want it to be reset.

A reminder that an alternative to this implementation would be to use the ObjectPool<T> implementation, as described in the Microsoft docs.

Ok, we have a complete implementation, time to take it for a spin!

Testing the implementation

To test the service I’m using a similar test service as I used in the previous post, where each instance gets a different ID for its lifetime, so that we can easily see when new instances are created:

public class TestService : IResettableService, IDisposable
{
private static int _id = 0;


<span class="token keyword">public</span> <span class="token return-type class-name"><span class="token keyword">int</span></span> Id <span class="token punctuation">{</span> <span class="token keyword">get</span><span class="token punctuation">;</span> <span class="token punctuation">}</span> <span class="token operator">=</span> Interlocked<span class="token punctuation">.</span><span class="token function">Increment</span><span class="token punctuation">(</span><span class="token keyword">ref</span> _id<span class="token punctuation">)</span><span class="token punctuation">;</span>

<span class="token keyword">public</span> <span class="token return-type class-name"><span class="token keyword">void</span></span> <span class="token function">Dispose</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=&gt;</span> Console<span class="token punctuation">.</span><span class="token function">WriteLine</span><span class="token punctuation">(</span><span class="token interpolation-string"><span class="token string">$"Disposing service: </span><span class="token interpolation"><span class="token punctuation">{</span><span class="token expression language-csharp">Id</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span>

<span class="token keyword">public</span> <span class="token return-type class-name"><span class="token keyword">void</span></span> <span class="token function">Reset</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">=&gt;</span> Console<span class="token punctuation">.</span><span class="token function">WriteLine</span><span class="token punctuation">(</span><span class="token interpolation-string"><span class="token string">$"Resetting service: </span><span class="token interpolation"><span class="token punctuation">{</span><span class="token expression language-csharp">Id</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span>

}

In addition, I’ve added some basic logs to the Reset() and Dispose() methods so we can more easily track what’s going on.

To test it out, I created a small console app. The app creates a ServiceCollection, builds an IServiceProvider and then generates a bunch of scopes in parallel. For each scope, it retrieves a TestService instance, and print its Id. It then disposes all the scopes. We then run the same sequence again:

var collection = new ServiceCollection();
collection.AddScopedPooling<TestService>();
var services = collection.BuildServiceProvider();


Console.WriteLine(“Generating scopes A”)
GenerateScopes(services);
Console.WriteLine()
Console.WriteLine(“Generating scopes B”)
GenerateScopes(services);


static void GenerateScopes(IServiceProvider services)
{
var count = 5;
List<IServiceScope> scopes = new(count);


<span class="token comment">// Simulate 5 parallel requests</span>
<span class="token keyword">for</span> <span class="token punctuation">(</span><span class="token class-name"><span class="token keyword">int</span></span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span> i <span class="token operator">&lt;</span> count<span class="token punctuation">;</span> i<span class="token operator">++</span><span class="token punctuation">)</span>
<span class="token punctuation">{</span>
    <span class="token comment">// Create a scope, but don't dispose it yet</span>
    <span class="token class-name"><span class="token keyword">var</span></span> scope <span class="token operator">=</span> services<span class="token punctuation">.</span><span class="token function">CreateScope</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    scopes<span class="token punctuation">.</span><span class="token function">Add</span><span class="token punctuation">(</span>scope<span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token comment">// Retrieve an instance of the pooled service</span>
    <span class="token class-name"><span class="token keyword">var</span></span> service <span class="token operator">=</span> scope<span class="token punctuation">.</span>ServiceProvider<span class="token punctuation">.</span><span class="token generic-method"><span class="token function">GetRequiredService</span><span class="token generic class-name"><span class="token punctuation">&lt;</span>IPooledService<span class="token punctuation">&lt;</span>TestService<span class="token punctuation">&gt;</span><span class="token punctuation">&gt;</span></span></span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span>Value<span class="token punctuation">;</span>
    Console<span class="token punctuation">.</span><span class="token function">WriteLine</span><span class="token punctuation">(</span><span class="token interpolation-string"><span class="token string">$"Received service: </span><span class="token interpolation"><span class="token punctuation">{</span><span class="token expression language-csharp">service<span class="token punctuation">.</span>Id</span><span class="token punctuation">}</span></span><span class="token string">"</span></span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

<span class="token keyword">foreach</span> <span class="token punctuation">(</span><span class="token class-name"><span class="token keyword">var</span></span> scope <span class="token keyword">in</span> scopes<span class="token punctuation">)</span>
<span class="token punctuation">{</span>
    scope<span class="token punctuation">.</span><span class="token function">Dispose</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
<span class="token punctuation">}</span>

}

When we run this code, we get the following sequence:

Generating scopes A
Received value: 1
Received value: 2
Received value: 3
Received value: 4
Received value: 5
Resetting service: 1
Resetting service: 2
Resetting service: 3
Disposing service: 4
Disposing service: 5


Generating scopes B
Received value: 1
Received value: 2
Received value: 3
Received value: 6
Received value: 7
Resetting service: 1
Resetting service: 2
Resetting service: 3
Disposing service: 6
Disposing service: 7

By looking at the Ids printed, we can see this works as expected:

So it seems the implementation is working as expected, but I think it’s worth thinking about the limitations and considering whether you should use something like this.

Limitations in the pooled lifetime implementation

The inspiration for the pooling lifetime was EF Core’s pooling of DbContext, but the above implementation is fundamentally a bit different. The DbContext was designed to “know” about pooling, and its internal implementation has knowledge of DbContextLease which tracks the origin of DbContext instances etc. That stands in contrast to the above implementation in which the implementation class doesn’t need to know about the pooling details (other than providing a Reset() implementation).

The net result of the DbContext implementation is that the usage of DbContext is identical, whether or not you use pooling. That again contrasts with the implementation in this post, in which you must use the IPooledService<T> intermediate abstraction.

Another thing to consider in the above implementation is that your pooled services can’t depend on scoped services, even though the services behave somewhat like scoped services themselves. That’s because the instances “live” longer than a single service scope, so the only lifetime that really makes sense for dependencies is singleton, or possibly transient.

Yet another aspect to consider is that the pool is very simplistic. It’s first-come first-served in both renting and return. Once the items are in the pool, they’re there to stay until they rented again. You could improve all that of course, but as you make things more complicated, you risk removing the benefits that pooling could bring.

Is pooling actually useful?

The original inspiration for the pooling lifetime was EF Core’s DbContext pooling, for which a single-threaded benchmark shows that pooling can improve performance and reduce allocation:

MethodNumBlogsMeanErrorStdDevGen 0Gen 1Gen 2Allocated
WithoutContextPooling1701.6 us26.62 us78.48 us11.7188--50.38 KB
WithContextPooling1350.1 us6.80 us14.64 us0.9766--4.63 KB

However, just because it is beneficial overall for EF Core, doesn’t necessarily mean it will always be beneficial. This is explicitly called out by Microsoft in a blog post discussing pooling of ValueTask instances, back in .NET 5

In employing such a pool, the developer is betting that they can implement a custom allocator (which is really what a pool is) that’s better than the general-purpose GC allocator. Beating the GC is not trivial. But, a developer might be able to, given knowledge they have of their specific scenario.

Memory allocation in .NET is very efficient; the runtime is fast at allocating memory and is even quick at cleaning up small objects. So although pooling means the allocator generally doesn’t need to run as much, that’s not really where you’re getting performance improvements.

One possible source of performance improvements from pooling can come when the objects being allocated are large. Large objects are generally more expensive for the GC, because the more memory allocated, the more often the GC has to run, and the more work it has to do (to zero out the memory etc).

Pooling can also provide an advantage if constructing the objects is expensive. This could be because the constructor itself does a bunch of work. It could also be because the DI container doesn’t need to calculate and reconstruct the full dependency graph for the object every time it’s requested. Or it could be because you need to use a limited OS resource.

However, there are interesting problems to think about here, which could undermine any garbage collection improvements you might expect to see from pooling.

First of all, if the Reset() method has to do more work than the GC would in collecting it, then you’ve immediately lost any advantage you could expect to get from pooling. But there’s an even more subtle issue.

The .NET GC is a generational garbage collector. Newly allocated objects are placed in Gen 0, and can typically be quickly cleaned up. If a given object survives a garbage collection because it is still in use, it is promoted to Gen 1. The longest living objects are eventually promoted to Gen 2.

In general, the higher the GC generation, the more expensive it is to clean up. Ideally the GC tries to scan all the Gen 0 objects to see if they’re still alive. This works well because generally Gen 0 objects reference other Gen 0 objects. Where things get tricky is if a Gen 2 object has a reference to a Gen 0 object. Then suddenly the GC needs to check the Gen 2 objects to find out if that Gen 0 object can be collected. And that’s a lot more expensive.

And what does pooling do? It makes objects live longer, so they end up in Gen 2. If those objects hold references to short-lived objects…then suddenly you’ve made GCs much more expensive.😬

These issues (and others) are discussed by Stephen Toub with Scott Hanselman in the Deep .NET video on ArrayPool.

So in conclusion: should you use this? Probably not, but I enjoyed exploring it 😀

Summary

This post followed on from my previous post, in which I discussed some theoretical/experimental dependency-injection lifetimes, based on the discussion in an episode of The Breakpoint Show. In this post I presented a possible implementation of a “pooled” lifetime, which could be used with arbitrary services. The implementation has a bunch of limitations that I dig into, and use it as an excuse to discuss whether pooling of generic objects makes sense or not.