Tech Tutorials Database
GeekArticles Microsoft Microsoft.NET
 

Throttling Outgoing HTTP Requests in a Distributed Environment Using RavenDB and .NET Core

 
Author: codeproject.com
Category: Microsoft.NET
Comments (0)

applications today interact with external services through APIs. As a service provider hosting an API for public use, it's typical to implement "rate limiting" so that you protect your services from being overloaded by your users. Rate limiting works by tracking how many requests are sent within a set time period from a specific a <em>client</em>, this presents a challenge when you have a high-throughput application that needs to call such an API. How do you avoid hitting the rate limit and causing exceptions in your application? Furthermore, what if the API <strong>doesn't return any headers to allow you to track the request limit dynamically?</strong> This was the case recently with a service I was calling. I needed a solution to throttle <em>outgoing are some existing solutions for throttling outgoing API requests in .NET using approaches like <a SemaphoreSlim</a> or by using a more sophisticated <a gate</a>. However, these approaches only work reliably in a single-instance environment, meaning that API requests are originating from one process who can track outgoing requests using an in-memory problem becomes more complex in a distributed environment. For example, in a message queue architecture, it's typical to have multiple consumers of a message queue. Each consumer is an isolated process that may be spread out across multiple physical or virtual servers. If each consumer processes messages independently and can potentially send outgoing API requests, how can you <strong>collectively</strong> throttle outgoing alt="Diagram of multiple processes sending requests to an external API being throttled to avoid a rate limit" src="/KB/IP/5260137/distributed-environment.png" style="height: 360px; width: 640px" class="lazyload" data-sizes="auto" data-srcset="/KB/IP/5260137/distributed-environment-r-400.png 400w, /KB/IP/5260137/distributed-environment.png may expect coordinating requests across multiple clients would involve complex bits of code to write and you'd be right! You need an <strong>orchestrator</strong> that can track requests and throttle them. This would normally involve multiple layers like a database and request are many possible solutions to this problem and in this article, I'll show you how you can use RavenDB to implement throttling in <strong>less than 40 lines of case you aren't familiar with <a it is a cross-platform, high-performance, scalable NoSQL document database. At first glance, it may seem similar to MongoDB as both of them are document databases but dig a little deeper and you'll soon find that is where the similarities will be taking advantage of some unique features of RavenDB well-suited for this problem: <a and <a are specifically designed for distributed scenarios, allowing high frequency updates that work reliably across a cluster. I discussed distributed counters briefly in my previous article, <a New in RavenDB counters will let us track outgoing requests across our client instances but we also need to track requests across a sliding time window. To accomplish that, we'll use another useful feature, <a Expiration</a>. RavenDB will track the document expiration time and will automatically remove the document once it Counters are attached to documents, pairing these two features will allow us to track requests over a specific time window. If the counter exceeds the rate limit during that window, we can wait until RavenDB removes the document once it a Sample code samples I will show are part of a .NET Core console application. The code has a mock API it calls (<code>ExternalApi</code>) and uses RavenDB to track requests for code for this article <a available on GitHub</a>. You will need an instance of RavenDB setup and if you're just starting out, I recommend creating a free instance using <a Cloud</a>. This will get you up and running within README for the sample explains how to set up the user secrets required and the steps to generate a client certificate used for a Request this demo, we will have a class that represents a client who is calling an external API. The class is set up like lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">class</span> IDocumentStore ExternalApi <span class="code-keyword">public</span> Client(IDocumentStore store, ExternalApi _store = _externalApi = <span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">await</span> Documents to Track Rate counters are stored on documents, the way we can track API requests is by creating a special document that will hold any metadata around the API requests and allow us to get and update start by declaring a class to represent our rate limiting lang="cs"><span class="code-keyword">class</span> <span class="code-keyword">public</span> <span class="code-keyword">string</span> Id { <span class="code-keyword">get</span>; <span class="code-keyword">set</span>; is all we need for now. RavenDB automatically will save the document key in the <code>Id</code> property. If we wanted to track more information about the API rate limits in this document, we could. For example, we could add properties for saving the max request limit and time window, rather than hard coding it in the lang="cs"><span class="code-keyword">class</span> <span class="code-keyword">public</span> <span class="code-keyword">string</span> Id { <span class="code-keyword">get</span>; <span class="code-keyword">set</span>; <span class="code-keyword">public</span> <span class="code-keyword">int</span> MaximumRequestLimit { <span class="code-keyword">get</span>; <span class="code-keyword">set</span>; <span class="code-keyword">public</span> <span class="code-keyword">int</span> SlidingTimeWindowInSeconds { <span class="code-keyword">get</span>; <span class="code-keyword">set</span>; will need to try and load the rate limit marker before we send our request. Within the <code>SendRequest</code> method, we'll open a RavenDB session and try to load the rate limit lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">using</span> (<span class="code-keyword">var</span> session = <span class="code-keyword">var</span> limiter = <span class="code-keyword">await</span> session.LoadAsync<RateLimit>(<span class="code-string">"</span><span <span class="code-keyword">await</span> the document ID of <code>RateLimit/ExternalApi</code>. Since we are trying to throttle outgoing requests to <code>ExternalApi</code>, I gave the document an ID that makes it easy to lookup. If we had multiple services we needed to throttle, we could store them side-by-side in other rate limit marker won't always be present. When we have a fresh database, the document will not be there and when we eventually enable document expiration, RavenDB will delete the document when it expires. If it isn't present, we need to create it and save it back to the lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">using</span> (<span class="code-keyword">var</span> session = <span class="code-keyword">var</span> limiter = <span class="code-keyword">await</span> session.LoadAsync<RateLimit>(<span class="code-string">"</span><span <span class="code-keyword">if</span> (limiter == <span limiter = <span class="code-keyword">new</span> Id = <span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">await</span> <span class="code-keyword">await</span> is nothing new yet here as far as working with RavenDB. By assigning the <code>Id</code> ourselves, that will be the document key RavenDB uses when storing the document. We store the entity to track changes and save immediately so the document is Requests Using RavenDB loaded (or created) our rate limit marker document. This is used to store counters, so we'll need to use the Counters API to retrieve any counters associated with our start by eagerly loading the <code>requests</code> lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">using</span> (<span class="code-keyword">var</span> session = <span class="code-keyword">var</span> limiter = <span class="code-keyword">await</span> session.LoadAsync<RateLimit>(<span class="code-string">"</span><span includeBuilder =<span class="code-keyword">></span> includeBuilder.IncludeCounter(<span class="code-string">"</span><span <span class="code-keyword">if</span> (limiter == <span limiter = <span class="code-keyword">new</span> Id = <span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">await</span> <span class="code-keyword">await</span> when retrieving counter values, RavenDB will send an request to the database to grab the current value of the counter. But since <code>session.LoadAsync</code> is already loading the document, it seems wasteful to fetch the counter value in a second request. To remove the extra roundtrip, we can <em>eagerly load</em> the counter value in the <code>LoadAsync</code> call using the <code>includeBuilder</code> lambda familiar?</strong> This is similar to how Entity Framework can eagerly load entities. Unlike many NoSQL solutions, RavenDB <a relationships</a> and can eagerly load related can now retrieve the counter value using the <code>CountersFor.GetAsync</code> lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">using</span> (<span class="code-keyword">var</span> session = <span class="code-keyword">var</span> limiter = <span class="code-keyword">await</span> session.LoadAsync<RateLimit>(<span class="code-string">"</span><span includeBuilder =<span class="code-keyword">></span> includeBuilder.IncludeCounter(<span class="code-string">"</span><span <span class="code-keyword">if</span> (limiter == <span limiter = <span class="code-keyword">new</span> Id = <span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">await</span> <span class="code-keyword">var</span> limitCounters = <span class="code-keyword">var</span> requests = <span class="code-keyword">await</span> limitCounters.GetAsync(<span class="code-string">"</span><span <span class="code-keyword">await</span> two lines of code, we can retrieve the request counter value. If the counter has no value yet, it will be <code>null</code>. Otherwise, it will return a <code>long</code> value. If the counter value exceeds our max limit, we can abort the lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">using</span> (<span class="code-keyword">var</span> session = <span class="code-keyword">var</span> limiter = <span class="code-keyword">await</span> session.LoadAsync<RateLimit>(<span class="code-string">"</span><span includeBuilder =<span class="code-keyword">></span> includeBuilder.IncludeCounter(<span class="code-string">"</span><span <span class="code-keyword">if</span> (limiter == <span limiter = <span class="code-keyword">new</span> Id = <span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">await</span> <span class="code-keyword">var</span> limitCounters = <span class="code-keyword">var</span> requests = <span class="code-keyword">await</span> limitCounters.GetAsync(<span class="code-string">"</span><span <span class="code-keyword">if</span> (requests != <span class="code-keyword">null</span> && requests <span class="code-keyword">></span>= <span class="code-keyword">return</span>; <span class="code-comment"> do not send <span class="code-keyword">await</span> the sample application, <code>REQUEST_LIMIT</code> is set to <code>30</code>. After 30 requests, we need to stop calling the external we haven't reached the threshold yet, we can then increment the counter. When you call <code>Increment</code> or <code>Decrement</code>, you also need to call <code>SaveChangesAsync</code> to persist the counter value. RavenDB treats this as a transaction so if the save fails, the counter will not be lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">using</span> (<span class="code-keyword">var</span> session = <span class="code-keyword">var</span> limiter = <span class="code-keyword">await</span> session.LoadAsync<RateLimit>(<span class="code-string">"</span><span includeBuilder =<span class="code-keyword">></span> includeBuilder.IncludeCounter(<span class="code-string">"</span><span <span class="code-keyword">if</span> (limiter == <span limiter = <span class="code-keyword">new</span> Id = <span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">await</span> <span class="code-keyword">var</span> limitCounters = <span class="code-keyword">var</span> requests = <span class="code-keyword">await</span> limitCounters.GetAsync(<span class="code-string">"</span><span <span class="code-keyword">if</span> (requests != <span class="code-keyword">null</span> && requests <span class="code-keyword">></span>= <span class="code-keyword">return</span>; <span class="code-comment"> do not send <span class="code-comment"> increment request limitCounters.Increment(<span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">await</span> is all the code we need to track the request count and abort the request if it exceeds our max limit. In a production application, you may choose to take some other action such as exponentially backing off to wait, defer execution until the sliding time window expires, or another of the sliding time window, this code successfully prevents us from sending requests if we exceed the request limit but once we do, it will never send any again until the counter is reset. To account for this, we'll add a time component so that requests will start back up after a specific window Time Windows with Document the sample app, you can only send 30 requests within a 30 second window. If the limit is reached within any 30 second time period, you need to wait for that window to expire before trying accomplish this, we can leverage another RavenDB feature: document expiration. You will first need to <a document expiration</a> in your database (it's disabled by default). Once enabled, it is only a matter of attaching a specific metadata key to your document with a UTC timestamp to make it lang="cs"><span class="code-keyword">public</span> <span class="code-keyword">async</span> Task <span class="code-keyword">using</span> (<span class="code-keyword">var</span> session = <span class="code-keyword">var</span> limiter = <span class="code-keyword">await</span> session.LoadAsync<RateLimit>(<span class="code-string">"</span><span includeBuilder =<span class="code-keyword">></span> includeBuilder.IncludeCounter(<span class="code-string">"</span><span <span class="code-keyword">if</span> (limiter == <span limiter = <span class="code-keyword">new</span> Id = <span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">var</span> metadata = <span class="code-keyword">await</span> <span class="code-keyword">var</span> limitCounters = <span class="code-keyword">var</span> requests = <span class="code-keyword">await</span> limitCounters.GetAsync(<span class="code-string">"</span><span <span class="code-keyword">if</span> (requests != <span class="code-keyword">null</span> && requests <span class="code-keyword">></span>= <span class="code-keyword">return</span>; <span class="code-comment"> do not send <span class="code-comment"> increment request limitCounters.Increment(<span class="code-string">"</span><span <span class="code-keyword">await</span> <span class="code-keyword">await</span> in another two lines of code, we've added the ability to expire a document after a certain time. In the sample, <code>SLIDING_TIME_WINDOW_IN_SECONDS</code> is set to 30 seconds. Once the document expires and is deleted, our code will create a new marker document with a blank set of counters, allowing it to continue making requests until the limit is reached. Since RavenDB will remove expired documents automatically in the background, we've created a sliding time It's worth calling out that by default RavenDB deletes expired documents every 60 seconds. When your document expires, it will not be <em>deleted</em> until that interval lapses. This means at most, the document will stick around <code>SLIDING_TIME_WINDOW_IN_SECONDS + 60 seconds</code> until it's actually deleted. You can <a this setting</a> to suit your Requests Across Multiple premise of this article was that in a distributed scenario, RavenDB will be the orchestrator for throttling requests. How does this solution pan you have the sample set up locally, you can spin up multiple instances of to see them work together. Here's a quick demonstration of multiple processes style="width:560px; height:315px" of this are several limitations to be aware of with this solution processes could concurrently create a new <code>RateLimit</code> document. To account for this, you could enable <a concurrency</a>.</li> <li>Multiple processes could increment if request counter is <code>N - 1</code>, which would result in extra requests possibly causing an API exception (if your rate limit was exceeded).</li> <li>When the request limit is exceeded, the program retries in a tight loop. In a production app, you would be better off deferring execution until the time window has limitations could be worked around using more error checking but in the real world, these are unlikely to cause much of an issue with appropriate retry logic and API exception handling. For example, I use <a queueing</a> and <a for distributed scenarios like this article, I showcased how you can throttle outgoing requests by using two features of RavenDB, Counters and Document Expiration. If you want an easy way to get started from scratch with RavenDB, check out my Pluralsight course, <a Started with RavenDB 4</a>. Otherwise, jump in and <a the new version now</a> or head over to the <a RavenDB February, 2020: Initial version</li></ul></body></html>

Read More...




Sponsored Links




Read Next: Perform CRUD operations on MySQL database using EF Core and ASP.NET Core



 

 

Comments



Post Your Comment:

Your Name:*
e-mail ID:(required for notification)*
Image Verification:�
 
 Subscribe