Rate Limiting

Most if not all tools we want to connect with restrict the number of calls their API serves using some form of rate limiting.

RateLimiter interface

No two providers implement rate limits the same way. Some use a fixed amount of requests per second, others deduct the weight of your query from a budget, etc. To accommodate for all of these use cases, our Provider class accepts a flexible RateLimiter interface that should provide you with everything you need to achieve your goals.

export type RateLimiter = <T>(
  options: { credentials: Credentials; logger: Logger },
  targetFunction: () => Promise<Response<T>>,
) => Promise<Response<T>>;

When an instance of Provider class is given a RateLimiter, it will proxy all requests in the form of a targetFunction. This gives the RateLimiter access to the credentials and logger from the RequestOptions, as well as an opportunity to capture the ProviderResponse, which will contain the response's status code, headers and body.

Example

Here's an example of a rate limiting implementation in the case of a provider giving a variable weight to requests depending on their size, and returning a limit-left field in its response body to indicate how many request points are remaining for the credentials used for a given time window.

import {
  Cache,
  Credentials,
  RateLimiter,
  ProviderResponse,
  HttpErrors
} from '@unito/integration-sdk';
import { createHash } from 'node:crypto';

// Cache shown here for simplicity. Centralize to reuse elsewhere in your integration as needed.
const cache = Cache.create();

export const rateLimiter: RateLimiter = async <T>(
  options: { credentials: Credentials },
  targetFunction: () => Promise<ProviderResponse<T>>,
): Promise<ProviderResponse<T>> => {
  const { apiKey } = getCredentials(options);

  // Take care of hashing sensible information before using it as a cache key or value, we don't want to risk a leak!
  const remainingLimitCacheKey = `rateLimiter:remainingLimit:${createHash('shake256').update(apiKey).digest('hex')}`;

  const remainingLimit: number | undefined = await cache.getValue(remainingLimitCacheKey);

  if (typeof remainingLimit === 'number' && remainingLimit <= 0) {
    const remainingLimitTime = await cache.getTtl(remainingLimitCacheKey);
    throw new HttpErrors.RateLimitExceededError(`Rate limit exceeded. Retry in ${remainingLimitTime} seconds.`);
  }

  // This provider response is expected to have a 'limit-left' and time-until-reset property to update the remaining
  // rate limit, in addition to any other payload you might have been fetching.
  const response =
    await (<Promise<ProviderResponse<{ 'limit-left'?: number, 'time-until-reset?': number }>>>targetFunction());
  const { 'limit-left': newRemainingLimit, 'time-until-reset?': ttl } = response.body;

  if (typeof newRemainingLimit === 'number') {
    // We could save the new value with ttl equal to time-until-reset (seconds) so we call the provider again
    // without worry once the rate limit is reset.
    await cache.setValue(remainingLimitCacheKey, newRemainingLimit, ttl);
  }

  return response;
};

Then, using it in a Provider instance is as easy as passing it in the options upon creation.

import { Provider } from '@unito/integration-sdk';
import { rateLimiter } from './rateLimiter.js;

export const provider = new Provider({
  prepareRequest: () => {
    return { url: 'https://myProviderUrl.com', headers: {} };
  },
  rateLimiter,
});

And now every call you make using this Provider instance will use this rateLimiter to make calls to the underlying provider's API!