Skip to content

CacheSynchronizer - add setting limiting the overall POST request body size #73

Description

@TonySkorik

Currently when trying to synchronize a large batch of keys between memcached cluster nodes (e.g. when storing them with MultiStoreAsync with cache sync on) we can encounter the POST body limitation on the server side. This limitation is imposed externally and memcached client has no way of knowing about it.

Let's add a setting on MemcachedConfiguration.SyncSettings as follows

/// <summary>
/// Maximal size of the sync request body. If the actual request we are going to send is larger, we chunk the keys to sync and expiration map into parts so every part resulting request size is smaller than this value. We than process these partial requests in parallel.
/// If any of the partial cache sync requests fails we return `SyncSuccess == false`.
/// </summary>
public long MaxPostBodySizeBytes { get; set; }

And then, when planning the sync request in CacheSynchronizer.SyncData or CacheSyncClient.SyncAsync, we should ensure that the actual request size that will be issued to the sync cache endpoint is not larger than MaxPostBodySizeBytes. We can do so by batching parts of the CacheSyncModel.KeyValues dictionary in parallel.

Since we are supporting CacheSyncModel.ExpirationMap we should send only the keys that correspond to the selected CacheSyncModel.KeyValues keys in each batch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions