Memory usage

The FairCom Server startup memory requirements are calculated using the following equation:

Baseline Memory = 
  Server binary size + 
  Data cache size + 
  Index cache size + 
  System file control blocks + 
  Lock wait-for graph + 
  Global array sizes

Server binary size:

Calculate the server binary size by adding the size of the executable on disk and the sizes of all the shared libraries that the server process loads.

Data cache size:

The data cache size is set using the DAT_MEMORY configuration option. It defaults to 100 MB.

Index cache size:

The data cache size is set using the IDX_MEMORY configuration option. It defaults to 100 MB.

System file control blocks:

At startup, the server allocates an array of pointers to hold the system file control blocks. The number of entries in the array is set to the FILES configuration option. Each time a file uses a previously unused system file number, the server allocates a system file control block for that system file number and memory use increases by the size of the system file control block structure (CTFILE).

Initial allocation size:

FILES * sizeof(CTFILE *)

Size when fully used:

FILES * sizeof(CTFILE *) + FILES * (sizeof(CTFILE) + alignment)

Example structure size for 64-bit Windows server:

sizeof(CTFILE) + alignment = 4720

Lock wait-for graph:

The server allocates a two-dimensional array that it uses to detect that a blocking lock request would cause a deadlock. The array uses two bits per entry and uses the maximum number of connections set by the CONNECTIONS configuration option for both dimensions of the array.

Allocation size in bytes:

CONNECTIONS * CONNECTIONS / 4

Global array sizes:

The server allocates a number of global arrays. Many of these arrays are sized based on the maximum number of supported connections that is set in the server binary at compile time. This value, MXU2, is currently set to 10287 for a 64-bit server and to 4143 for a 32-bit server.

Some of the sizes of the global arrays depend on the CACHE_LINE configuration option, since this option causes those array elements to be aligned on a multiple of the specified cache line size. Supported CACHE_LINE values are: 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, and 1024. If an invalid CACHE_LINE option is specified in the configuration file, the server logs a warning message to CTSTATUS.FCS and uses the default value.

The total global array size for a 64-bit server is approximately 74 MB when using the default CACHE_LINE size of 128.

Base memory use for each ISAM connection is approximated as follows. <fils> is the fils parameter value specified by the client when connecting to the server.

Thread stack size + 
Communication buffer size + 
Record compression buffer size + 
Schema conversion buffer size + 
Connection level state information + 
ISAM level state information + 
Block encryption buffer + 
User file control blocks + 
Transaction mark list buffer + 
Preimage space hash bins + 
ISAM context hash bins + 
ISAM context state + 
User level lock hash bins+ 
Preimage space entries + 
Lock table entries

Thread stack size:

Each thread allocates a stack. The thread stack size is set as follows:

On Windows systems:

On Unix systems:

Communication buffer size:

The communication buffer is initially 4 KB. When the client reads or writes a record that exceeds the current communication buffer size, the server allocates a sufficiently-sized communication buffer to hold the record. The communication buffer remains allocated at that larger size until the client disconnects.

Record compression buffer size:

A connection that reads or writes records that use data record compression allocates a buffer to hold the uncompressed record image. This buffer remains allocated until the client disconnects.

Schema conversion buffer size:

A connection that reads records from a table that has had a hot alter operation performed on it might need to convert records from an older version of the record schema to the current schema version. As a performance enhancement, the server allocates a schema conversion buffer to hold the converted record image. This buffer remains allocated until the client disconnects.

Connection level state information:

The server allocates memory to hold connection level state information in the format of a CTGV structure.

Example CTGV structure size for 64-bit Windows: 14736 bytes

ISAM level state information:

The server allocates memory to hold ISAM state information. The amount of memory depends on the sizes of the CTIS, CTIS1, and CTIS2 structures, the size of the FILNO data type, and the maximum number of files requested by the client when it connects to the server. The server automatically increases the file number limit as needed when the client requests using a file number that exceeds the current file number limit, up to the MAX_FILES_PER_USER server configuration option.

The formula for calculating ISAM memory use is as follows, where <fils> is the maximum number of files requested by the client:

sizeof(CTIS) +
<fils> * sizeof(CTIS1) +
<fils> * sizeof(CTIS2 *) +
2 * <fils> * sizeof(FILNO *) +
<fils> * MAX_KEY_SEG * sizeof(CTIS2) +
2 * <fils> * MAX_DAT_KEY * sizeof(FILNO)

Typical values for 64-bit Windows are:

sizeof(CTIS) = 3160
sizeof(CTIS1) = 80
sizeof(CTIS2 *) = 8
sizeof(FILNO) = 4
sizeof(CTIS2) = 12
MAX_KEY_SEG = 16
MAX_DAT_KEY = 64

Block encryption buffer:

For each connection the server allocates a buffer for encrypting data in blocks of size 32768 rounded up to a multiple of the LOG_PAGE_SIZE server configuration option (which defaults to 8192).

User file control blocks:

The server allocates memory to hold connection level state information about each file that a connection has opened. The amount of memory depends on the size of the FUSR structure and the maximum number of files requested by the client when it connects to the server. The server automatically increases the file number limit as needed when the client requests using a file number that exceeds the current file number limit, up to the MAX_FILES_PER_USER server configuration option, which defaults to 32767.

The formula for calculating user file control block memory use is as follows, where <fils> is the maximum number of files requested by the client:

<fils> * sizeof(FUSR)

A typical value for 64-bit Windows is:

sizeof(FUSR) = 504

Transaction mark list buffer:

The server allocates a buffer to manage transaction marks in index nodes. The buffer size is given by the following formula:

PAGE_SIZE / 128 * 16 * 6

PAGE_SIZE defaults to 32768.

Preimage space hash bins:

The server allocates hash bins for efficiently searching preimage space. The memory use is given by the following formula:

PREIMAGE_HASH * sizeof(SHADLST *)

PREIMAGE_HASH defaults to 128.

sizeof(SHALST *) is 8 for 64-bit systems and 4 for 32-bit systems.

ISAM context hash bins:

The server allocates hash bins for efficiently searching ISAM contexts. The memory use is given by the following formula:

CONTEXT_HASH * sizeof(ctICON *)

CONTEXT_HASH defaults to 6.

sizeof(ctICON *) is 8 for 64-bit systems and 4 for 32-bit systems.

ISAM context state:

Each ISAM context allocates memory for each index whose state is stored in the ISAM context. The formula to calculate the memory use is as follows:

(sizeof(ctCONBUF) + key length) rounded up to multiple of sizeof(ctCONBUF)

sizeof(ctCONBUF) is 24 for 64-bit systems and 16 for 32-bit systems.

User level lock hash bins:

The server allocates hash bins for efficiently searching the user lock table. The memory use is given by the following formula:

LOCK_HASH * sizeof(ULOKHSH)

LOCK_HASH defaults to 16.

sizeof(ULOKHSH) is 16 for 64-bit systems and 8 for 32-bit systems.

Preimage space entries:

Each operation on a transaction controlled file creates a preimage space entry, which contains information about the operation. This information can include a data record image or a key value.

Memory used for each preimage space entry:

sizeof(SHADLST) + data length

data length is the size of the record image or a key value

Example structure size for 64-bit Windows server:

sizeof(SHADLST) = 128

Lock table entries:

Each record that is locked has a system lock table entry (RECLOK). Each record lock request creates a user lock table entry (LOKS). Because more than one read lock can be held on a particular record, when using read locks more than one user level lock entry can exist for a system lock table entry.

Table read locks and table write locks can be used to reduce memory use by locking the entire table rather than locking a large number of individual records.

Memory used for each record that is locked:

sizeof(RECLOK) + alignment

Memory used for each record lock that a connection has acquired:

sizeof(LOKS) + alignment

Example sizes for 64-bit Windows server:

sizeof(RECLOK) + alignment = 88

sizeof(LOKS) + alignment = 40