| CRM64Pro GDK v0.17.0 A free cross-platform game development kit built on top of SDL 3.0

Loading...
Searching...
No Matches
Archive
Core modules

## Description

Archive Manager and Archive objects for managing CDC data container files [v26.02.0].

### Overview

The Archive module provides management for CDC (CRM64Pro Data Container) files through the Archive Manager and Archive objects. CDC is an extensible and open data container format based on data blocks, designed specifically for CRM64Pro GDK as the successor to the old DPF format developed for CRM32Pro SDK.

### Key features

- Block-based storage: Flexible data organization using typed and named blocks
- Fast indexing: Variable size index for rapid search, load, save and delete operations
- Data integrity: IO corruption and file tampering detection
- Compression: Optional zlib compression for data blocks
- Encryption: AES-256 encryption for individual blocks or entire archive
- Soft deletion: Recoverable block deletion with defragmentation support

### CDC format

The CDC format defines a minimal 8-byte magic header containing the implementation version and a set of interface methods. The format is designed to be extensible, allowing different implementations.

Magic header | 8 bytes containing CDC implementation version
Block identification | Blocks identified by type (cannot start with "?SD_") and name
Maximum file size | 4GB

### CDCv1.1 implementation

Currently, the only available implementation is CDCv1.1 with the following features:

Header | 256 bytes for storing copyright, creation and modification dates, etc.
Index | Variable size index (from 128 to 4096 blocks) for fast operations
Integrity | Detects IO data corruption and file tampering
Storage modes | Raw, compressed (zlib) or encrypted (AES-256)
Archive security | AES-256 index encryption with 4-32 byte key

File operations

load() | Load CDC file with IO data corruption check
save() | Save CDC file (automatically called after successful block addition)
defrag() | Physically delete soft-deleted blocks, remove fragmentation and resize maximum blocks
getFragmentation() | Get fragmentation level caused by soft deletions and overwrites

Block operations

blockAdd() | Add new block: skips if identical exists, overwrites payload if fits, or adds new
blockAddFile() | Add file as block with type "C64_FILE" and name as [directory]+filename+[extension]
blockLoad() | Load block data (soft-deleted blocks are not loaded)
blockSearch() | Search for block (extremely fast due to fixed index)
blockAt() | Access block by index (extremely fast due to fixed index)
blockDelete() | Soft-delete block (remains physically present but not loadable)
blockUndelete() | Recover a soft-deleted block
getNumberOfBlocks() | Get total number of data blocks

Security operations

secure() | Encrypt index with AES-256 to prevent block access without password
unSecure() | Decrypt index with valid password (password is unrecoverable if lost)

### Manager type

This is a sharing manager: it shares a CDC ID among multiple owners when they request to open an already opened file based on the file path [directory]+file+[extension] as the key. The CDC is effectively closed only when the last owner closes it.

### Best practices

- Use compression (zlib) for large data blocks to reduce file size
- Use AES-256 encryption for sensitive data blocks
- Run defrag() periodically to reclaim space from soft-deleted blocks and reduce fragmentation
- Store the archive password securely, it cannot be recovered if lost
- Avoid block type names starting with "?SD_" and block names starting with "#" (reserved)
- Use blockAddFile() for adding external files with automatic type and naming
- Access the manager exclusively through Main::IArchiveMgr()

Note
  The Archive Manager is a singleton, automatically created once Main is instantiated. You can get a reference to this manager using Main::IArchiveMgr() method.
  The Archive Manager is automatically released when Main::Terminate() is called. At this time, any resource still loaded will be released, avoiding resource leaks.

## Classes
class | CRM64Pro::Archive
| Archive Object class. More...
class | CRM64Pro::ArchiveMgr
| Archive Manager class. More...

## Macros
#define | CDCv1_sPARAM1 64
#define | CDCv1_sPARAM2 16

## Enumerations
enum | CRM64Pro::eArchiveBlockStore { CRM64Pro::ABS_RAW = 1 , CRM64Pro::ABS_COMPRESSED = 2 , CRM64Pro::ABS_CRYPTED = 3 }
| Archive block storing method. More...

## Functions
bool | CRM64Pro::Archive::info (Sint32 iMode=0) override
| Request CDC object information.
const string & | CRM64Pro::Archive::getName () const override
| Get the name.
Uint32 | CRM64Pro::Archive::getID () const override
| Get the ID.
Sint32 | CRM64Pro::Archive::blockAdd (Block &myBlock)
| Add a data block.
Sint32 | CRM64Pro::Archive::blockAddFile (const string &sFile, eArchiveBlockStore eABS=ABS_COMPRESSED)
| Add a file.
Sint32 | CRM64Pro::Archive::blockDelete (Block &myBlock)
| Delete a data block.
Sint32 | CRM64Pro::Archive::blockUndelete (Uint32 iBlock, Block &myBlock)
| Recover a deleted data block.
Sint32 | CRM64Pro::Archive::blockSearch (Block &myBlock)
| Search for a given data block.
Sint32 | CRM64Pro::Archive::blockAt (Uint32 iBlock, Block &myBlock)
| Access to the given data block position.
Sint32 | CRM64Pro::Archive::blockLoad (Block &myBlock)
| Load a data block.
Sint32 | CRM64Pro::Archive::blockLoadReleaseData (Block &myBlock)
| Free a loaded data block.
Uint32 | CRM64Pro::Archive::getBlockCount () const
| Get the number of used data blocks.
Uint32 | CRM64Pro::Archive::getBlockLimit () const
| Get the number of maximum data blocks.
float | CRM64Pro::Archive::getFragmentation () const
| Get the fragmentation.
Sint32 | CRM64Pro::Archive::secure (const unsigned char *myKey, Uint32 iKeySize)
| Secure the CDC.
Sint32 | CRM64Pro::Archive::unSecure ()
| Unsecure a CDC file.
bool | CRM64Pro::ArchiveMgr::info (Sint32 iMode=0) override
| Request Archive Manager information.
Sint32 | CRM64Pro::ArchiveMgr::create (const string &sFile, Uint32 iNumBlocks=0) override
| Create a new CDC file.
bool | CRM64Pro::ArchiveMgr::close (Sint32 idCDC) override
| Close and destroy a CDC object.
Sint32 | CRM64Pro::ArchiveMgr::getCount () const override
| Get number of loaded objects.
Sint32 | CRM64Pro::ArchiveMgr::setName (Sint32, const string &) override
| This method is not implemented.
Archive * | CRM64Pro::ArchiveMgr::get (Sint32 idCDC=0)
| Get a pointer to the CDC using its handler.
Sint32 | CRM64Pro::ArchiveMgr::defrag (const string &sFile, Uint32 iNumBlocks=0)
| Defragment a CDC file.
Sint32 | CRM64Pro::ArchiveMgr::load (const string &sFile, const unsigned char *myKey=nullptr, Uint32 iKeySize=0)
| Load a CDC file.

## Macro Definition Documentation

## ◆ CDCv1_sPARAM1

#define CDCv1_sPARAM1 64

CDC v1.x: size of custom char array attribute 1

## ◆ CDCv1_sPARAM2

#define CDCv1_sPARAM2 16

CDC v1.x: size of custom char array attribute 2

## Enumeration Type Documentation

## ◆ eArchiveBlockStore

enum CRM64Pro::eArchiveBlockStore

Archive block storing method.

Enumerator
ABS_RAW | Store block in raw form.
ABS_COMPRESSED | Store block compressed.
ABS_CRYPTED | Store block encrypted.

## Function Documentation

## ◆ info() [1/2]

bool CRM64Pro::Archive::info | ( | Sint32 | iMode = 0 | ) |
| override

Request CDC object information.

Writes information to the default log.

Parameters

  iMode | unused for the time being.

Returns
  true on success or false on failure.

## ◆ getName()

const string & CRM64Pro::Archive::getName | ( |  | ) | const
| override

Get the name.

Returns
  The object name.

## ◆ getID()

Uint32 CRM64Pro::Archive::getID | ( |  | ) | const
| override

Get the ID.

Returns
  Object ID.

## ◆ blockAdd()

Sint32 CRM64Pro::Archive::blockAdd | ( | Block & | myBlock | ) |

Add a data block.

Parameters

  myBlock | filled Block struct, at least it needs pData and iSize. Custom attributes depend on CDC version.

Returns
  0 on success, or a negative error code on failure.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ blockAddFile()

Sint32 CRM64Pro::Archive::blockAddFile | ( | const string & | sFile,
|  | eArchiveBlockStore | eABS = ABS_COMPRESSED )

Add a file.

Parameters

  sFile | string containing the [directory]+filename+[extension] to be added. Directory separators '\' and '/' are supported.
  eABS | Check eArchiveBlockStore enum for details. Default is ABS_COMPRESSED.

Returns
  0 on success, or a negative error code on failure.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ blockDelete()

Sint32 CRM64Pro::Archive::blockDelete | ( | Block & | myBlock | ) |

Delete a data block.

Parameters

  myBlock | filled Block struct with custom attributes depending on CDC version.

Returns
  0 on success, or a negative error code on failure.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ blockUndelete()

Sint32 CRM64Pro::Archive::blockUndelete | ( | Uint32 | iBlock,
|  | Block & | myBlock )

Recover a deleted data block.

Parameters

  iBlock | block index position from [0 to used_blocks - 1] which contains a deleted data block. Use getBlockCount() for getting used_blocks.
  myBlock | filled Block struct with custom attributes depending on CDC version. pData must be nullptr. If recovered, myBlock will contain all attributes except pData.

Returns
  0 on success, or a negative error code on failure.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ blockSearch()

Sint32 CRM64Pro::Archive::blockSearch | ( | Block & | myBlock | ) |

Search for a given data block.

Parameters

  myBlock | filled Block struct with custom attributes depending on CDC version. If found, myBlock will contain all attributes except pData.

Returns
  [0 to used_blocks - 1] on success indicating the block index position, or a negative error code on failure. Use getBlockCount() for the used_blocks value.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ blockAt()

Sint32 CRM64Pro::Archive::blockAt | ( | Uint32 | iBlock,
|  | Block & | myBlock )

Access to the given data block position.

Parameters

  iBlock | block index position from [0 to used_blocks - 1]. Use getBlockCount() for the used_blocks value.
  myBlock | if the block index position is found, myBlock will contain all attributes except pData.

Returns
  0 on success, or a negative error code on failure.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ blockLoad()

Sint32 CRM64Pro::Archive::blockLoad | ( | Block & | myBlock | ) |

Load a data block.

Parameters

  myBlock | filled Block struct with custom attributes depending on CDC version. pData must be nullptr. If found, myBlock will contain all attributes plus the pData loaded.

Returns
  0 on success, or a negative error code on failure.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ blockLoadReleaseData()

Sint32 CRM64Pro::Archive::blockLoadReleaseData | ( | Block & | myBlock | ) |

Free a loaded data block.

Parameters

  myBlock | Block struct with pData loaded by blockLoad() method.

Returns
  Always returns 0.

## ◆ getBlockCount()

Uint32 CRM64Pro::Archive::getBlockCount | ( |  | ) | const

Get the number of used data blocks.

Returns
  [0 to block_limit] the number of used blocks (used_blocks).

## ◆ getBlockLimit()

Uint32 CRM64Pro::Archive::getBlockLimit | ( |  | ) | const

Get the number of maximum data blocks.

This limit can be increased using defrag() method.

Returns
  Positive maximum number (block_limit) of blocks.

## ◆ getFragmentation()

float CRM64Pro::Archive::getFragmentation | ( |  | ) | const

Get the fragmentation.

Returns
  Float value in the [0-100] range expressed as a percentage (lower is better, meaning less fragmentation), or a negative error code on failure.

Note
  Depending on CDC version, deleting blocks can fragment the store.
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ secure()

Sint32 CRM64Pro::Archive::secure | ( | const unsigned char * | myKey,
|  | Uint32 | iKeySize )

Secure the CDC.

Parameters

  myKey | unsigned char array of iKeySize containing the key.
  iKeySize | the key size in bytes. For CDC v1.1, the key size must be from 4 to 32 bytes.

Returns
  0 on success, or a negative error code on failure.

Warning
  Securing a CDC file and forgetting the key will make its content unreadable.

Note
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ unSecure()

Sint32 CRM64Pro::Archive::unSecure | ( |  | ) |

Unsecure a CDC file.

Returns
  0 on success, or a negative error code on failure.

Note
  The exact behaviour of this method depends on the CDC version. Please check the CDC implementation description on ArchiveMgr Detailed Description.

## ◆ info() [2/2]

bool CRM64Pro::ArchiveMgr::info | ( | Sint32 | iMode = 0 | ) |
| override

Request Archive Manager information.

Writes information to the default log.

Parameters

  iMode | -1 to display only manager information. 0 (default) to display manager and all objects. Specific object ID to display manager and only that object.

Returns
  true on success or false on failure.

## ◆ create()

Sint32 CRM64Pro::ArchiveMgr::create | ( | const string & | sFile,
|  | Uint32 | iNumBlocks = 0 )
| override

Create a new CDC file.

Parameters

  sFile | string containing [directory]+filename+[extension]. The .cdc extension is added automatically if missing. Directory separators '\' and '/' are supported.
  iNumBlocks | Maximum number of data blocks. The minimum is 64.

Returns
  greater than 0 on success (the CDC id) or a negative error code on failure.

Note
  If the file already exists, returns an error. New CDCs use the latest version available, though older versions can still be loaded.

## ◆ close()

bool CRM64Pro::ArchiveMgr::close | ( | Sint32 | idCDC | ) |
| override

Close and destroy a CDC object.

Parameters

  idCDC | 0 for closing all CDCs or the CDC id.

Returns
  true if at least one CDC was closed or a shared reference was released, or false if nothing changed.

Note
  If you forget to close a CDC, it will be automatically closed once the GDK is terminated.

## ◆ getCount()

Sint32 CRM64Pro::ArchiveMgr::getCount | ( |  | ) | const
| override

Get number of loaded objects.

Returns
  the number of CDC files.

## ◆ setName()

Sint32 CRM64Pro::ArchiveMgr::setName | ( | Sint32 | idCDC,
|  | const string & | sName )
| override

This method is not implemented.

Parameters

  idCDC | Unused
  sName | Unused

Returns
  Does not return any value.

## ◆ get()

Archive * CRM64Pro::ArchiveMgr::get | ( | Sint32 | idCDC = 0 | ) |

Get a pointer to the CDC using its handler.

By default it returns the first CDC loaded.

Parameters

  idCDC | CDC id.

Returns
  nullptr if the CDC was not found, otherwise a pointer to the CDC object.

## ◆ defrag()

Sint32 CRM64Pro::ArchiveMgr::defrag | ( | const string & | sFile,
|  | Uint32 | iNumBlocks = 0 )

Defragment a CDC file.

Parameters

  sFile | string containing [directory]+filename+[extension]. Directory separators '\' and '/' are supported.
  iNumBlocks | Maximum number of data blocks (minimum 64). Default 0 keeps current max block number. Can be used to resize the maximum number of data blocks.

Returns
  0 on success, or a negative error code on failure.

Note
  Depending on CDC version, the storing method can produce fragmentation when blocks are deleted. The CDC file must be closed and unsecured to defragment.
  Exact behavior depends on CDC version. See ArchiveMgr Detailed Description for CDC implementation details.

## ◆ load()

Sint32 CRM64Pro::ArchiveMgr::load | ( | const string & | sFile,
|  | const unsigned char * | myKey = nullptr,
|  | Uint32 | iKeySize = 0 )

Load a CDC file.

The Archive Manager shares ids based on file path as the key, so it checks if a CDC is already loaded before creating a new one.

Parameters

  sFile | string containing [directory]+filename+[extension]. Directory separators '\' and '/' are supported.
  myKey | unsigned char array containing the key. Only used for secured CDC files. Default nullptr.
  iKeySize | key size from 4 to 32 bytes. 0 for unsecured CDC (default).

Returns
  greater than 0 on success (the CDC id) or a negative error code on failure.