CBitBuffer Class - Bit-Level Data Serialization in MQL5
The CBitBuffer class provides a robust foundation for bit-level data serialization in MQL5, offering fine-grained control over data storage and retrieval. The class includes support for various data types, including variable-length integers (VLQ with ZigZag encoding), and serialization of strings and structures, which are excellent for optimizing space. The class employs optimizations like internal buffering and exponential array growth to enhance performance and provides a comprehensive error handling system. It's particularly useful for network communication or file storage where minimizing data size is crucial (e.g., compression of tick data).
Key Features:
- Bit-level Operations: Allows writing and reading data bit by bit, or in specified bit lengths up to 64 bits.
- Data Type Support: Includes methods for bool, char, uchar, short, ushort, int, uint, long, ulong, datetime, float, double, string, and struct.
- Variable-Length Integers (VLQ): Implements VLQ encoding for int, uint, long, and ulong values, which can significantly save space for frequently small integer values.
- Error Handling: Provides an ENUM_BIT_BUFFER_ERROR enum and GetLastError()/GetLastErrorString() methods for robust error management.
- Buffer Management: Offers functions to clear, finalize, set raw buffer content, save to file, and load from file.
- Internal Buffering: Uses internal 64-bit buffers (m_writeBufferInternal, m_readBufferInternal) to optimize bit-level operations by accumulating bits before writing to or reading from the main ulong[] array.
//+------------------------------------------------------------------+
//| CBitBuffer Class |
//| Reads/writes individual bits or bit sequences to ulong[] buffer. |
//| Supports efficient bit manipulation and mixed read/write ops. |
//+------------------------------------------------------------------+
class CBitBuffer
{
public: // Core bit operations
bool WriteBit(bool bit); // Writes a single bit
bool WriteBits(ulong value, int numberOfBits); // Writes N bits from a ulong
bool ReadBit(); // Reads a single bit
ulong ReadBits(int numberOfBits); // Reads N bits as a ulong
ulong PeekBits(int numberOfBits); // Reads N bits without advancing position
public: // Position and size management
bool SetReadPosition(long bitPosition); // Sets read position in bits
long GetReadPosition();
bool ResetReadPosition(); // Resets read position to 0
bool SkipBits(long bitsToSkip); // Skips N bits from current read position
long GetTotalWrittenBits(); // Total bits written
long GetTotalBytesWritten(); // Total bytes written
long GetTotalBytesAllocated(); // Total allocated bytes
long GetRemainingReadBits(); // Remaining bits to read
public: // Data type specific read/write operations
bool WriteBool(bool value);
bool WriteChar(char value);
bool WriteUChar(uchar value);
bool WriteShort(short value);
bool WriteUShort(ushort value);
bool WriteInt(int value);
bool WriteUInt(uint value);
bool WriteLong(long value);
bool WriteULong(ulong value);
bool WriteDatetime(datetime value);
bool WriteFloat(float value); // Writes a 32-bit float
bool WriteDouble(double value); // Writes a 64-bit double
bool WriteString(string value); // Writes string with length prefix
template<typename T>
bool WriteStruct(T &struct_object); // Writes struct with length prefix
bool ReadBool();
char ReadChar();
uchar ReadUChar();
short ReadShort();
ushort ReadUShort();
int ReadInt();
uint ReadUInt();
long ReadLong();
ulong ReadULong();
datetime ReadDatetime();
float ReadFloat(); // Reads a 32-bit float
double ReadDouble(); // Reads a 64-bit double
string ReadString(); // Reads string with length prefix
template<typename T>
T ReadStruct(); // Reads struct with length prefix
public: // Variable-length encoding for integers (VLQ)
bool WriteVarInt(int value); // Writes signed int using ZigZag + VLQ
bool WriteVarUInt(uint value); // Writes unsigned int using VLQ
bool WriteVarLong(long value); // Writes signed long using ZigZag + VLQ
bool WriteVarULong(ulong value); // Writes unsigned long using VLQ
int ReadVarInt(); // Reads signed int using ZigZag + VLQ
uint ReadVarUInt(); // Reads unsigned int using VLQ
long ReadVarLong(); // Reads signed long using ZigZag + VLQ
ulong ReadVarULong(); // Reads unsigned long using VLQ
public: // Buffer management
void Clear(); // Clears buffer content and resets state
bool GetFinalizedBuffer(ulong &destinationArray[]); // Copies buffer content to array
bool SetRawBuffer(const ulong &sourceBuffer[]); // Sets buffer content from array
bool Save(string filename); // Saves buffer to file
bool Load(string filename); // Loads buffer from file
public: // Error handling
ENUM_BIT_BUFFER_ERROR GetLastError(); // Returns last error code
string GetLastErrorString(); // Returns error description string
void ClearLastError() ; // Clears last error
void PrintHex(); // Prints main buffer in hex (debugging)
};
The complete test example "CBitBuffer_Test.mq5" provided above is the best way to detect errors and verify the class's capabilities.
The header file "TicksShort.mqh" provides an alternative implementation of 'TicksShort' tick compression library by fxsaber, utilizing VLQ encoding from 'CBitBuffer' class.
Updates:
2025.07.21 - v.1.01 :
- The CBitBuffer class actively prevents illegal attempts to write after a read operation has begun, which helps in maintaining data integrity.
- Users receive a specific error (BIT_BUFFER_MIXED_OPERATION_ERROR) when they attempt to mix operations.
2025.07.22 - v.1.02 :
- Changed design of the class to allow the mixed read/write operations, by correct handling of partial flushes/refills, which helps in maintaining data integrity when switching between read and write modes.
- Removed the variable (m_operationMode) and the error code (BIT_BUFFER_MIXED_OPERATION_ERROR).
- Cleaned the code comments for better clarity of intent.
- Updated examples in "CBitBuffer_Test.mq5" to cover more test cases.