UNIXworkcode
1 # Buffer 2 3 This buffer implementation can be used to read from or write to memory like you would do with a stream. 4 5 This allows the use of `cx_stream_copy()` (see [](streams.h.md)) to copy contents from one buffer to another, 6 or from a file or network stream to the buffer and vice versa. 7 8 More features for convenient use of the buffer can be enabled, like automatic memory management, 9 automatic resizing of the buffer space, or automatic flushing of contents. 10 11 The functions `cxBufferRead()` and `cxBufferWrite()` are `cx_read_func` and `cx_write_func` compatible, 12 which in turn have a compatible signature to `fread()` and `fwrite()`. 13 However, due to the different pointer type the function pointers do not type check out of the box. 14 For convenience, the macros `cxBufferReadFunc` and `cxBufferWriteFunc` are defined, which perform the necessary cast. 15 16 ## Example 17 18 In the following example we use a `CxBuffer`, the `bprintf()` macro from the [UCX printf header](printf.h.md), 19 and a [UCX map](map.h.md) to create a simple HTTP GET request. 20 21 ```C 22 #include <cx/buffer.h> 23 #include <cx/printf.h> 24 #include <cx/map.h> 25 26 void send_get_request( 27 void *net_target, 28 cx_write_func net_write, 29 const char *host, 30 const char *path, 31 CxMap *header 32 ) { 33 // initialize buffer and use stack memory for small requests 34 char stackmem[128]; 35 CxBuffer buf; 36 cxBufferInit( 37 &buf, stackmem, sizeof(stackmem), 38 cxDefaultAllocator, 39 CX_BUFFER_COPY_ON_EXTEND // move to heap when request is larger 40 ); 41 42 // basic request data 43 cx_bprintf(&buf, "GET %s HTTP/1.1\r\n", path); 44 cx_bprintf(&buf, "Host: %s\r\n", host); 45 46 // add headers 47 CxMapIterator iter = cxMapIterator(header); 48 cx_foreach(CxMapEntry*, entry, iter) { 49 cxstring name = cx_strn(entry->key->data, entry->key->len); 50 cx_bprintf(&buf, "%" CX_PRIstr ": %s\r\n", 51 CX_SFMT(name), entry->value 52 ); 53 } 54 55 // blank line terminates 56 cxBufferWrite("\r\n", 1, 2, &buf); 57 58 // write request to the network stream and destroy the buffer 59 net_write(buf.space, 1, buf.size, net_target); 60 cxBufferDestroy(&buf); 61 } 62 ``` 63 64 ## Create 65 66 ```C 67 #include <cx/buffer.h> 68 69 int cxBufferInit(CxBuffer *buffer, void *space, size_t capacity, 70 const CxAllocator *allocator, int flags); 71 72 CxBuffer *cxBufferCreate(void *space,size_t capacity, 73 const CxAllocator *allocator, int flags); 74 75 // available flags: 76 #define CX_BUFFER_DEFAULT 77 #define CX_BUFFER_FREE_CONTENTS 78 #define CX_BUFFER_AUTO_EXTEND 79 #define CX_BUFFER_COPY_ON_WRITE 80 #define CX_BUFFER_COPY_ON_EXTEND 81 ``` 82 83 For creating a UCX buffer, you have two options: initialize a pre-allocated structure, or allocate and initialize a new structure in one call. 84 85 For the first option, you can call `cxBufferInit()` with the `buffer` argument pointing to an uninitialized `CxBuffer` structure. 86 You can pass a pre-allocated `space`, the desired `capacity`, and an `allocator`. 87 If `space` is `NULL`, the `allocator` is used to allocate enough space to match the desired `capacity`. 88 89 The `flags` argument is a bit-wise-or combination of the constants listed below. 90 91 The function `cxBufferCreate()` is equivalent, except that it uses the `allocator` to allocate a new `CxBuffer` structure, instead of initializing an existing one. 92 93 If any allocation fails, `cxBufferInit()` returns non-zero, or `cxBufferCreate()` returns `NULL`, respectively. 94 95 | Flag | Description | 96 |--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| 97 | CX_BUFFER_DEFAULT | Alias for zero, meaning no flags are set. | 98 | CX_BUFFER_FREE_CONTENTS | When the buffer structure is [destroyed](#destroy), the memory for the contents is also automatically deallocated. | 99 | CX_BUFFER_AUTO_EXTEND | When a write operation would exceed the buffers capacity, the buffer is grown automatically with a call to `cxBufferMinimumCapacity()`. | 100 | CX_BUFFER_COPY_ON_WRITE | Any write operation will result in allocating a copy of the data, first. This is particularly useful, when a buffer is initialized with read-only memory. | 101 | CX_BUFFER_COPY_ON_EXTEND | Only when a write operation would exceed the buffers capacity, the copy-on-write operation is performed. This is useful, when a buffer is initialized with read-write memory which is allocated on the stack. | 102 103 104 ## Destroy 105 106 ```C 107 #include <cx/buffer.h> 108 109 void cxBufferDestroy(CxBuffer *buffer); 110 111 void cxBufferFree(CxBuffer *buffer); 112 ``` 113 114 The above functions destroy the buffer and deallocate the buffer's memory when the `CX_BUFFER_FREE_CONTENTS` flag is set. 115 116 The function `cxBufferDestroy()` is to be used when the buffer was initialized with `cxBufferInit()`, 117 and the function `cxBufferFree()` is to be used when the buffer was created with `cxBufferCreate()`. 118 The only difference is, that `cxBufferFree()` additionally deallocates the memory of the `CxBuffer` structure. 119 120 ## Capacity Management 121 122 ```C 123 #include <cx/buffer.h> 124 125 int cxBufferReserve(CxBuffer *buffer, size_t capacity); 126 127 int cxBufferMinimumCapacity(CxBuffer *buffer, size_t capacity); 128 129 void cxBufferShrink(CxBuffer *buffer, size_t reserve); 130 ``` 131 132 The function `cxBufferMinimumCapacity()` guarantees a buffer capacity of _at least_ `capacity`. 133 It will grow the capacity in powers of two until the system's page size is reached. 134 Then, the new capacity will be a multiple of the page size. 135 The function returns non-zero if increasing the capacity was attempted unsuccessfully. 136 137 The function `cxBufferReserve()`, on the other hand, will reallocate the buffer's space to match exactly the requested `capacity` 138 and the contents are truncated if required. 139 140 You should use `cxBufferReserve()` when you know precisely the required capacity beforehand 141 and `cxBufferMinimumCapacity()` when it is likely that the buffer needs to regrow soon. 142 143 The function `cxBufferShrink()` can be used to shrink the capacity of the buffer to its current size 144 plus a number of `reserve` bytes. 145 If the current capacity is not larger than the size plus the reserve bytes, the function will do nothing. 146 147 > If the buffer is in a copy-on-write state, `cxBufferMinimumCapacity()` will perform the copy-on-write action 148 > before reallocating the space. 149 > The function `cxBufferShrink()` on the other hand does _nothing_ when the buffer is in a copy-on-write state, 150 > because it does not make much sense to copy the memory just to have it in a smaller memory region. 151 152 ## Write 153 154 ```C 155 #include <cx/buffer.h> 156 157 size_t cxBufferWrite(const void *ptr, size_t size, size_t nitems, 158 CxBuffer *buffer); 159 160 int cxBufferPut(CxBuffer *buffer, int c); 161 162 size_t cxBufferPutString(CxBuffer *buffer, const char *str); 163 164 int cxBufferTerminate(CxBuffer *buffer); 165 166 size_t cxBufferAppend(const void *ptr, size_t size, size_t nitems, 167 CxBuffer *buffer); 168 ``` 169 170 The primary function for writing to a buffer is `cxBufferWrite()` 171 which writes up to `nitems` with `size` bytes each from the memory pointed to by `ptr` to the buffer. 172 173 When the capacity of the buffer is not sufficient and the `CX_BUFFER_AUTO_EXTEND` is not set in the buffer, 174 items that do not fit into the buffer are discarded. 175 The function always returns the actual number of items successfully written. 176 This equals the number of bytes if and only if `size=1`. 177 178 The function `cxBufferPut()` is a `putc()`-like wrapper for `cxBufferWrite()` which writes the character `c`, 179 converted to an `unsigned char` to the buffer. 180 Just like `putc()` this function returns the written character on success, and `EOF` on failure, 181 but it does _not_ set `errno` on failure. 182 183 The function `cxBufferPutString()` is a convenience function, 184 that uses stdlib `strlen()` to compute the length of `str` and then invokes `cxBufferWrite()`. 185 186 All the above functions advance the buffer position by the number of bytes written 187 and cause the _size_ of the buffer to grow, if necessary, to contain all written bytes. 188 On the other hand, `cxBufferTerminate()` writes a zero-byte at the current position, 189 effectively creating a zero-terminated string whose size equals the buffer size. 190 191 The function `cxBufferAppend()` writes the data to the end of the buffer (given by its size) regardless of the current position, 192 and it also does _not_ advance the position. 193 If the write operation triggered a [flush](#flushing), however, the position will be shifted left alongside the shifted buffer contents. 194 In case the data at which the current position points gets flushed, the new position will be zero. 195 196 ## Read 197 198 ```C 199 #include <cx/buffer.h> 200 201 size_t cxBufferRead(void *ptr, size_t size, size_t nitems, 202 CxBuffer *buffer); 203 204 int cxBufferGet(CxBuffer *buffer); 205 206 bool cxBufferEof(const CxBuffer *buffer); 207 ``` 208 209 The function `cxBufferRead()` reads `nitems` number of items of `size` bytes each from the `buffer` 210 and stores them into the memory pointed to by `ptr`, which must be large enough to hold the contents. 211 The function returns the actual bytes read, which might be lower if the desired number of items is not available. 212 213 The function `cxBufferGet()` is a `fgetc()`-like function which returns the next byte in the buffer converted to an `int`. 214 Similar to `fgetc()` it returns `EOF` when there are no more bytes in the buffer. 215 216 When all bytes from the buffer have been read, the function `cxBufferEof()` returns true. 217 218 All read functions advance the position in the buffer by the number of read bytes. 219 220 > When you want to read from a buffer that you previously used for writing, do not forget to set the position 221 > in the buffer to zero, e.g. by calling `cxBufferSeek()`. 222 223 ## Reset and Clear 224 225 ```C 226 #include <cx/buffer.h> 227 228 void cxBufferReset(CxBuffer *buffer); 229 230 void cxBufferClear(CxBuffer *buffer); 231 232 size_t cxBufferPop(CxBuffer *buffer, 233 size_t size, size_t nitems); 234 ``` 235 236 The function `cxBufferReset()` sets both size and position of the buffer to zero, 237 and `cxBufferClear()` additionally uses `memset()` to set every byte in the buffer to zero. 238 239 > When clearing the buffer, only the "live" data, i.e., bytes with indices `[0..size)`, are cleared. 240 > If you want to clear the entire buffer's memory, you would need to set the size to the capacity, first. 241 242 The function `cxBufferPop()` erases the last `nitems` with the given `size` and returns the number of items that could be erased. 243 When the buffer contains fewer bytes than requested and the number of bytes is not divisible by the item `size`, 244 the remainder will stay in the buffer. 245 246 > If the `CX_BUFFER_COPY_ON_WRITE` flag is set, `cxBufferClear()` behaves exactly like `cxBufferReset()`, 247 > because writing to the contents is disallowed. 248 >{style="note"} 249 250 ## Random Access 251 252 ```C 253 #include <cx/buffer.h> 254 255 int cxBufferSeek(CxBuffer *buffer, off_t offset, int whence); 256 ``` 257 258 The function `cxBufferSeek()` is a `fseek()`-like function that sets the current position in the buffer 259 relative to the start (when `whence` is `SEEK_SET`), the current position (when `whence` is `SEEK_CUR`), 260 or end (when `whence` is `SEEK_END`) of the buffer. 261 262 If the resulting position is negative, or larger than the size (i.e. the first unused byte), 263 this function returns non-zero and sets `errno` to `EINVAL`. 264 265 > Note that the behavior of `cxBufferSeek()` when seeking beyond the end of the buffer is not exactly the same as for POSIX `fseek()`. 266 267 ## Shift Contents 268 269 ```C 270 #include <cx/buffer.h> 271 272 int cxBufferShift(CxBuffer *buffer, off_t shift); 273 274 int cxBufferShiftRight(CxBuffer *buffer, size_t shift); 275 276 int cxBufferShiftLeft(CxBuffer *buffer, size_t shift); 277 ``` 278 279 The function `cxBufferShift()` moves the contents within the buffer by the specified `shift` offset, 280 where a negative offset means a shift to the left, and a positive offset means a shift to the right. 281 It also adjusts the current position within the buffer, and in the case of a right shift also the size, by the same offset. 282 283 Data shifted to the left is always discarded when the new position of a byte would be smaller than zero. 284 When bytes are discarded, the new position of the buffer is set to zero. 285 286 When data is shift to the right, the behavior depends on the `CX_BUFFER_AUTO_EXTEND` flag. 287 If set, the function extends the buffer's capacity before moving the data. 288 Otherwise, the function discards all data that would exceed the buffer's capacity, and both the size and the position are equal to the capacity 289 (which means, `cxBufferEof()` returns `true` after the operation). 290 291 The functions `cxBufferShiftRight()` and `cxBufferShiftLeft()` accept a larger (but in both cases positive) shift offset, 292 which usually makes little sense on a 64-bit platform where `off_t` is already large enough to represent any reasonable offset. 293 You may, however, still use those functions to express more explicitly in your code in which direction you want the contents to be shifted. 294 295 ## Flushing 296 297 ```C 298 #include <cx/buffer.h> 299 300 typedef struct cx_buffer_flush_config_s { 301 size_t threshold; 302 size_t blksize; 303 size_t blkmax; 304 void *target; 305 cx_write_func wfunc; 306 } CxBufferFlushConfig; 307 308 int cxBufferEnableFlushing(CxBuffer *buffer, 309 CxBufferFlushConfig config); 310 311 size_t cxBufferFlush(CxBuffer *buffer); 312 ``` 313 314 With the function `cxBufferEnableFlushing()` you can configure a flushing strategy for the contents of the buffer. 315 316 Flushing, once enabled, may happen in the following cases: 317 1. when data is written to the buffer, the capacity is insufficient, and `CX_BUFFER_AUTO_EXTEND` is _not_ enabled 318 2. when data is written to the buffer, and the required capacity exceeds the `threshold` configuration 319 3. when `cxBufferFlush()` is called explicitly 320 321 > By combining the `CX_BUFFER_AUTO_EXTEND` flag and the `threshold` setting, 322 > you can create a buffer that initially starts with a small capacity, grows up to a certain threshold, 323 > and starts flushing only when that threshold is exceeded. 324 325 Flushing happens by invoking the `wfunc` up to `blkmax` times, writing up to `blksize` bytes from the buffer to the `target` with each call. 326 The target might not accept all bytes (i.e., the `wfunc` return value indicates that fewer items have been written than requested), 327 in which case the remaining data remains in the buffer. 328 That means the buffer is effectively [shifted](#shift-contents) left by the number of successfully flushed bytes. 329 330 > When you write large amounts of data to a buffer, multiple flush cycles might happen. 331 > After the first flush operations are completed, the reclaimed space in the buffer is filled first, but if that 332 > is not enough, another flush may be triggered within the same invocation of the write operation. 333 > 334 > That means as much data is written to the buffer and/or flushed as possible, until neither the flush target nor the buffer accept more data. 335 > {style="note"} 336 337 > The function `cxBufferFlush()` simply returns zero when flushing was not enabled via `cxBufferEnableFlushing()`. 338 339 <seealso> 340 <category ref="apidoc"> 341 <a href="https://ucx.sourceforge.io/api/buffer_8h.html">buffer.h</a> 342 </category> 343 </seealso> 344