NAME

AG_DataSource — agar data source access

SYNOPSIS

#include <agar/core.h>

DESCRIPTION

AG_DataSource provides a generic interface to different types of data sources such as AG_FileSource for files, AG_CoreSource for fixed-size memory, AG_AutoCoreSource for autoallocated memory, AG_ConstCoreSource for read-only memory and AG_NetSocketSource for AG_Net(3) sockets.

INTERFACE

AG_DataSource *
AG_OpenFile(const char *path, const char *mode);

AG_DataSource *
AG_OpenFileHandle(FILE *f);

AG_DataSource *
AG_OpenCore(void *p, AG_Size size);

AG_DataSource *
AG_OpenConstCore(const void *p, AG_Size size);

AG_DataSource *
AG_OpenAutoCore(void);

AG_DataSource *
AG_OpenNetSocket(AG_NetSocket *ns);

void
AG_CloseDataSource(AG_DataSource *ds);

int
AG_Read(AG_DataSource *ds, void *buf, AG_Size size);

int
AG_ReadAt(AG_DataSource *ds, void *buf, AG_Size size, AG_Offset pos);

int
AG_Write(AG_DataSource *ds, const void *buf, AG_Size size);

int
AG_WriteAt(AG_DataSource *ds, const void *buf, AG_Size size, AG_Offset pos);

int
AG_ReadP(AG_DataSource *ds, void *buf, AG_Size size, AG_Size *nRead);

int
AG_ReadAtP(AG_DataSource *ds, void *buf, AG_Size size, AG_Offset pos, AG_Size *nRead);

int
AG_WriteP(AG_DataSource *ds, const void *buf, AG_Size size, AG_Size *nWrote);

int
AG_WriteAtP(AG_DataSource *ds, const void *buf, AG_Size size, AG_Offset pos, AG_Size *nWrote);

AG_Offset
AG_Tell(AG_DataSource *ds);

int
AG_Seek(AG_DataSource *ds, AG_Offset offs, enum ag_seek_mode mode);

void
AG_LockDataSource(AG_DataSource *ds);

void
AG_UnlockDataSource(AG_DataSource *ds);

AG_ByteOrder
AG_SetByteOrder(AG_DataSource *ds, AG_ByteOrder order);

int
AG_SetSourceDebug(AG_DataSource *ds, int enable);

void
AG_DataSourceInit(AG_DataSource *ds);

void
AG_DataSourceDestroy(AG_DataSource *ds);

void
AG_DataSourceSetErrorFn(AG_DataSource *ds, void (*fn)(AG_Event *), const char *fmt, ...);

void
AG_DataSourceError(AG_DataSource *ds, const char *fmt, ...);

int
AG_DataSourceRealloc(AG_CoreSource *dsCore, AG_Size size);

The AG_OpenFile() function opens the file at path, where mode is a fopen(3) style mode string. AG_OpenFileHandle() creates a new data source for a previously opened file.

The AG_OpenCore() and AG_OpenConstCore() functions create new data sources referencing the region of memory p of size bytes.

AG_OpenAutoCore() creates a new data source using dynamically-allocated memory (accessible as the data member of the structure).

AG_OpenNetSocket() creates a new data source using a network socket (see AG_Net(3)).

The AG_CloseDataSource() function closes the data source, freeing any data allocated by the AG_DataSource layer (such as the data buffer allocated by AG_OpenAutoCore()). For network sockets opened with AG_OpenNetSocket(), the underlying socket is left open.

AG_Read() reads size bytes from the data source into the destination buffer buf. AG_Write() writes size bytes from the source buffer buf to the destination data source. AG_ReadAt() and AG_WriteAt() allow a source/target position (byte offset) to be specified. If an error has occurred, return -1 with an error message, otherwise 0 on success. Partial transfers are treated as errors.

The AG_ReadP(), AG_WriteP(), AG_ReadAtP() and AG_WriteAtP() variants do not treat partial reads or writes as errors, returning the total number of bytes transferred into the nRead or nWrote argument (if not NULL). Depending on the underlying data source, a byte count of 0 may indicate either an end-of-file condition or a closed socket.

AG_Tell() returns the current position in the data source. If the underlying data source does not support this operation, a value of 0 is returned.

AG_Seek() seeks to the given position in the data source. Acceptable values for mode include AG_SEEK_SET (relative to data start), AG_SEEK_CUR (relative to current position), AG_SEEK_END (relative to data end).

The AG_LockDataSource() and AG_UnlockDataSource() functions acquire and release the exclusive lock protecting this data source, and are no-ops if thread support is disabled.

AG_SetByteOrder() sets the effective byte order of the stream. Integer read/write operations must honor this setting. Byte orders are AG_BYTEORDER_BE for big-endian and AG_BYTEORDER_LE for little-endian. To determine the byte order of the current architecture, use macro AG_BYTEORDER macro (which evaluates to either AG_BIG_ENDIAN or AG_LITTLE_ENDIAN).

AG_SetSourceDebug() enables (1) or disables (0) the inclusion and checking of serialization markers on a data source. When enabled, typed I/O routines such as AG_WriteUint8() will always write a marker (type code) before every byte, and AG_ReadUint8() will always expect a type code (and verify that it matches, or raise an exception). AG_SetSourceDebug() returns the previous setting. Serialization markers are enabled implicitely for AG_Object(3) with AG_OBJECT_DEBUG_DATA. Available only with AG_DEBUG, no-op otherwise.

The AG_DataSourceInit() and AG_DataSourceDestroy() functions are used when implementing new data source types. They are used internally by the AG_Open*() and AG_Close*() functions.

AG_DataSourceSetErrorFn() configures an alternate handler routine for data source exceptions (which can occur when using routines such as AG_ReadUint32(), for example on I/O error). From the handler routine, a pointer to the AG_DataSource can be retrieved using AG_SELF, and the error message is retrieved using AG_STRING(1). The default exception handler calls AG_FatalError(3).

The AG_DataSourceError() function raises a data source error, with the optional error message string. It is intended for use in custom I/O routines which do not return an error status. If fmt is NULL, the error is obtained from AG_GetError(3).

The AG_DataSourceRealloc() routine explicitely resizes the buffer of a data source previously created with AG_OpenAutoCore(). While the buffer is already resized automatically as data is written to the source, setting an explicit buffer size may be desirable in some situations.

INTEGER OPERATIONS

The following functions read and write integer values using the byte order specified for the data source.

Uint8
AG_ReadUint8(AG_DataSource *ds);

Sint8
AG_ReadSint8(AG_DataSource *ds);

Uint16
AG_ReadUint16(AG_DataSource *ds);

Sint16
AG_ReadSint16(AG_DataSource *ds);

Uint32
AG_ReadUint32(AG_DataSource *ds);

Sint32
AG_ReadSint32(AG_DataSource *ds);

int
AG_ReadSint32(AG_DataSource *ds, Sint32 *v);

Uint64
AG_ReadUint64(AG_DataSource *ds);

Sint64
AG_ReadSint64(AG_DataSource *ds);

void
AG_WriteUint8(AG_DataSource *ds, Uint8 value);

void
AG_WriteSint8(AG_DataSource *ds, Sint8 value);

void
AG_WriteUint16(AG_DataSource *ds, Uint16 value);

void
AG_WriteSint16(AG_DataSource *ds, Sint16 value);

void
AG_WriteUint32(AG_DataSource *ds, Uint32 value);

void
AG_WriteSint32(AG_DataSource *ds, Sint32 value);

void
AG_WriteUint64(AG_DataSource *ds, Uint64 value);

void
AG_WriteSint64(AG_DataSource *ds, Sint64 value);

void
AG_WriteUint8At(AG_DataSource *ds, Uint8 value, AG_Offset offs);

void
AG_WriteSint8At(AG_DataSource *ds, Sint8 value, AG_Offset offs);

void
AG_WriteUint16At(AG_DataSource *ds, Uint16 value, AG_Offset offs);

void
AG_WriteSint16At(AG_DataSource *ds, Sint16 value, AG_Offset offs);

void
AG_WriteUint32At(AG_DataSource *ds, Uint32 value, AG_Offset offs);

void
AG_WriteSint32At(AG_DataSource *ds, Sint32 value, AG_Offset offs);

void
AG_WriteUint64At(AG_DataSource *ds, Uint64 value, AG_Offset offs);

void
AG_WriteSint64At(AG_DataSource *ds, Sint64 value, AG_Offset offs);

The AG_Read[SU]intN() functions read a N-bit integer from ds. They swap the byte order if the host byte order differs from that of the data source.

The AG_Write[SU]intN() functions write a N-byte integer to ds.

Both AG_Read[SU]inN() and AG_Write[SU]intN() swap the byte order if the host byte order differs from that of the data source.

The AG_Write[SU]intNAt() functions write an integer to the specified position in the data source, swapping the byte order as needed.

FLOATING POINT OPERATIONS

The following routines read and write floating-point numbers in IEEE.754 representation.

float
AG_ReadFloat(AG_DataSource *ds);

double
AG_ReadDouble(AG_DataSource *ds);

void
AG_WriteFloat(AG_DataSource *ds, float f);

void
AG_WriteFloatAt(AG_DataSource *ds, float f, AG_Offset pos);

void
AG_WriteDouble(AG_DataSource *ds, double f);

void
AG_WriteDoubleAt(AG_DataSource *ds, double f, AG_Offset pos);

AG_ReadFloat() and AG_ReadDouble() read a floating-point value from the data source.

AG_WriteFloat() and AG_WriteDouble() write a floating-point value to the data source. The AG_Write*At() variants write the value at a given position.

All AG_Read*v() functions return 0 on success and -1 on failure, without raising any exceptions. The other functions will raise a data source exception if an failuer (e.g., an I/O error) occurred.

STRING OPERATIONS

The following functions read and write C strings. The serialized representation includes an unsigned 32-bit count followed by the (possibly padded or NUL-terminated) string of characters itself.

char *
AG_ReadString(AG_DataSource *ds);

char *
AG_ReadStringLen(AG_DataSource *ds, AG_Size maxLen);

AG_Size
AG_CopyString(char *buf, AG_DataSource *ds, size buf_size);

char *
AG_ReadStringPadded(AG_DataSource *ds, AG_Size len);

AG_Size
AG_CopyStringPadded(char *buf, AG_DataSource *ds, size buf_size);

char *
AG_ReadNulString(AG_DataSource *ds);

char *
AG_ReadNulStringLen(AG_DataSource *ds, AG_Size maxLen);

AG_Size
AG_CopyNulString(char *buf, AG_DataSource *ds, size buf_size);

void
AG_SkipString(AG_DataSource *ds);

void
AG_SkipStringPadded(AG_DataSource *ds);

void
AG_WriteString(AG_DataSource *ds, const char *s);

void
AG_WriteStringPadded(AG_DataSource *ds, const char *s, AG_Size len);

AG_ReadString() reads a string of up to AG_LOAD_STRING_MAX bytes from ds. On success, a newly-allocated, NUL-terminated copy of string is returned. AG_ReadStringLen() reads a string of up to maxLen bytes in length.

AG_CopyString() reads an encoded string and returns its contents into a fixed-size buffer buf of buf_size. AG_CopyString() returns the number of bytes that would have been copied were buf_size unlimited.

AG_ReadStringPadded() reads a fixed-length string record of len bytes in length. AG_CopyStringPadded() reads a fixed-length string record and copies the NUL-terminated result into a fixed-size buffer buf of buf_size.

The AG_ReadNulString(), AG_ReadNulStringLen() and AG_CopyNulString() routines read a serialized, 32-bit length-encoded string which includes the NUL termination in the encoding.

The AG_SkipString() routine skips over the string at the current position in the buffer.

The AG_WriteString() function writes a C string to a data source, in a variable-length encoding. The encoding is a 32-bit representation of strlen(3) followed by the string itself.

AG_WriteStringPadded() serializes a string into a fixed-length record composed of a 32-bit representation of strlen(3) followed by the string plus extra padding such that the serialized record is always guaranteed to be length bytes + 4 in size.

On failure, the AG_WriteString() routines raise a data source exception.

INTERNAL API

New types may be implemented by deriving the AG_DataSource structure:

typedef AG_DataSource {
	AG_Mutex lock;                  /* Lock on all operations */
	enum ag_byte_order byte_order;  /* Byte order of source */
	AG_Size wrLast;                 /* Last bytes written */
	AG_Size rdLast;                 /* Last bytes read */
	AG_Size wrTotal;                /* Total bytes written */
	AG_Size rdTotal;                /* Total bytes read */

	int (*read)(AG_DataSource *, void *, AG_Size, AG_Size *);
	int (*read_at)(AG_DataSource *, void *, AG_Size, AG_Offset,
	               AG_Size *);

	int (*write)(AG_DataSource *, const void *, AG_Size,
	             AG_Size *);
	int (*write_at)(AG_DataSource *, const void *, AG_Size,
	                AG_Offset, AG_Size *);

	AG_Offset (*tell)(AG_DataSource *);

	int (*seek)(AG_DataSource *, AG_Offset offs,
	            enum ag_seek_mode mode);

	void (*close)(AG_DataSource *);
} AG_DataSource;

The byte_order field is set by AG_SetByteOrder() and controls the endianness of integer serialization operations such as AG_ReadUint32().

The wrLast, rdLast, wrTotal and rdTotal fields keep count of the read/written bytes, and are automatically incremented by serialization operations such as AG_ReadUint32().

The read() operation reads size bytes from the data source and into buf, returning the total number of bytes read into rv. read_at() reads data at a specified offset.

The write() operation writes size bytes from buf to the data source, returning the total number of bytes written into rv. The write_at() variant writes the data at a specified offset.

tell() returns the current offset.

seek() moves to the specified offset and returns 0 on success and -1 on failure.

close() closes the data source.

EXAMPLES

The following code writes an integer, float and string to file.out:

AG_DataSource *ds;

if ((ds = AG_OpenFile("file.out", "wb")) == NULL) {
	AG_FatalError(NULL);
}
AG_WriteUint16(ds, 0x1234);
AG_WriteFloat(ds, 1.234f);
AG_WriteString(ds, "hello");
AG_CloseFile(ds);

The following code reads the data back:

AG_DataSource *ds;
Uint16 u16;
float flt;
char *s;

if ((ds = AG_OpenFile("file.out", "rb")) == NULL) {
	AG_FatalError(NULL);
}
u16 = AG_ReadUint16(ds);
flt = AG_ReadFloat(ds);
s = AG_ReadString(ds);

AG_Verbose("Read: Int=%u, Float=%f, String=\"%s\"\n", u16, flt, s);

AG_CloseFile(ds);
free(s);

SEE ALSO

AG_ByteSwap(3), AG_Intro(3), AG_Net(3), AG_Object(3), AG_Version(3)

HISTORY

A similar interface called ‘AG_Netbuf’ first appeared in Agar 1.0. The current AG_DataSource interface appeared in Agar 1.3.0. Exception handling and error-checking variants of the primitive I/O routines appeared in Agar 1.3.3. The interface to network sockets appeared in Agar 1.5.0. AG_ReadStringPadded() and AG_CopyStringPadded() appeared in Agar 1.6.0.