This guide covers the FieldedText meta data system: what it is, how to create it, and how to work with field types and sequences.
Meta data (represented by the FtMeta class) is a schema that defines how to parse and format fielded text files. It describes:
Think of meta data as a "recipe" for reading/writing fielded text files.
There are 3 ways ways to create meta data:
In both cases, the Meta and its elements (such as Fields) will first be initialised with default values. So it is only necessary to specify properties which do not hold default values.
Create FtMeta and set properties in code:
import { FtDataType, FtDecimalMetaField, FtMeta } from "@pbkware/fielded-text-web";
const meta = new FtMeta();
// Set properties which are not default
meta.lineCommentChar = "#";
meta.headingLineCount = 1;
// Define fields
const nameField = meta.fieldList.new(FtDataType.String);
nameField.name = "CustomerName";
nameField.headings = ["Name"];
const amountField = meta.fieldList.new(FtDataType.Decimal);
amountField.name = "Amount";
amountField.headings = ["Amount"];
(amountField as FtDecimalMetaField).format = "N2"; // 2 decimal places
// Define root sequence
const rootSeq = meta.sequenceList.new();
rootSeq.name = "Root";
rootSeq.root = true;
rootSeq.itemList.new(nameField);
rootSeq.itemList.new(amountField);
When to use:
Load meta data from an XML string:
import { FtXmlMetaSerialization } from "@pbkware/fielded-text-web";
const xmlMeta = `<?xml version="1.0" encoding="utf-8"?>
<FieldedText HeadingLineCount="1">
<Field Name="Name" Headings="Name"/>
<Field Name="Age" DataType="Integer" Headings="Age"/>
</FieldedText>`;
// Load meta data from XML
const metaReader = new FtXmlMetaSerialization();
const meta = metaReader.deserialize(xmlMeta);
When to use:
A fielded text editor normally has the capability to interactively develop the Meta for a text file and getting immediate visual feedback on its correctness. Once the Meta is completed, it can be saved as an XML file for use with other applications.
Fielded text editors are listed at https://fieldedtext.org/software/
The diagram shows the elements within a Meta:
Root / Main
├──Fields
├──Substitutions
├──Sequences
├──Sequence Items
├──Sequence Redirects
A sequence item references a field. Note that a field can be referenced by more than one sequence item.
A sequence redirect references a sequence. Note that a sequence can be referenced by more than one sequence redirect.
The root of a Meta (FtMeta) specifies properties globally applicable to the Meta.
FieldedText supports six standard data types:
These correspond to FtDataType const object. The meta field types all descend from FtMetaField.
Programmatically, you create a new field of a particular type using the FtMetaFieldList.new() function. For example:
const field = meta.fieldList.new(FtDataType.String);
The properties on a Meta field define how a corresponding field instance in a record is read or written. The properties are accessed as:
field.name = 'CustomerName';
field.headings = ['Name'];
const dataType = field.dataType;
Some properties only exist on the type descendant. To access these it is necessary to cast:
const field = meta.fieldList.new(FtDataType.Boolean);
field.name = 'IsActive';
(field as FtBooleanMetaField).styles = FtBooleanStyles.IgnoreCase;
(field as FtBooleanMetaField).trueText = 'Yes';
(field as FtBooleanMetaField).falseText = 'No';
Float, Integer and Decimal fields have a styles and format property which determines how the numbers are formatted (and parsed) in the text file.
const field = meta.fieldList.new(FtDataType.Integer);
field.name = "Quantity";
(field as FtIntegerMetaField).styles = DotNetNumberStyles.integer;
(field as FtIntegerMetaField).format = "N0"; // No decimals, with thousands separator
Likewise, DateTime fields also have a styles and a format property.
const field = meta.fieldList.new(FtDataType.DateTime);
field.name = 'OrderDate';
(field as FtDateTimeMetaField).styles = DotNetDateTimeStyles.allowInnerWhite;
(field as FtDateTimeMetaField).format = 'yyyy-MM-dd';
The FtMetaField.constant property allows you to specify that a field contains a constant value. When set to true, the record/instance field (FtField not FtMetaField), will be initialised with the value (or null) specified in the FtMetaField. Accordingly, this record/instance field does not need to be assigned a value when writing records.
const field = meta.fieldList.new(FtDataType.String);
field.name = "Currency Symbol"; // always US
(field as FtStringMetaField).value = "$";
By default, fields are delimited. That is they are separated by a delimiter character (specified by FtMeta.delimiterChar). Delimited fields can have variable widths. However fields can also be specified as having a fixed width (FtMetaField.fixedWidth). In this case, the field's value will be padded or truncated to ensure it has the width specified by the property FtMetaField.width
const field = meta.fieldList.new(FtDataType.String);
field.width = 8;
field.fixedWidth = true;
field.valueTruncateType = FtTruncateType.TruncateChar;
field.valueTruncateChar = 'X';
field.valuePadCharType = FtPadCharType.Specified;
field.valuePadChar = ' ';
If a field is delimited, then it can be quoted - that is, its text representation is surrounded by a quote character (specified by FtMeta.quoteChar). This is necessary, when text representation contains delimiter character or new line character(s). The way the field is quoted is specified by FtMetaField.valueQuotedType. By default, this is set to FtQuotedType.Optional, meaning it is optional when reading records and when writing records, it will only be included if necessary.
const field = meta.fieldList.new(FtDataType.String);
field.valueQuotedType = FtQuotedType.Always;
The number of headings a field has is specified by the readonly property FtMetaField.headingCount. This will always reflect the value specified in FtMeta.headingLineCount. The actual headings are set in the meta with the FtMetaField.headings property. This property accepts an array of strings where each element represents the field's heading at the heading line corresponding to the element's index. The array will be truncated or padded with empty strings to ensure its length matches headingCount.
const field = meta.fieldList.new(FtDataType.String);
meta.headingLineCount = 3;
const headingCount = field.headingCount; // headingCount will be 3, as set in the meta
field.headings = ["Line 1", "Line 2", "Line 3"];
field.headings = ["Line 1"]; // Heading lines 2 and 3 will be auto-filled with empty strings for this field
field.headings = ["Line 1", "Line 2", "Line 3", "Line 4"]; // Extra heading lines will be ignored for this field
Many of the heading related properties specified at the Meta level can also be specified at the field level (eg. headingQuotedType). If they are specified at field level, they override the value specified at the Meta level for that field.
Meta Sequences (FtMetaSequence) define the order and grouping of fields in records. A record always has one root sequence (specified by the sequence's root property set to true). However it can contain extra sequences if these are invoked by sequence redirects.
A meta sequence consists of list of meta sequence items (FtMetaSequenceItem). Each item references a Meta Field (FtMetaField). It is the order of the sequence items in the sequence which determines the order of the fields in the record for that sequence.
For example, in the code snippet below, the fields will appear in records with the order: field2, field3, field1.
const field1 = meta.fieldList.new(FtDataType.Integer);
const field2 = meta.fieldList.new(FtDataType.String);
const field3 = meta.fieldList.new(FtDataType.Boolean);
const rootSequence = meta.sequenceList.new();
rootSequence.name = "Root";
rootSequence.root = true; // Mark as root sequence
// Add fields in order
rootSequence.itemList.new(field2);
rootSequence.itemList.new(field3);
rootSequence.itemList.new(field1);
A sequence item can also specify sequence redirects.
Note that in XML format, the root sequence does not need to be specified if there are no redirects. Instead it will be inferred from the list of fields in the XML and the order of those fields in the XML.
A sequence redirect allows a record to contain different fields based on the value contained in a key field.
See the Advanced Guide for details on sequence redirects.
Meta can be serialized/deserialized to/from XML with the FtXmlMetaSerialization class.
The serialize() function generates a XML string from a FtMeta. The optional options parameter provides some control over formatting of the XML. It also has an option explicitIndicies.
By default, the position (or index) of a sequence item is inferred from its position/order in the XML string. Likewise, the index of a field when inferring the root index, is normally determined by its position/order in the XML string. However the <Field> and <Item> XML elements can have an Index attribute. This explicitly specifies the index of the item (or field) in a sequence. When an XML string is deserialized, the deserialization will honour the value in index attributes as much as possible. To include these Index attributes, set explicitIndicies to true.
const options: FtMetaSerializerOptions = {
explicitIndices: true,
indentChars: " "
}
const xml = FtXmlMetaSerialization.serialize(meta, options);
The deserialize() function converts from an XML string to FtMeta. If it finds any errors in the XML, it will add warnings in the (optional) warnings parameter and attempt to work around them. For example, if an invalid property value is specified, it will use the default value for that property instead.
const metaXml = `<?xml version="1.0" encoding="utf-8"?>
<FieldedText HeadingLineCount="1">
<Field Name="Name" Headings="Name"/>
<Field Name="Age" DataType="Integer" Headings="Age"/>
</FieldedText>`;
const warnings = new Array<string>();
const meta = FtXmlMetaSerialization.deserialize(metaXml, warnings);
const noErrors = warnings.length === 0