MapLibre Tile Specification v2
Note
This is a live specification that evolves continuously. Features marked as are under active development and may change in future versions. Stable features are those without experimental tags.
Warning
MLT v2 is experimental. The layout on this page can change without a version bump, and no compatibility is promised between releases. Do not store v2 tiles in a cache that cannot be invalidated, and do not ship a v2 decoder that cannot be updated.
v2 is implemented in the Rust mlt-core crate behind the unstable-v2 cargo feature.
This page describes that implementation.
Where the two disagree, the implementation is authoritative.
v2 uses the same data model as v1: tiles, layers, features, columns and streams as described in the Overview. Only the byte layout differs. A v2 layer decodes into the same in-memory representation as a v1 layer.
Differences from v1¶
| v1 | v2 | |
|---|---|---|
| Stream role | stream_type byte per stream |
Implied by position |
| Stream value count | num_values varint per stream |
Omitted when it equals the context count |
| Stream byte length | byte_length varint per stream |
Omitted on a raw stream, whose count and element width give it |
| Geometry stream set | stream_count + a stream_type byte each |
One GeoLayout nibble in the layer header |
| Nullability | Boolean-RLE Present stream |
Raw bitfield, shareable between columns |
| Column metadata | All columns' metadata, then all columns' data | Metadata and data adjacent per column |
| RLE | All runs, then all values, with runs in the header |
Interleaved (run, value) pairs, no run count |
| Integer payloads | None, VarInt, FastPFOR (256-value big-endian blocks) | Adds bit packing. FastPFOR uses 128-value little-endian blocks |
| Float payloads | Raw IEEE 754 words | Adds ALP and float dictionaries |
| String dictionaries | Plain, FSST | Adds front coding |
| Vertex-scoped values | Not implemented | M-value columns |
| Maps and lists | Not implemented | Shredded into one leaf column per path |
Encoders MUST select each encoding by comparing the stored size of each candidate.
Tile Layout¶
Unchanged from v1.
A tile is a concatenation of tagged layer records.
A v2 layer has tag = 0x02.
A tile MAY contain both v1 and v2 layers. A decoder MUST skip a layer whose tag it does not know.
Layer Body¶
body := [string name] non-empty, UTF-8, VarInt length prefix
[varint extent] non-zero
[varint feature_count]
[u8 layer_layout]
[shared presence bitfield] * n n = the layout byte's shared presence count
geometry_section
[varint column_counts] see Column Counts
column * column_count
m_value_section only when the layout byte's m-value bit is set
The body MUST end exactly at the layer's size.
Trailing bytes are an error.
feature_count is the default value count for every stream in the layer.
See Value Count.
Column Counts¶
column_count is the number of ids and properties, not counting geometry or m-values.
m_value_count is the number of m-value columns.
When the layout byte's m-value bit is clear, column_counts is column_count as a plain varint.
When it is set, column_counts is the Morton code of both counts as a varint: bit i of column_count sits at bit 2i and bit i of m_value_count at bit 2i + 1.
A layer with up to 15 counted columns and up to 7 m-value columns fits both counts in one byte.
m_value_count MUST be non-zero when the bit is set, so a code whose odd bits are all clear MUST be rejected.
| Bit | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
|---|---|---|---|---|---|---|---|---|
| Field | varint continuation | column_count bit 3 | m_value_count bit 2 | column_count bit 2 | m_value_count bit 1 | column_count bit 1 | m_value_count bit 0 | column_count bit 0 |
Layer Layout Byte¶
| Bits | Field |
|---|---|
| 7 | An m-value section ends the body |
| 6-4 | Number of shared presence bitfields, 0-7 |
| 3-0 | Geometry layout |
Shared Presence Bitfields¶
A presence bitfield holds one bit per feature, set when the feature has a value. Several columns MAY reference the same bitfield.
The n shared bitfields follow the layout byte back to back, in index order.
Each is ceil(feature_count / 8) bytes, LSB-first: bit i % 8 of byte i / 8 is feature i.
Bits past feature_count in the final byte are padding and MUST be ignored.
A column references a shared bitfield through its presence nibble. Encoders SHOULD only share a bitfield that more than one column references. Encoders SHOULD store an unshared bitfield inline. Shared bitfields are ordered by the first column that references each.
Geometry Section¶
The geometry column is not counted in column_count.
It always follows the shared presence bitfields.
Which streams it contains is given by the geometry layout nibble.
geometry_section := [types stream] count = feature_count
[geo lengths] \
[part lengths] } present per the layout
[ring lengths] /
[triangle lengths] \ tessellated layouts only
[index buffer] /
[vertex stream] every vertex, or the distinct ones
[vertex offsets] dictionary layouts only
Streams appear in exactly this order. Each stream's role is given by its position.
Geometry Layout¶
The low nibble of the layer layout byte:
| Code | Name | Streams, in order |
|---|---|---|
0x0 |
Points | Types, Vertices |
0x1 |
PointsDict | Types, VertexDict, VertexOffsets |
0x2 |
MultiPoints | Types, GeoLengths, Vertices |
0x3 |
MultiPointsDict | Types, GeoLengths, VertexDict, VertexOffsets |
0x4 |
Lines | Types, PartLengths, Vertices |
0x5 |
LinesDict | Types, PartLengths, VertexDict, VertexOffsets |
0x6 |
MultiLines | Types, GeoLengths, PartLengths, Vertices |
0x7 |
MultiLinesDict | Types, GeoLengths, PartLengths, VertexDict, VertexOffsets |
0x8 |
Polygons | Types, PartLengths, RingLengths, Vertices |
0x9 |
PolygonsDict | Types, PartLengths, RingLengths, VertexDict, VertexOffsets |
0xA |
MultiPolygons | Types, GeoLengths, PartLengths, RingLengths, Vertices |
0xB |
MultiPolygonsDict | Types, GeoLengths, PartLengths, RingLengths, VertexDict, VertexOffsets |
0xC |
TessPolygons | Types, TriLengths, IndexBuffer, Vertices |
0xD |
TessPolygonsWithOutlines | Types, GeoLengths, PartLengths, RingLengths, TriLengths, IndexBuffer, Vertices |
0xE and 0xF are unassigned and MUST be rejected.
There is no layout with ring lengths but without part lengths.
A tessellated layer either has no outline topology (0xC) or all three outline streams (0xD).
A tessellated layer that needs only some of the outline streams uses 0xD and writes the others as empty streams.
The meaning of the topology streams, the length threshold rules, componentwise delta coding, Hilbert-sorted vertex dictionaries and Morton codes are unchanged from v1.
Columns¶
column_count columns follow the geometry section.
column := [u8 column_type]
[string name] unless the data type is Id or LongId
[presence bitfield] only when the presence nibble is Inline
[data streams] one, or a set, per the data type
An inline presence bitfield is ceil(feature_count / 8) LSB-first bytes, laid out like a shared one.
Column Names¶
A layer has one namespace of column names, so a name identifies exactly one column of it and a style expression such as ["get", "foo"] resolves to that column:
| Column | Name |
|---|---|
| A counted column of values | Its name field |
| A shared dictionary child | The group's prefix followed by the child's name, concatenated with nothing between |
| A nested column | Its name field |
| An m-value column | Its name field |
A name a second column repeats MUST be rejected, whichever kinds of column repeat it, and an encoder MUST NOT write such a layer.
An Id or LongId column has no name field, and the geometry column is not a counted column, so neither takes a name.
A shared dictionary group's own name is a prefix rather than a column name, and MAY repeat one.
A struct field name and a map key name a value inside their column, not a column, and are unique only where their own sections say.
Column Type Byte¶
| Bits | Field |
|---|---|
| 7-4 | Presence, or the dictionary kind when the data type is 0xF |
| 3-0 | Data type |
Data Types¶
| Code | Type | Notes |
|---|---|---|
0x0 |
Id |
Feature id, up to 32 bits. No name field. |
0x1 |
LongId |
Feature id, up to 64 bits. No name field. |
0x2 |
Bool |
|
0x3 |
Int8 |
|
0x4 |
UInt8 |
|
0x5 |
Int32 |
|
0x6 |
UInt32 |
|
0x7 |
Int64 |
|
0x8 |
UInt64 |
|
0x9 |
Float |
IEEE 754 binary32 |
0xA |
Double |
IEEE 754 binary64 |
0xB |
String |
Layout given by the leading stream, see String Columns |
0xC |
Struct |
A fixed set of named fields, see Nested Properties |
0xD |
List |
A repeated value, see Nested Properties |
0xE |
Map |
String keys chosen per value, see Nested Properties |
0xF |
Shared dictionary | See Shared Dictionary Columns |
There are no nullable variants of these codes. Nullability is given by the presence nibble. There is no geometry code, since the geometry column is not a counted column.
A layer MUST contain at most one Id or LongId column.
Presence Nibble¶
| Nibble | Meaning |
|---|---|
0 |
Every feature has a value. No bitfield is stored. |
1 |
An inline bitfield follows the type byte and the name, if any. |
2-8 |
The layer's shared bitfield at index nibble - 2. |
9-15 |
Reserved, MUST be rejected. |
A shared reference at or past the count in the layout byte MUST be rejected.
The presence nibble determines the column's value count: feature_count for nibble 0, otherwise the population count of the referenced bitfield.
Scalar Columns¶
Boolean, integer and id columns have a single data stream.
A float or double column has a single data stream, unless its logical encoding is Dict.
A Dict float column has a second stream holding the distinct values that the first stream's codes index into.
String Columns¶
The extension bits of the leading stream's encoding byte give the column's layout:
| Extension | Layout | Streams, in order |
|---|---|---|
00 |
Plain | Lengths, Values |
01 |
Dict | Codes, DictLengths, DictValues |
10 |
FSST | Lengths, SymbolLengths, SymbolTable, Corpus |
11 |
FsstDict | Codes, DictLengths, SymbolLengths, SymbolTable, Corpus |
Lengths and Codes hold one value per present value.
The remaining streams carry their own counts, or, for byte blobs, take their count from byte_length.
When the DictValues or Corpus stream's encoding byte names front coding, the preceding lengths stream holds 2N values: N shared-prefix lengths, then N suffix lengths.
Shared Dictionary Columns¶
Data type 0xF introduces a dictionary followed by the columns that index into it.
Its high nibble gives the dictionary kind instead of a presence:
| Nibble | Kind | Corpus streams, in order |
|---|---|---|
0 |
Plain | DictLengths, DictValues |
1 |
FSST | DictLengths, SymbolLengths, SymbolTable, Corpus |
Other nibbles are reserved and MUST be rejected.
shared_dict := [u8 column_type] low nibble 0xF
[string name] the group's shared prefix
[varint child_count]
[corpus streams] per the kind above
child * child_count
child := [u8 column_type] data type MUST be String
[string name]
[presence bitfield] only when the presence nibble is Inline
[codes stream] one dictionary index per present value
Each child has its own presence nibble and MAY reference any of the layer's shared bitfields.
Front coding of the dictionary is given by the encoding byte of the last corpus stream, as for a lone string column.
Nested Properties¶
A property whose value is a map or a list, nestable.
v2 shreds a nested value the way ORC does. The column is a tree of nodes. Every leaf of the tree holds one flat stream set, encoded exactly as a column of its data type. The structure lives in the presence and length streams of the interior nodes, never beside the values. A key that every feature shares is written once, in the tree, rather than once per feature.
The tree is written depth first, each node's streams where the node sits, following v2's rule that a column's metadata and its data are adjacent.
nested_column := [u8 column_type] presence nibble over 0xC, 0xD or 0xE
[string name]
[presence bitfield] only when the presence nibble is Inline
body of the kind the type byte named
A nested column is one entry of column_count, however many leaves it shreds into.
Its name comes from the layer's one namespace of column names.
The type byte is an ordinary column type byte: the high nibble is the column's presence over the layer's features, and MAY name a shared bitfield.
The low nibble MUST be 0xC, 0xD or 0xE; a scalar root is an ordinary column and MUST be written as one.
Node Type Byte¶
Every node below the root begins with one:
| Bits | Field |
|---|---|
| 7-4 | Node presence |
| 3-0 | Data type |
0x2-0xB name a leaf, 0xC-0xE name an interior node.
0x0, 0x1 and 0xF MUST be rejected: a feature id belongs to a feature, and a shared dictionary introduces counted columns.
Node Presence Nibble¶
| Nibble | Meaning |
|---|---|
0 |
Every value the parent hands this node is present. Nothing is stored. |
1 |
A presence stream follows the node type byte, and the field name if there is one. |
2-15 |
Reserved, MUST be rejected. |
A node below the root cannot use the layer's shared bitfields.
Those are feature_count bits long, and only the root of a nested column runs over features.
A presence stream is a Bool-family stream of logical None over physical 00 or 01, one bit per value the parent hands the node, LSB-first.
It is a stream rather than the raw bitfield a column writes, because under a List or a Map the number of bits is not known until the lengths have been decoded, and the stream header lets a decoder step over it either way.
Counts¶
Each node is handed a number of values by its parent, its parent count:
| Node | Parent count |
|---|---|
| The root | The column's value count: feature_count, or its presence bitfield's population count |
| A struct field | The struct node's present count |
| A list element | The sum of the list node's lengths |
| A map key or map value | The sum of the map node's lengths |
A node's present count is its parent count under nibble 0, and its presence stream's population count otherwise.
That is what its data streams hold one value each of.
A node's streams take their implied count from this context, exactly as a column's do, with one exception.
The sum of a lengths stream is not known until its payload is decoded, which a decoder may defer, so nothing implies a count at or below the first List or Map on the path from the root.
The boundary sits at that node rather than below it: the node's own streams are already past it.
There, as in an m-value column, bit 7 of the encoding byte MUST be 1 on each of:
| Stream | Written by |
|---|---|
| The lengths stream | Every List and Map node, the first one included |
| The leading key stream | Every Map node |
| The presence stream | Every node that has one, the first List or Map included |
| The leading data stream | Every leaf |
A List or a Map writes a count on its own lengths and presence streams even where its parent count would imply one, so that reading a node's header never depends on what sat above the node.
An encoder SHOULD set bit 7 on every other stream that can carry one, and a decoder MUST accept a count wherever one is written.
The streams after a string stream set's leading one read as they do in a string column.
A byte blob is the exception it always is: bit 7 MUST be 0 and its count stays byte_length.
A decoder that has decoded both a lengths stream and the counts below it MUST reject a disagreement.
A nested column MUST NOT nest more than 8 levels deep, counting the root as the first.
Struct Nodes¶
0xC. A fixed set of named, individually typed fields.
struct_body := [varint field_count] non-zero
field * field_count
field := [u8 node_type]
[string field_name]
[presence stream] only when the node presence nibble is 1
body
A field reads like a counted column: its type byte, then its name, then its nulls, then its data. Field names MUST be unique within their struct. A field present on every value of its struct stores no presence at all, which is what makes a struct the cheap shredding of a stable key set: the key is paid for once, in the tree.
List Nodes¶
0xD. A repeated value of one type.
list_body := [lengths stream] Int family, one length per present list
[u8 node_type] the element node
[presence stream] only when the node presence nibble is 1
body
The lengths stream holds element counts, not offsets, as the geometry section's do. The element node has no name.
A list that is null and a list that is empty are different: a null list has its presence bit clear and no length, an empty list has its presence bit set and a length of 0.
Map Nodes¶
0xE. String keys chosen per value, over one value type.
map_body := [lengths stream] Int family, one length per present map
[key streams] a string column's stream set, one key per entry
[u8 node_type] the value node
[presence stream] only when the node presence nibble is 1
body
The keys are the streams a string column holds, laid out per the extension bits of the leading one, holding one key per entry of every present map. They carry no node type byte and no presence: a key is neither null nor of any other type, so neither is representable. The value node has no name.
A Map and a Struct express the same thing when every key holds the same type.
A Struct spends one presence stream per key and nothing per entry; a Map spends one key per entry and nothing per key.
Encoders MUST pick between them by comparing the stored size, as they do for every other encoding.
A value whose keys hold different types is only a Struct.
Leaf Nodes¶
0x2-0xB. A leaf holds exactly the data streams a column of the same data type holds: one stream for a boolean or integer, one or two for a float, and the set its leading stream's extension bits name for a string.
It reads them against its own count, and is otherwise the same column.
Where Nested Columns May Appear¶
| Position | Nested |
|---|---|
| A counted column | Allowed |
| A node inside a nested column | Allowed |
| An m-value column | MUST be rejected |
| A shared dictionary child | MUST be rejected, a child MUST be String |
M-Values¶
An m-value is a measurement taken at a vertex rather than at a feature:
- a distance along a road,
- a timestamp per GPS fix,
- a width that varies along a river.
A layer stores them as named columns that run over its vertices instead of its features.
The section is present only when bit 7 of the layer layout byte is set, and is then the last thing in the layer body. Its column count comes from the layer's column counts varint, so the section itself starts with its first column.
m_value_section := m_value_column * m_value_count
m_value_column := [u8 column_type] presence nibble over data type, as for a column
[string name]
[presence bitfield] only when the presence nibble is Inline
[data streams] one, or a set, per the data type
An m-value column reads the same column type byte, the same presence nibble and the same data streams as a column.
Only its value count differs.
Its name comes from the layer's one namespace of column names, which the counted columns share.
An m-value column is not counted in column_count.
Data Types in M-Value Columns¶
Bool through String, codes 0x2-0xB, carry exactly the streams they carry as a property column.
0x0, 0x1 and 0xC-0xF are reserved and MUST be rejected.
A vertex carries a measurement, not a structure.
The Vertex Sequence¶
A layer's vertex sequence is every vertex of every feature, in feature order, as the geometry section's topology streams lay them out.
Each feature holds one contiguous run of it, of vertex_count(f) vertices.
An m-value column holds one value per vertex of that run, in the same order.
- A polygon ring's closing vertex is not stored, so it has no m-value.
- Under a dictionary layout the sequence is the one the vertex offsets stream spells out, not the distinct vertices the vertex stream holds. Two vertices that share a dictionary entry still have an m-value each.
- Under
TessPolygonsWithOutlinesthe sequence is the outline vertices, which the index buffer indexes into.
Nulls¶
M-value columns are nullable, but they are at the feature level, not the vertex level. This means a single vertex cannot be null. A feature's m-value can be null though.
Value Count¶
A column's value count is the sum of vertex_count(f) over the features whose presence bit is set, which is every feature under presence nibble 0.
That count is only known once the geometry topology has been decoded, which a decoder may defer, so it is not an implied count.
An m-value column's leading data stream MUST set bit 7 of its encoding byte and write the count explicitly.
The remaining streams of a string or float-dictionary column carry their own counts, as they do on a counted column, and a decoder that reads a non-blob one without an explicit count takes the leading stream's count as the implied one.
An encoder SHOULD write an explicit count on every stream that can carry one, since none of those counts are implied here.
A byte blob is the exception. Bit 7 MUST be 0 on it and its count stays byte_length, as on any other column.
A decoder that has decoded both the geometry and an m-value column MUST reject a count that disagrees with the geometry.
Values are one flat sequence across feature boundaries. Delta coding and RLE run through them without a break at each feature.
Geometry Layouts¶
An m-value section requires a geometry layout whose topology gives every feature's vertex count.
| Layout | M-values |
|---|---|
0x0 Points, 0x1 PointsDict |
MUST be rejected |
0x2-0xB |
Allowed |
0xC TessPolygons |
MUST be rejected |
0xD TessPolygonsWithOutlines |
Allowed |
A point layer holds one vertex per feature, so a vertex-scoped column would be a property column with extra rules.
Encode it as a property column.
TessPolygons carries no outline topology, so no feature's vertex count can be read from it.
A tessellated layer that needs m-values uses 0xD.
Streams¶
stream := [u8 encoding_byte]
[varint num_values] only when bit 7 of the encoding byte is set
[varint byte_length] unless logical `None` over physical `00`, see Byte Length
[varint parameters] per the logical encoding, see Parameters
[u8 payload[byte_length]]
Encoding Byte¶
| Bits | Field |
|---|---|
| 7 | An explicit num_values varint follows |
| 6-4 | Logical encoding, numbered within the stream's family |
| 3-2 | Physical encoding, interpreted per logical encoding |
| 1-0 | Extension. Carries the layout of a string column's leading stream. MUST be 0 on every other stream. |
Value Count¶
Bit 7 is set only when the stream's value count differs from the implied count. The implied count is:
feature_count, for the geometry types stream and for the data stream of a column with presence nibble0- the presence bitfield's population count, for the data stream of any other column
byte_length, for a byte blob
Bit 7 MUST be 0 on a byte blob.
Bit 7 MUST be 1 on an m-value column's leading data stream, which has no implied count.
Bit 7 MUST be 1 on the streams a nested node at or below a List or Map writes, which have none either.
For an RLE stream the value count is the decoded element count.
The number of (run, value) pairs is not stored.
A decoder reads pairs until byte_length is exhausted.
Byte Length¶
byte_length is present unless the stream is logical None over physical 00.
On logical None the payload is the elements as they are, so that pattern says the length follows from the value count and the element width the stream's type fixes:
| Stream | byte_length |
|---|---|
The data stream of an Int64, UInt64 or LongId column |
num_values * 8 |
Any other Int, Str or Vertex stream, which holds 32-bit words |
num_values * 4 |
A Float column's values |
num_values * 4 |
A Double column's values |
num_values * 8 |
A Bool bitmap |
ceil(num_values / 8) |
A byte blob takes its count from byte_length, so it MUST NOT use the pattern.
Every other logical encoding that reads the physical field MUST reject it.
RLE, DeltaRLE and BitPacked reserve the field as 0 and still write byte_length.
An encoder SHOULD use it wherever it is allowed, since it is one varint shorter.
Families¶
The logical encodings available to a stream depend on what it holds.
Each family numbers its own members from 0.
| Family | Used by | 0 |
1 |
2 |
3 |
4 |
|---|---|---|---|---|---|---|
| Int | Lengths, offsets, ids, integer columns, geometry topology, nested list and map lengths | None | Delta | RLE | DeltaRLE | BitPacked |
| Str | A string column's leading stream | None | Delta | RLE | DeltaRLE | BitPacked |
| Bool | A boolean column's data stream, a nested node's presence stream | None | RLE† | |||
| Float | A float or double column's data stream | None | RLE† | ALP | Dict | |
| Vertex | The geometry vertex stream | None | Delta | Componentwise Delta | Morton | |
| Bytes | Byte blobs: string values, dictionaries, FSST symbol tables | None | FrontCoded |
† Reserved but not implemented. A decoder MUST reject it. The payload format of RLE over booleans is not yet decided.
The Str family has the same members as Int.
It differs only in the use of the extension bits.
A code not listed for the stream's family MUST be rejected.
Physical Field¶
For a stream of integer words, meaning the Int, Str and Vertex families and the code or scaled-integer stream of a Float Dict or Alp encoding:
| Bits | Physical |
|---|---|
00 |
None, without byte_length. Only on logical None, see Byte Length |
01 |
None: fixed-width little-endian words |
10 |
VarInt |
11 |
SIMD-FastPFOR, 128-value little-endian blocks |
For a stream of opaque fixed-width elements, meaning a Bool bitmap, raw Float values or a Bytes blob:
| Bits | Physical |
|---|---|
00 |
Elements as they are, without byte_length. Only on logical None, and never on a blob, see Byte Length |
01 |
Elements as they are |
10, 11 |
Unassigned, MUST be rejected |
RLE, DeltaRLE and BitPacked define their own physical layout.
The physical field MUST be 0 for them.
Note
v1 FastPFOR uses 256-value big-endian blocks and v2 FastPFOR uses 128-value little-endian blocks. They are not interchangeable. A decoder MUST select the variant by the layer tag.
Parameters¶
Two logical encodings carry parameters, written after byte_length:
| Encoding | Parameters |
|---|---|
| ALP | e (varint), f (varint), base (ZigZag varint). See ALP. |
| Morton | bits (varint), shift (varint). The grid the codes are laid on. |
Payload Encodings¶
Logical encodings whose payload differs from v1, or which v1 does not have:
- RLE / DeltaRLE: interleaved varint
(run_length, value)pairs with no run count. v1 writes all runs, then all values, and storesrunsin the header. - BitPacked:
[u8 width][ceil(count * width / 8) bytes]. Values are laid LSB-first end to end, so valueioccupies bitsi * width ...widthis1-32. - Dict on a float column: one code per element, followed by a second stream of the distinct values.
- Morton on a vertex stream: always delta-coded over a sorted dictionary. v2 has no plain or RLE Morton encoding.
- FrontCoded on a byte blob: a sorted dictionary stored as shared-prefix lengths plus suffixes.
All of these are specified in Encoding Definitions.
Examples¶
Every fixture under test/synthetic/0x02/ can be annotated as below hexdump.
If you are building an decoder, the .json snapshot are what an decoder MUST produce from it.
The same annotation can be produced for any tile with the mlt CLI:
A single Point at (13, 42) with id 100.
The types stream has feature_count values, so bit 7 of its encoding byte is clear.
The vertex stream has two words for one point, so it sets bit 7 and writes its count.
00000000 | layer[0] (25 B)
00000000 18 . | size: 24 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 01 . | feature_count: 1
0000000b 00 . | layout: 0x00 Points
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 0000 -> geometry layout = Points
0000000c | geometry (8 B)
0000000c | types (3 B)
0000000c | header (2 B)
0000000c 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 1 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000000d 01 . | byte_length: 1
0000000e 00 . | data [Length(VarBinary) Int(None)/VarInt, 1 values, 1 B]
| decoded: [0]
0000000f | vertices (5 B)
0000000f | header (3 B)
0000000f a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000010 02 . | num_values: 2
00000011 02 . | byte_length: 2
00000012 1a 54 .T | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 2 values, 2 B]
| decoded: [13, 42]
00000014 01 . | column_count: 1
00000015 | columns (4 B)
00000015 | column[0] Id (4 B)
00000015 00 . | type: 0x00 AllPresent Id
| └ bits 7-4 = 0000 -> presence = AllPresent
| └ bits 3-0 = 0000 -> data type = Id
00000016 | data (3 B)
00000016 | header (2 B)
00000016 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 1 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000017 01 . | byte_length: 1
00000018 64 d | data [Data(None) Int(None)/VarInt, 1 values, 1 B]
| decoded: [100]
The Polygons geometry layout: part and ring length streams, no GeoLengths, no vertex offsets.
00000000 | layer[0] (39 B)
00000000 26 & | size: 38 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 01 . | feature_count: 1
0000000b 08 . | layout: 0x08 Polygons
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 1000 -> geometry layout = Polygons
0000000c | geometry (26 B)
0000000c | types (3 B)
0000000c | header (2 B)
0000000c 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 1 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000000d 01 . | byte_length: 1
0000000e 02 . | data [Length(VarBinary) Int(None)/VarInt, 1 values, 1 B]
| decoded: [2]
0000000f | part_lengths (3 B)
0000000f | header (2 B)
0000000f 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 1 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000010 01 . | byte_length: 1
00000011 02 . | data [Length(Parts) Int(None)/VarInt, 1 values, 1 B]
| decoded: [2]
00000012 | ring_lengths (5 B)
00000012 | header (3 B)
00000012 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000013 02 . | num_values: 2
00000014 02 . | byte_length: 2
00000015 03 03 .. | data [Length(Rings) Int(None)/VarInt, 2 values, 2 B]
| decoded: [3, 3]
00000017 | vertices (15 B)
00000017 | header (3 B)
00000017 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000018 0c . | num_values: 12
00000019 0c . | byte_length: 12
0000001a 16 68 78 28 13 63 08 58 3b 13 28 27 .hx(.c.X;.(' | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 12 values, 12 B]
| decoded: [11, 52, 71, 72, 61, 22, 65, 66, 35, 56, 55, 36]
00000026 00 . | column_count: 0
Two columns referencing one shared bitfield, and an interleaved-RLE types stream.
00000000 | layer[0] (43 B)
00000000 2a * | size: 42 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 04 . | feature_count: 4
0000000b 10 . | layout: 0x10 Points
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 001 -> shared presence bitfields = 1
| └ bits 3-0 = 0000 -> geometry layout = Points
0000000c | shared_presence (1 B)
0000000c 05 . | present[0] [Present Bool(None)/None, 4 values, 1 B]
| decoded: 4 present-bits: 1010
0000000d | geometry (15 B)
0000000d | types (4 B)
0000000d | header (2 B)
0000000d 20 | encoding: 0x20 logical=Rle physical=implied
| └ bit 7 = 0 -> has_explicit_count = false -> 4 values from context
| └ bits 6-4 = 010 -> logical = Rle, numbered for integer stream
| └ bits 3-2 = 00 -> physical = implied
| └ bits 1-0 = 00 -> extension = 0
0000000e 02 . | byte_length: 2
0000000f 04 00 .. | data [Length(VarBinary) Int(Rle(Interleaved { num_rle_values: 4 }))/VarInt, 4 values, 2 B]
| decoded: [0, 0, 0, 0]
00000011 | vertices (11 B)
00000011 | header (3 B)
00000011 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000012 08 . | num_values: 8
00000013 08 . | byte_length: 8
00000014 1a 54 00 00 00 00 00 00 .T...... | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 8 values, 8 B]
| decoded: [13, 42, 13, 42, 13, 42, 13, 42]
0000001c 02 . | column_count: 2
0000001d | columns (14 B)
0000001d | column[0] OptU32 "a" (7 B)
0000001d 26 & | type: 0x26 Shared(0) U32
| └ bits 7-4 = 0010 -> presence = Shared(0)
| └ bits 3-0 = 0110 -> data type = U32
0000001e 01 61 .a | name: "a"
00000020 | data (4 B)
00000020 | header (2 B)
00000020 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 2 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000021 02 . | byte_length: 2
00000022 00 02 .. | data [Data(None) Int(None)/VarInt, 2 values, 2 B]
| decoded: [0, 2]
00000024 | column[1] OptU32 "b" (7 B)
00000024 26 & | type: 0x26 Shared(0) U32
| └ bits 7-4 = 0010 -> presence = Shared(0)
| └ bits 3-0 = 0110 -> data type = U32
00000025 01 62 .b | name: "b"
00000027 | data (4 B)
00000027 | header (2 B)
00000027 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 2 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000028 02 . | byte_length: 2
00000029 00 02 .. | data [Data(None) Int(None)/VarInt, 2 values, 2 B]
| decoded: [0, 2]
A Dict string column.
The extension bits of the leading stream give the layout.
00000000 | layer[0] (70 B)
00000000 45 E | size: 69 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 02 . | feature_count: 2
0000000b 00 . | layout: 0x00 Points
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 0000 -> geometry layout = Points
0000000c | geometry (11 B)
0000000c | types (4 B)
0000000c | header (2 B)
0000000c 20 | encoding: 0x20 logical=Rle physical=implied
| └ bit 7 = 0 -> has_explicit_count = false -> 2 values from context
| └ bits 6-4 = 010 -> logical = Rle, numbered for integer stream
| └ bits 3-2 = 00 -> physical = implied
| └ bits 1-0 = 00 -> extension = 0
0000000d 02 . | byte_length: 2
0000000e 02 00 .. | data [Length(VarBinary) Int(Rle(Interleaved { num_rle_values: 2 }))/VarInt, 2 values, 2 B]
| decoded: [0, 0]
00000010 | vertices (7 B)
00000010 | header (3 B)
00000010 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000011 04 . | num_values: 4
00000012 04 . | byte_length: 4
00000013 16 68 78 28 .hx( | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 4 values, 4 B]
| decoded: [11, 52, 71, 72]
00000017 01 . | column_count: 1
00000018 | columns (46 B)
00000018 | column[0] OptStr "val" (46 B)
00000018 1b . | type: 0x1B Inline Str
| └ bits 7-4 = 0001 -> presence = Inline
| └ bits 3-0 = 1011 -> data type = Str
00000019 03 76 61 6c .val | name: "val"
0000001d 03 . | present [Present Bool(None)/None, 2 values, 1 B]
| decoded: 2 present-bits: 11
0000001e | codes (4 B)
0000001e | header (2 B)
0000001e 21 ! | encoding: 0x21 logical=Rle physical=implied
| └ bit 7 = 0 -> has_explicit_count = false -> 2 values from context
| └ bits 6-4 = 010 -> logical = Rle, numbered for string column
| └ bits 3-2 = 00 -> physical = implied
| └ bits 1-0 = 01 -> string layout = Dict
0000001f 02 . | byte_length: 2
00000020 02 00 .. | data [Offset(String) Int(Rle(Interleaved { num_rle_values: 2 }))/VarInt, 2 values, 2 B]
| decoded: [0, 0]
00000022 | dict_lengths (4 B)
00000022 | header (3 B)
00000022 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000023 01 . | num_values: 1
00000024 01 . | byte_length: 1
00000025 1e . | data [Length(Dictionary) Int(None)/VarInt, 1 values, 1 B]
| decoded: [30]
00000026 | dict_values (32 B)
00000026 | header (2 B)
00000026 04 . | encoding: 0x04 logical=None physical=WithLen
| └ bit 7 = 0 -> has_explicit_count = false -> a blob's byte length is its value count
| └ bits 6-4 = 000 -> logical = None, numbered for byte blob
| └ bits 3-2 = 01 -> physical = WithLen
| └ bits 1-0 = 00 -> extension = 0
00000027 1e . | byte_length: 30
00000028 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 AAAAAAAAAAAAAAAA | data [Data(Single) Int(None)/None, 30 values, 30 B]
00000038 41 41 41 41 41 41 41 41 41 41 41 41 41 41 AAAAAAAAAAAAAA |
| decoded: utf-8 "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
A double column stored as scaled integers, with e, f and base following the byte length.
00000000 | layer[0] (51 B)
00000000 32 2 | size: 50 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 06 . | feature_count: 6
0000000b 00 . | layout: 0x00 Points
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 0000 -> geometry layout = Points
0000000c | geometry (19 B)
0000000c | types (4 B)
0000000c | header (2 B)
0000000c 20 | encoding: 0x20 logical=Rle physical=implied
| └ bit 7 = 0 -> has_explicit_count = false -> 6 values from context
| └ bits 6-4 = 010 -> logical = Rle, numbered for integer stream
| └ bits 3-2 = 00 -> physical = implied
| └ bits 1-0 = 00 -> extension = 0
0000000d 02 . | byte_length: 2
0000000e 06 00 .. | data [Length(VarBinary) Int(Rle(Interleaved { num_rle_values: 6 }))/VarInt, 6 values, 2 B]
| decoded: [0, 0, 0, 0, 0, 0]
00000010 | vertices (15 B)
00000010 | header (3 B)
00000010 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000011 0c . | num_values: 12
00000012 0c . | byte_length: 12
00000013 1a 54 00 00 00 00 00 00 00 00 00 00 .T.......... | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 12 values, 12 B]
| decoded: [13, 42, 13, 42, 13, 42, 13, 42, 13, 42, 13, 42]
0000001f 01 . | column_count: 1
00000020 | columns (19 B)
00000020 | column[0] OptF64 "val" (19 B)
00000020 1a . | type: 0x1A Inline F64
| └ bits 7-4 = 0001 -> presence = Inline
| └ bits 3-0 = 1010 -> data type = F64
00000021 03 76 61 6c .val | name: "val"
00000025 2d - | present [Present Bool(None)/None, 6 values, 1 B]
| decoded: 6 present-bits: 101101
00000026 | data (13 B)
00000026 | header (6 B)
00000026 28 ( | encoding: 0x28 logical=Alp physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 4 values from context
| └ bits 6-4 = 010 -> logical = Alp, numbered for float column
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000027 07 . | byte_length: 7
00000028 02 . | alp_e: 2
00000029 00 . | alp_f: 0
0000002a c1 03 .. | alp_base: -225
0000002c 96 01 fa 01 f7 02 00 ....... | data [Data(None) Float(Alp(Alp { e: 2, f: 0, base: -225 }))/VarInt, 4 values, 7 B]
| decoded: [-75, 25, 150, -225]
Two vertex-scoped columns over a line layer, one of them null on some features.
00000000 | layer[0] (76 B)
00000000 4b K | size: 75 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 03 . | feature_count: 3
0000000b 84 . | layout: 0x84 Lines
| └ bit 7 = 1 -> an m-value section ends the body
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 0100 -> geometry layout = Lines
0000000c | geometry (29 B)
0000000c | types (5 B)
0000000c | header (2 B)
0000000c 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 3 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000000d 03 . | byte_length: 3
0000000e 01 01 01 ... | data [Length(VarBinary) Int(None)/VarInt, 3 values, 3 B]
| decoded: [1, 1, 1]
00000011 | part_lengths (5 B)
00000011 | header (2 B)
00000011 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 3 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000012 03 . | byte_length: 3
00000013 03 03 02 ... | data [Length(Parts) Int(None)/VarInt, 3 values, 3 B]
| decoded: [3, 3, 2]
00000016 | vertices (19 B)
00000016 | header (3 B)
00000016 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000017 10 . | num_values: 16
00000018 10 . | byte_length: 16
00000019 16 68 78 28 13 63 4b 18 64 3b 77 28 0f 1c 0e 0e .hx(.cK.d;w(.... | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 16 values, 16 B]
| decoded: [11, 52, 71, 72, 61, 22, 23, 34, 73, 4, 13, 24, 5, 38, 12, 45]
00000029 08 . | column_counts: columns = 0, m-values = 2
0000002a | m_values (34 B)
0000002a | m_value[0] U32 "dist" (17 B)
0000002a 06 . | type: 0x06 AllPresent U32
| └ bits 7-4 = 0000 -> presence = AllPresent
| └ bits 3-0 = 0110 -> data type = U32
0000002b 04 64 69 73 74 .dist | name: "dist"
00000030 | data (11 B)
00000030 | header (3 B)
00000030 98 . | encoding: 0x98 logical=Delta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 001 -> logical = Delta, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000031 08 . | num_values: 8
00000032 08 . | byte_length: 8
00000033 00 14 1e 31 1e 32 4f 18 ...1.2O. | data [Data(None) Int(Delta)/VarInt, 8 values, 8 B]
| decoded: [0, 10, 25, 0, 15, 40, 0, 12]
0000003b | m_value[1] OptI32 "height" (17 B)
0000003b 15 . | type: 0x15 Inline I32
| └ bits 7-4 = 0001 -> presence = Inline
| └ bits 3-0 = 0101 -> data type = I32
0000003c 06 68 65 69 67 68 74 .height | name: "height"
00000043 05 . | present [Present Bool(None)/None, 3 values, 1 B]
| decoded: 3 present-bits: 101
00000044 | data (8 B)
00000044 | header (3 B)
00000044 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000045 05 . | num_values: 5
00000046 05 . | byte_length: 5
00000047 05 08 12 04 0e ..... | data [Data(None) Int(None)/VarInt, 5 values, 5 B]
| decoded: [-3, 4, 9, 2, 7]
A Struct column of two fields, one of them null on some features.
The field names are written once, in the tree.
The nullable field's presence stream is a raw bitmap, so its header carries no byte length.
00000000 | layer[0] (64 B)
00000000 3f ? | size: 63 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 03 . | feature_count: 3
0000000b 00 . | layout: 0x00 Points
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 0000 -> geometry layout = Points
0000000c | geometry (14 B)
0000000c | types (5 B)
0000000c | header (2 B)
0000000c 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 3 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000000d 03 . | byte_length: 3
0000000e 00 00 00 ... | data [Length(VarBinary) Int(None)/VarInt, 3 values, 3 B]
| decoded: [0, 0, 0]
00000011 | vertices (9 B)
00000011 | header (3 B)
00000011 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000012 06 . | num_values: 6
00000013 06 . | byte_length: 6
00000014 16 68 78 28 13 63 .hx(.c | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 6 values, 6 B]
| decoded: [11, 52, 71, 72, 61, 22]
0000001a 01 . | column_count: 1
0000001b | columns (37 B)
0000001b | column[0] Struct "obj" (37 B)
0000001b 0c . | type: 0x0C AllPresent Struct
| └ bits 7-4 = 0000 -> presence = AllPresent
| └ bits 3-0 = 1100 -> data type = Struct
0000001c 03 6f 62 6a .obj | name: "obj"
00000020 02 . | field_count: 2
00000021 | field[0] Str "name" (19 B)
00000021 0b . | type: 0x0B AllPresent Str
| └ bits 7-4 = 0000 -> node presence = AllPresent
| └ bits 3-0 = 1011 -> data type = Str
00000022 04 6e 61 6d 65 .name | name: "name"
00000027 | lengths (5 B)
00000027 | header (2 B)
00000027 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 3 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for string column
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> string layout = Plain
00000028 03 . | byte_length: 3
00000029 02 02 02 ... | data [Length(VarBinary) Int(None)/VarInt, 3 values, 3 B]
| decoded: [2, 2, 2]
0000002c | values (8 B)
0000002c | header (2 B)
0000002c 04 . | encoding: 0x04 logical=None physical=WithLen
| └ bit 7 = 0 -> has_explicit_count = false -> a blob's byte length is its value count
| └ bits 6-4 = 000 -> logical = None, numbered for byte blob
| └ bits 3-2 = 01 -> physical = WithLen
| └ bits 1-0 = 00 -> extension = 0
0000002d 06 . | byte_length: 6
0000002e 61 62 63 64 65 66 abcdef | data [Data(None) Int(None)/None, 6 values, 6 B]
| decoded: utf-8 "abcdef"
00000034 | field[1] I32 "rank" (12 B)
00000034 15 . | type: 0x15 Stream I32
| └ bits 7-4 = 0001 -> node presence = Stream
| └ bits 3-0 = 0101 -> data type = I32
00000035 04 72 61 6e 6b .rank | name: "rank"
0000003a | present (2 B)
0000003a | header (1 B)
0000003a 00 . | encoding: 0x00 logical=None physical=NoLen
| └ bit 7 = 0 -> has_explicit_count = false -> 3 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for bool column
| └ bits 3-2 = 00 -> physical = NoLen
| └ bits 1-0 = 00 -> extension = 0
0000003b 05 . | data [Present Bool(None)/None, 3 values, 1 B]
| decoded: 3 present-bits: 101
0000003c | data (4 B)
0000003c | header (2 B)
0000003c 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 2 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000003d 02 . | byte_length: 2
0000003e 0e 12 .. | data [Data(None) Int(None)/VarInt, 2 values, 2 B]
| decoded: [7, 9]
A List of structs.
The lengths stream counts elements per feature, and it and every stream below it write their own count.
00000000 | layer[0] (67 B)
00000000 42 B | size: 66 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 02 . | feature_count: 2
0000000b 00 . | layout: 0x00 Points
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 0000 -> geometry layout = Points
0000000c | geometry (11 B)
0000000c | types (4 B)
0000000c | header (2 B)
0000000c 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 2 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000000d 02 . | byte_length: 2
0000000e 00 00 .. | data [Length(VarBinary) Int(None)/VarInt, 2 values, 2 B]
| decoded: [0, 0]
00000010 | vertices (7 B)
00000010 | header (3 B)
00000010 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000011 04 . | num_values: 4
00000012 04 . | byte_length: 4
00000013 16 68 78 28 .hx( | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 4 values, 4 B]
| decoded: [11, 52, 71, 72]
00000017 01 . | column_count: 1
00000018 | columns (43 B)
00000018 | column[0] List "items" (43 B)
00000018 0d . | type: 0x0D AllPresent List
| └ bits 7-4 = 0000 -> presence = AllPresent
| └ bits 3-0 = 1101 -> data type = List
00000019 05 69 74 65 6d 73 .items | name: "items"
0000001f | lengths (5 B)
0000001f | header (3 B)
0000001f 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000020 02 . | num_values: 2
00000021 02 . | byte_length: 2
00000022 02 01 .. | data [Length(Nested) Int(None)/VarInt, 2 values, 2 B]
| decoded: [2, 1]
00000024 | element Struct (31 B)
00000024 0c . | type: 0x0C AllPresent Struct
| └ bits 7-4 = 0000 -> node presence = AllPresent
| └ bits 3-0 = 1100 -> data type = Struct
00000025 02 . | field_count: 2
00000026 | field[0] I32 "id" (10 B)
00000026 05 . | type: 0x05 AllPresent I32
| └ bits 7-4 = 0000 -> node presence = AllPresent
| └ bits 3-0 = 0101 -> data type = I32
00000027 02 69 64 .id | name: "id"
0000002a | data (6 B)
0000002a | header (3 B)
0000002a 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000002b 03 . | num_values: 3
0000002c 03 . | byte_length: 3
0000002d 02 04 06 ... | data [Data(None) Int(None)/VarInt, 3 values, 3 B]
| decoded: [1, 2, 3]
00000030 | field[1] Str "tag" (19 B)
00000030 0b . | type: 0x0B AllPresent Str
| └ bits 7-4 = 0000 -> node presence = AllPresent
| └ bits 3-0 = 1011 -> data type = Str
00000031 03 74 61 67 .tag | name: "tag"
00000035 | lengths (6 B)
00000035 | header (3 B)
00000035 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for string column
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> string layout = Plain
00000036 03 . | num_values: 3
00000037 03 . | byte_length: 3
00000038 02 02 02 ... | data [Length(VarBinary) Int(None)/VarInt, 3 values, 3 B]
| decoded: [2, 2, 2]
0000003b | values (8 B)
0000003b | header (2 B)
0000003b 04 . | encoding: 0x04 logical=None physical=WithLen
| └ bit 7 = 0 -> has_explicit_count = false -> a blob's byte length is its value count
| └ bits 6-4 = 000 -> logical = None, numbered for byte blob
| └ bits 3-2 = 01 -> physical = WithLen
| └ bits 1-0 = 00 -> extension = 0
0000003c 06 . | byte_length: 6
0000003d 68 69 6c 6f 68 69 hilohi | data [Data(None) Int(None)/None, 6 values, 6 B]
| decoded: utf-8 "hilohi"
A Map of strings, its keys a dictionary-coded string stream set.
00000000 | layer[0] (78 B)
00000000 4d M | size: 77 (varint) - tag + body
00000001 02 . | tag: 0x02 -> Tag02
00000002 06 6c 61 79 65 72 31 .layer1 | name: "layer1"
00000009 50 P | extent: 80
0000000a 02 . | feature_count: 2
0000000b 00 . | layout: 0x00 Points
| └ bit 7 = 0 -> no m-value section
| └ bits 6-4 = 000 -> shared presence bitfields = 0
| └ bits 3-0 = 0000 -> geometry layout = Points
0000000c | geometry (11 B)
0000000c | types (4 B)
0000000c | header (2 B)
0000000c 08 . | encoding: 0x08 logical=None physical=VarInt
| └ bit 7 = 0 -> has_explicit_count = false -> 2 values from context
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000000d 02 . | byte_length: 2
0000000e 00 00 .. | data [Length(VarBinary) Int(None)/VarInt, 2 values, 2 B]
| decoded: [0, 0]
00000010 | vertices (7 B)
00000010 | header (3 B)
00000010 a8 . | encoding: 0xA8 logical=CwDelta physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 010 -> logical = CwDelta, numbered for vertex stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
00000011 04 . | num_values: 4
00000012 04 . | byte_length: 4
00000013 16 68 78 28 .hx( | data [Data(Vertex) Vertex(ComponentwiseDelta)/VarInt, 4 values, 4 B]
| decoded: [11, 52, 71, 72]
00000017 01 . | column_count: 1
00000018 | columns (54 B)
00000018 | column[0] Map "tags" (54 B)
00000018 0e . | type: 0x0E AllPresent Map
| └ bits 7-4 = 0000 -> presence = AllPresent
| └ bits 3-0 = 1110 -> data type = Map
00000019 04 74 61 67 73 .tags | name: "tags"
0000001e | lengths (5 B)
0000001e | header (3 B)
0000001e 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000001f 02 . | num_values: 2
00000020 02 . | byte_length: 2
00000021 02 02 .. | data [Length(Nested) Int(None)/VarInt, 2 values, 2 B]
| decoded: [2, 2]
00000023 | keys (18 B)
00000023 | codes (7 B)
00000023 | header (3 B)
00000023 89 . | encoding: 0x89 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for string column
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 01 -> string layout = Dict
00000024 04 . | num_values: 4
00000025 04 . | byte_length: 4
00000026 00 01 00 01 .... | data [Offset(String) Int(None)/VarInt, 4 values, 4 B]
| decoded: [0, 1, 0, 1]
0000002a | dict_lengths (5 B)
0000002a | header (3 B)
0000002a 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for integer stream
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> extension = 0
0000002b 02 . | num_values: 2
0000002c 02 . | byte_length: 2
0000002d 02 02 .. | data [Length(Dictionary) Int(None)/VarInt, 2 values, 2 B]
| decoded: [2, 2]
0000002f | dict_values (6 B)
0000002f | header (2 B)
0000002f 04 . | encoding: 0x04 logical=None physical=WithLen
| └ bit 7 = 0 -> has_explicit_count = false -> a blob's byte length is its value count
| └ bits 6-4 = 000 -> logical = None, numbered for byte blob
| └ bits 3-2 = 01 -> physical = WithLen
| └ bits 1-0 = 00 -> extension = 0
00000030 04 . | byte_length: 4
00000031 65 6e 64 65 ende | data [Data(Single) Int(None)/None, 4 values, 4 B]
| decoded: utf-8 "ende"
00000035 | value Str (25 B)
00000035 0b . | type: 0x0B AllPresent Str
| └ bits 7-4 = 0000 -> node presence = AllPresent
| └ bits 3-0 = 1011 -> data type = Str
00000036 | lengths (7 B)
00000036 | header (3 B)
00000036 88 . | encoding: 0x88 logical=None physical=VarInt
| └ bit 7 = 1 -> has_explicit_count = true -> a num_values varint follows
| └ bits 6-4 = 000 -> logical = None, numbered for string column
| └ bits 3-2 = 10 -> physical = VarInt
| └ bits 1-0 = 00 -> string layout = Plain
00000037 04 . | num_values: 4
00000038 04 . | byte_length: 4
00000039 03 04 04 04 .... | data [Length(VarBinary) Int(None)/VarInt, 4 values, 4 B]
| decoded: [3, 4, 4, 4]
0000003d | values (17 B)
0000003d | header (2 B)
0000003d 04 . | encoding: 0x04 logical=None physical=WithLen
| └ bit 7 = 0 -> has_explicit_count = false -> a blob's byte length is its value count
| └ bits 6-4 = 000 -> logical = None, numbered for byte blob
| └ bits 3-2 = 01 -> physical = WithLen
| └ bits 1-0 = 00 -> extension = 0
0000003e 0f . | byte_length: 15
0000003f 73 65 61 6d 65 65 72 68 69 6c 6c 62 65 72 67 seameerhillberg | data [Data(None) Int(None)/None, 15 values, 15 B]
| decoded: utf-8 "seameerhillberg"
Planned Features¶
The following are not yet specified or implemented.
Z-Values¶
Elevation as a third coordinate.
z could be interleaved with x and y in the vertex buffer, or stored as an m-value column.
Interleaving suits GPU upload.
A separate column compresses better and costs nothing when unused.
Which of the two v2 uses is undecided.