capnproto

language.md (31493B)

---

 ---
 layout: page
 title: Schema Language
 ---
 
 # Schema Language
 
 Like Protocol Buffers and Thrift (but unlike JSON or MessagePack), Cap'n Proto messages are
 strongly-typed and not self-describing. You must define your message structure in a special
 language, then invoke the Cap'n Proto compiler (`capnp compile`) to generate source code to
 manipulate that message type in your desired language.
 
 For example:
 
 {% highlight capnp %}
 @0xdbb9ad1f14bf0b36;  # unique file ID, generated by `capnp id`
 
 struct Person {
   name @0 :Text;
   birthdate @3 :Date;
 
   email @1 :Text;
   phones @2 :List(PhoneNumber);
 
   struct PhoneNumber {
     number @0 :Text;
     type @1 :Type;
 
     enum Type {
       mobile @0;
       home @1;
       work @2;
     }
   }
 }
 
 struct Date {
   year @0 :Int16;
   month @1 :UInt8;
   day @2 :UInt8;
 }
 {% endhighlight %}
 
 Some notes:
 
 * Types come after names. The name is by far the most important thing to see, especially when
   quickly skimming, so we put it up front where it is most visible.  Sorry, C got it wrong.
 * The `@N` annotations show how the protocol evolved over time, so that the system can make sure
   to maintain compatibility with older versions. Fields (and enumerants, and interface methods)
   must be numbered consecutively starting from zero in the order in which they were added. In this
   example, it looks like the `birthdate` field was added to the `Person` structure recently -- its
   number is higher than the `email` and `phones` fields. Unlike Protobufs, you cannot skip numbers
   when defining fields -- but there was never any reason to do so anyway.
 
 ## Language Reference
 
 ### Comments
 
 Comments are indicated by hash signs and extend to the end of the line:
 
 {% highlight capnp %}
 # This is a comment.
 {% endhighlight %}
 
 Comments meant as documentation should appear _after_ the declaration, either on the same line, or
 on a subsequent line. Doc comments for aggregate definitions should appear on the line after the
 opening brace.
 
 {% highlight capnp %}
 struct Date {
   # A standard Gregorian calendar date.
 
   year @0 :Int16;
   # The year.  Must include the century.
   # Negative value indicates BC.
 
   month @1 :UInt8;   # Month number, 1-12.
   day @2 :UInt8;     # Day number, 1-30.
 }
 {% endhighlight %}
 
 Placing the comment _after_ the declaration rather than before makes the code more readable,
 especially when doc comments grow long. You almost always need to see the declaration before you
 can start reading the comment.
 
 ### Built-in Types
 
 The following types are automatically defined:
 
 * **Void:** `Void`
 * **Boolean:** `Bool`
 * **Integers:** `Int8`, `Int16`, `Int32`, `Int64`
 * **Unsigned integers:** `UInt8`, `UInt16`, `UInt32`, `UInt64`
 * **Floating-point:** `Float32`, `Float64`
 * **Blobs:** `Text`, `Data`
 * **Lists:** `List(T)`
 
 Notes:
 
 * The `Void` type has exactly one possible value, and thus can be encoded in zero bits. It is
   rarely used, but can be useful as a union member.
 * `Text` is always UTF-8 encoded and NUL-terminated.
 * `Data` is a completely arbitrary sequence of bytes.
 * `List` is a parameterized type, where the parameter is the element type. For example,
   `List(Int32)`, `List(Person)`, and `List(List(Text))` are all valid.
 
 ### Structs
 
 A struct has a set of named, typed fields, numbered consecutively starting from zero.
 
 {% highlight capnp %}
 struct Person {
   name @0 :Text;
   email @1 :Text;
 }
 {% endhighlight %}
 
 Fields can have default values:
 
 {% highlight capnp %}
 foo @0 :Int32 = 123;
 bar @1 :Text = "blah";
 baz @2 :List(Bool) = [ true, false, false, true ];
 qux @3 :Person = (name = "Bob", email = "bob@example.com");
 corge @4 :Void = void;
 grault @5 :Data = 0x"a1 40 33";
 {% endhighlight %}
 
 ### Unions
 
 A union is two or more fields of a struct which are stored in the same location. Only one of
 these fields can be set at a time, and a separate tag is maintained to track which one is
 currently set. Unlike in C, unions are not types, they are simply properties of fields, therefore
 union declarations do not look like types.
 
 {% highlight capnp %}
 struct Person {
   # ...
 
   employment :union {
     # We assume that a person is only one of these.
     unemployed @4 :Void;
     employer @5 :Company;
     school @6 :School;
     selfEmployed @7 :Void;
   }
 }
 {% endhighlight %}
 
 Additionally, unions can be unnamed.  Each struct can contain no more than one unnamed union.  Use
 unnamed unions in cases where you would struggle to think of an appropriate name for the union,
 because the union represents the main body of the struct.
 
 {% highlight capnp %}
 struct Shape {
   area @0 :Float64;
 
   union {
     circle @1 :Float64;      # radius
     square @2 :Float64;      # width
   }
 }
 {% endhighlight %}
 
 Notes:
 
 * Unions members are numbered in the same number space as fields of the containing struct.
   Remember that the purpose of the numbers is to indicate the evolution order of the
   struct. The system needs to know when the union fields were declared relative to the non-union
   fields.
 
 * Notice that we used the "useless" `Void` type here. We don't have any extra information to store
   for the `unemployed` or `selfEmployed` cases, but we still want the union to distinguish these
   states from others.
 
 * By default, when a struct is initialized, the lowest-numbered field in the union is "set".  If
   you do not want any field set by default, simply declare a field called "unset" and make it the
   lowest-numbered field.
 
 * You can move an existing field into a new union without breaking compatibility with existing
   data, as long as all of the other fields in the union are new.  Since the existing field is
   necessarily the lowest-numbered in the union, it will be the union's default field.
 
 **Wait, why aren't unions first-class types?**
 
 Requiring unions to be declared inside a struct, rather than living as free-standing types, has
 some important advantages:
 
 * If unions were first-class types, then union members would clearly have to be numbered separately
   from the containing type's fields.  This means that the compiler, when deciding how to position
   the union in its containing struct, would have to conservatively assume that any kind of new
   field might be added to the union in the future.  To support this, all unions would have to
   be allocated as separate objects embedded by pointer, wasting space.
 
 * A free-standing union would be a liability for protocol evolution, because no additional data
   can be attached to it later on.  Consider, for example, a type which represents a parser token.
   This type is naturally a union: it may be a keyword, identifier, numeric literal, quoted string,
   etc.  So the author defines it as a union, and the type is used widely.  Later on, the developer
   wants to attach information to the token indicating its line and column number in the source
   file.  Unfortunately, this is impossible without updating all users of the type, because the new
   information ought to apply to _all_ token instances, not just specific members of the union.  On
   the other hand, if unions must be embedded within structs, it is always possible to add new
   fields to the struct later on.
 
 * When evolving a protocol it is common to discover that some existing field really should have
   been enclosed in a union, because new fields being added are mutually exclusive with it.  With
   Cap'n Proto's unions, it is actually possible to "retroactively unionize" such a field without
   changing its layout.  This allows you to continue being able to read old data without wasting
   space when writing new data.  This is only possible when unions are declared within their
   containing struct.
 
 Cap'n Proto's unconventional approach to unions provides these advantages without any real down
 side:  where you would conventionally define a free-standing union type, in Cap'n Proto you
 may simply define a struct type that contains only that union (probably unnamed), and you have
 achieved the same effect.  Thus, aside from being slightly unintuitive, it is strictly superior.
 
 ### Groups
 
 A group is a set of fields that are encapsulated in their own scope.
 
 {% highlight capnp %}
 struct Person {
   # ...
 
   address :group {
     # Note:  This is a terrible way to use groups, and meant
     #        only to demonstrate the syntax.
     houseNumber @8 :UInt32;
     street @9 :Text;
     city @10 :Text;
     country @11 :Text;
   }
 }
 {% endhighlight %}
 
 Interface-wise, the above group behaves as if you had defined a nested struct called `Address` and
 then a field `address :Address`.  However, a group is _not_ a separate object from its containing
 struct: the fields are numbered in the same space as the containing struct's fields, and are laid
 out exactly the same as if they hadn't been grouped at all.  Essentially, a group is just a
 namespace.
 
 Groups on their own (as in the above example) are useless, almost as much so as the `Void` type.
 They become interesting when used together with unions.
 
 {% highlight capnp %}
 struct Shape {
   area @0 :Float64;
 
   union {
     circle :group {
       radius @1 :Float64;
     }
     rectangle :group {
       width @2 :Float64;
       height @3 :Float64;
     }
   }
 }
 {% endhighlight %}
 
 There are two main reason to use groups with unions:
 
 1. They are often more self-documenting.  Notice that `radius` is now a member of `circle`, so
    we don't need a comment to explain that the value of `circle` is its radius.
 2. You can add additional members later on, without breaking compatibility.  Notice how we upgraded
    `square` to `rectangle` above, adding a `height` field.  This definition is actually
    wire-compatible with the previous version of the `Shape` example from the "union" section
    (aside from the fact that `height` will always be zero when reading old data -- hey, it's not
    a perfect example).  In real-world use, it is common to realize after the fact that you need to
    add some information to a struct that only applies when one particular union field is set.
    Without the ability to upgrade to a group, you would have to define the new field separately,
    and have it waste space when not relevant.
 
 Note that a named union is actually exactly equivalent to a named group containing an unnamed
 union.
 
 **Wait, weren't groups considered a misfeature in Protobufs?  Why did you do this again?**
 
 They are useful in unions, which Protobufs did not have.  Meanwhile, you cannot have a "repeated
 group" in Cap'n Proto, which was the case that got into the most trouble with Protobufs.
 
 ### Dynamically-typed Fields
 
 A struct may have a field with type `AnyPointer`.  This field's value can be of any pointer type --
 i.e. any struct, interface, list, or blob.  This is essentially like a `void*` in C.
 
 See also [generics](#generic-types).
 
 ### Enums
 
 An enum is a type with a small finite set of symbolic values.
 
 {% highlight capnp %}
 enum Rfc3092Variable {
   foo @0;
   bar @1;
   baz @2;
   qux @3;
   # ...
 }
 {% endhighlight %}
 
 Like fields, enumerants must be numbered sequentially starting from zero. In languages where
 enums have numeric values, these numbers will be used, but in general Cap'n Proto enums should not
 be considered numeric.
 
 ### Interfaces
 
 An interface has a collection of methods, each of which takes some parameters and return some
 results.  Like struct fields, methods are numbered.  Interfaces support inheritance, including
 multiple inheritance.
 
 {% highlight capnp %}
 interface Node {
   isDirectory @0 () -> (result :Bool);
 }
 
 interface Directory extends(Node) {
   list @0 () -> (list :List(Entry));
   struct Entry {
     name @0 :Text;
     node @1 :Node;
   }
 
   create @1 (name :Text) -> (file :File);
   mkdir @2 (name :Text) -> (directory :Directory);
   open @3 (name :Text) -> (node :Node);
   delete @4 (name :Text);
   link @5 (name :Text, node :Node);
 }
 
 interface File extends(Node) {
   size @0 () -> (size :UInt64);
   read @1 (startAt :UInt64 = 0, amount :UInt64 = 0xffffffffffffffff)
        -> (data :Data);
   # Default params = read entire file.
 
   write @2 (startAt :UInt64, data :Data);
   truncate @3 (size :UInt64);
 }
 {% endhighlight %}
 
 Notice something interesting here: `Node`, `Directory`, and `File` are interfaces, but several
 methods take these types as parameters or return them as results.  `Directory.Entry` is a struct,
 but it contains a `Node`, which is an interface.  Structs (and primitive types) are passed over RPC
 by value, but interfaces are passed by reference. So when `Directory.list` is called remotely, the
 content of a `List(Entry)` (including the text of each `name`) is transmitted back, but for the
 `node` field, only a reference to some remote `Node` object is sent.
 
 When an address of an object is transmitted, the RPC system automatically manages making sure that
 the recipient gets permission to call the addressed object -- because if the recipient wasn't
 meant to have access, the sender shouldn't have sent the reference in the first place. This makes
 it very easy to develop secure protocols with Cap'n Proto -- you almost don't need to think about
 access control at all. This feature is what makes Cap'n Proto a "capability-based" RPC system -- a
 reference to an object inherently represents a "capability" to access it.
 
 ### Generic Types
 
 A struct or interface type may be parameterized, making it "generic". For example, this is useful
 for defining type-safe containers:
 
 {% highlight capnp %}
 struct Map(Key, Value) {
   entries @0 :List(Entry);
   struct Entry {
     key @0 :Key;
     value @1 :Value;
   }
 }
 
 struct People {
   byName @0 :Map(Text, Person);
   # Maps names to Person instances.
 }
 {% endhighlight %}
 
 Cap'n Proto generics work very similarly to Java generics or C++ templates. Some notes:
 
 * Only pointer types (structs, lists, blobs, and interfaces) can be used as generic parameters,
   much like in Java. This is a pragmatic limitation: allowing parameters to have non-pointer types
   would mean that different parameterizations of a struct could have completely different layouts,
   which would excessively complicate the Cap'n Proto implementation.
 
 * A type declaration nested inside a generic type may use the type parameters of the outer type,
   as you can see in the example above. This differs from Java, but matches C++. If you want to
   refer to a nested type from outside the outer type, you must specify the parameters on the outer
   type, not the inner. For example, `Map(Text, Person).Entry` is a valid type;
   `Map.Entry(Text, Person)` is NOT valid. (Of course, an inner type may declare additional generic
   parameters.)
 
 * If you refer to a generic type but omit its parameters (e.g. declare a field of type `Map` rather
   than `Map(T, U)`), it is as if you specified `AnyPointer` for each parameter. Note that such
   a type is wire-compatible with any specific parameterization, so long as you interpret the
   `AnyPointer`s as the correct type at runtime.
 
 * Relatedly, it is safe to cast a generic interface of a specific parameterization to a generic
   interface where all parameters are `AnyPointer` and vice versa, as long as the `AnyPointer`s are
   treated as the correct type at runtime. This means that e.g. you can implement a server in a
   generic way that is correct for all parameterizations but call it from clients using a specific
   parameterization.
 
 * The encoding of a generic type is exactly the same as the encoding of a type produced by
   substituting the type parameters manually. For example, `Map(Text, Person)` is encoded exactly
   the same as:
 
   <figure class="highlight"><pre><code class="language-capnp" data-lang="capnp"><span></span><span class="k">struct</span> <span class="n">PersonMap</span> {
     <span class="c1"># Encoded the same as Map(Text, Person).</span>
     <span class="n">entries</span> <span class="nd">@0</span> <span class="nc">:List(Entry)</span>;
     <span class="k">struct</span> <span class="n">Entry</span> {
       <span class="n">key</span> <span class="nd">@0</span> <span class="nc">:Text</span>;
       <span class="n">value</span> <span class="nd">@1</span> <span class="nc">:Person</span>;
     }
   }</code></pre></figure>
 
   {% comment %}
   Highlighter manually invoked because of: https://github.com/jekyll/jekyll/issues/588
   Original code was:
     struct PersonMap {
       # Encoded the same as Map(Text, Person).
       entries @0 :List(Entry);
       struct Entry {
         key @0 :Text;
         value @1 :Person;
       }
     }
   {% endcomment %}
 
   Therefore, it is possible to upgrade non-generic types to generic types while retaining
   backwards-compatibility.
 
 * Similarly, a generic interface's protocol is exactly the same as the interface obtained by
   manually substituting the generic parameters.
 
 ### Generic Methods
 
 Interface methods may also have "implicit" generic parameters that apply to a particular method
 call. This commonly applies to "factory" methods. For example:
 
 {% highlight capnp %}
 interface Assignable(T) {
   # A generic interface, with non-generic methods.
   get @0 () -> (value :T);
   set @1 (value :T) -> ();
 }
 
 interface AssignableFactory {
   newAssignable @0 [T] (initialValue :T)
       -> (assignable :Assignable(T));
   # A generic method.
 }
 {% endhighlight %}
 
 Here, the method `newAssignable()` is generic. The return type of the method depends on the input
 type.
 
 Ideally, calls to a generic method should not have to explicitly specify the method's type
 parameters, because they should be inferred from the types of the method's regular parameters.
 However, this may not always be possible; it depends on the programming language and API details.
 
 Note that if a method's generic parameter is used only in its returns, not its parameters, then
 this implies that the returned value is appropriate for any parameterization. For example:
 
 {% highlight capnp %}
 newUnsetAssignable @1 [T] () -> (assignable :Assignable(T));
 # Create a new assignable. `get()` on the returned object will
 # throw an exception until `set()` has been called at least once.
 {% endhighlight %}
 
 Because of the way this method is designed, the returned `Assignable` is initially valid for any
 `T`. Effectively, it doesn't take on a type until the first time `set()` is called, and then `T`
 retroactively becomes the type of value passed to `set()`.
 
 In contrast, if it's the case that the returned type is unknown, then you should NOT declare it
 as generic. Instead, use `AnyPointer`, or omit a type's parameters (since they default to
 `AnyPointer`). For example:
 
 {% highlight capnp %}
 getNamedAssignable @2 (name :Text) -> (assignable :Assignable);
 # Get the `Assignable` with the given name. It is the
 # responsibility of the caller to keep track of the type of each
 # named `Assignable` and cast the returned object appropriately.
 {% endhighlight %}
 
 Here, we omitted the parameters to `Assignable` in the return type, because the returned object
 has a specific type parameterization but it is not locally knowable.
 
 ### Constants
 
 You can define constants in Cap'n Proto.  These don't affect what is sent on the wire, but they
 will be included in the generated code, and can be [evaluated using the `capnp`
 tool](capnp-tool.html#evaluating-constants).
 
 {% highlight capnp %}
 const pi :Float32 = 3.14159;
 const bob :Person = (name = "Bob", email = "bob@example.com");
 const secret :Data = 0x"9f98739c2b53835e 6720a00907abd42f";
 {% endhighlight %}
 
 Additionally, you may refer to a constant inside another value (e.g. another constant, or a default
 value of a field).
 
 {% highlight capnp %}
 const foo :Int32 = 123;
 const bar :Text = "Hello";
 const baz :SomeStruct = (id = .foo, message = .bar);
 {% endhighlight %}
 
 Note that when substituting a constant into another value, the constant's name must be qualified
 with its scope.  E.g. if a constant `qux` is declared nested in a type `Corge`, it would need to
 be referenced as `Corge.qux` rather than just `qux`, even when used within the `Corge` scope.
 Constants declared at the top-level scope are prefixed just with `.`.  This rule helps to make it
 clear that the name refers to a user-defined constant, rather than a literal value (like `true` or
 `inf`) or an enum value.
 
 ### Nesting, Scope, and Aliases
 
 You can nest constant, alias, and type definitions inside structs and interfaces (but not enums).
 This has no effect on any definition involved except to define the scope of its name. So in Java
 terms, inner classes are always "static". To name a nested type from another scope, separate the
 path with `.`s.
 
 {% highlight capnp %}
 struct Foo {
   struct Bar {
     #...
   }
   bar @0 :Bar;
 }
 
 struct Baz {
   bar @0 :Foo.Bar;
 }
 {% endhighlight %}
 
 If typing long scopes becomes cumbersome, you can use `using` to declare an alias.
 
 {% highlight capnp %}
 struct Qux {
   using Foo.Bar;
   bar @0 :Bar;
 }
 
 struct Corge {
   using T = Foo.Bar;
   bar @0 :T;
 }
 {% endhighlight %}
 
 ### Imports
 
 An `import` expression names the scope of some other file:
 
 {% highlight capnp %}
 struct Foo {
   baz @0 :import "bar.capnp".Baz;
   # Use type "Baz" defined in bar.capnp.
 }
 {% endhighlight %}
 
 Of course, typically it's more readable to define an alias:
 
 {% highlight capnp %}
 using Bar = import "bar.capnp";
 
 struct Foo {
   baz @0 :Bar.Baz;
   # Use type "Baz" defined in bar.capnp.
 }
 {% endhighlight %}
 
 Or even:
 
 {% highlight capnp %}
 using import "bar.capnp".Baz;
 
 struct Foo {
   baz @0 :Baz;
   # Use type "Baz" defined in bar.capnp.
 }
 {% endhighlight %}
 
 The above imports specify relative paths.  If the path begins with a `/`, it is absolute -- in
 this case, the `capnp` tool searches for the file in each of the search path directories specified
 with `-I`.
 
 ### Annotations
 
 Sometimes you want to attach extra information to parts of your protocol that isn't part of the
 Cap'n Proto language.  This information might control details of a particular code generator, or
 you might even read it at run time to assist in some kind of dynamic message processing.  For
 example, you might create a field annotation which means "hide from the public", and when you send
 a message to an external user, you might invoke some code first that iterates over your message and
 removes all of these hidden fields.
 
 You may declare annotations and use them like so:
 
 {% highlight capnp %}
 annotation foo(struct, enum) :Text;
 # Declare an annotation 'foo' which applies to struct and enum types.
 
 struct MyType $foo("bar") {
   # Apply 'foo' to to MyType.
 
   # ...
 }
 {% endhighlight %}
 
 The possible targets for an annotation are: `file`, `struct`, `field`, `union`, `group`, `enum`,
 `enumerant`, `interface`, `method`, `parameter`, `annotation`, `const`.
 You may also specify `*` to cover them all.
 
 {% highlight capnp %}
 annotation baz(*) :Int32;
 # 'baz' can annotate anything!
 
 $baz(1);  # Annotate the file.
 
 struct MyStruct $baz(2) {
   myField @0 :Text = "default" $baz(3);
   myUnion :union $baz(4) {
     # ...
   }
 }
 
 enum MyEnum $baz(5) {
   myEnumerant @0 $baz(6);
 }
 
 interface MyInterface $baz(7) {
   myMethod @0 (myParam :Text $baz(9)) -> () $baz(8);
 }
 
 annotation myAnnotation(struct) :Int32 $baz(10);
 const myConst :Int32 = 123 $baz(11);
 {% endhighlight %}
 
 `Void` annotations can omit the value.  Struct-typed annotations are also allowed.  Tip:  If
 you want an annotation to have a default value, declare it as a struct with a single field with
 a default value.
 
 {% highlight capnp %}
 annotation qux(struct, field) :Void;
 
 struct MyStruct $qux {
   string @0 :Text $qux;
   number @1 :Int32 $qux;
 }
 
 annotation corge(file) :MyStruct;
 
 $corge(string = "hello", number = 123);
 
 struct Grault {
   value @0 :Int32 = 123;
 }
 
 annotation grault(file) :Grault;
 
 $grault();  # value defaults to 123
 $grault(value = 456);
 {% endhighlight %}
 
 ### Unique IDs
 
 A Cap'n Proto file must have a unique 64-bit ID, and each type and annotation defined therein may
 also have an ID.  Use `capnp id` to generate a new ID randomly.  ID specifications begin with `@`:
 
 {% highlight capnp %}
 @0xdbb9ad1f14bf0b36;
 # file ID
 
 struct Foo @0x8db435604d0d3723 {
   # ...
 }
 
 enum Bar @0xb400f69b5334aab3 {
   # ...
 }
 
 interface Baz @0xf7141baba3c12691 {
   # ...
 }
 
 annotation qux @0xf8a1bedf44c89f00 (field) :Text;
 {% endhighlight %}
 
 If you omit the ID for a type or annotation, one will be assigned automatically.  This default
 ID is derived by taking the first 8 bytes of the MD5 hash of the parent scope's ID concatenated
 with the declaration's name (where the "parent scope" is the file for top-level declarations, or
 the outer type for nested declarations).  You can see the automatically-generated IDs by "compiling"
 your file with the `-ocapnp` flag, which echos the schema back to the terminal annotated with
 extra information, e.g. `capnp compile -ocapnp myschema.capnp`.  In general, you would only specify
 an explicit ID for a declaration if that declaration has been renamed or moved and you want the ID
 to stay the same for backwards-compatibility.
 
 IDs exist to provide a relatively short yet unambiguous way to refer to a type or annotation from
 another context.  They may be used for representing schemas, for tagging dynamically-typed fields,
 etc.  Most languages prefer instead to define a symbolic global namespace e.g. full of "packages",
 but this would have some important disadvantages in the context of Cap'n Proto:
 
 * Programmers often feel the need to change symbolic names and organization in order to make their
   code cleaner, but the renamed code should still work with existing encoded data.
 * It's easy for symbolic names to collide, and these collisions could be hard to detect in a large
   distributed system with many different binaries using different versions of protocols.
 * Fully-qualified type names may be large and waste space when transmitted on the wire.
 
 Note that IDs are 64-bit (actually, 63-bit, as the first bit is always 1).  Random collisions
 are possible, but unlikely -- there would have to be on the order of a billion types before this
 becomes a real concern.  Collisions from misuse (e.g. copying an example without changing the ID)
 are much more likely.
 
 ## Evolving Your Protocol
 
 A protocol can be changed in the following ways without breaking backwards-compatibility, and
 without changing the [canonical](encoding.html#canonicalization) encoding of a message:
 
 * New types, constants, and aliases can be added anywhere, since they obviously don't affect the
   encoding of any existing type.
 
 * New fields, enumerants, and methods may be added to structs, enums, and interfaces, respectively,
   as long as each new member's number is larger than all previous members.  Similarly, new fields
   may be added to existing groups and unions.
 
 * New parameters may be added to a method.  The new parameters must be added to the end of the
   parameter list and must have default values.
 
 * Members can be re-arranged in the source code, so long as their numbers stay the same.
 
 * Any symbolic name can be changed, as long as the type ID / ordinal numbers stay the same.  Note
   that type declarations have an implicit ID generated based on their name and parent's ID, but
   you can use `capnp compile -ocapnp myschema.capnp` to find out what that number is, and then
   declare it explicitly after your rename.
 
 * Type definitions can be moved to different scopes, as long as the type ID is declared
   explicitly.
 
 * A field can be moved into a group or a union, as long as the group/union and all other fields
   within it are new.  In other words, a field can be replaced with a group or union containing an
   equivalent field and some new fields.
 
 * A non-generic type can be made [generic](#generic-types), and new generic parameters may be
   added to an existing generic type. Other types used inside the body of the newly-generic type can
   be replaced with the new generic parameter so long as all existing users of the type are updated
   to bind that generic parameter to the type it replaced. For example:
 
   <figure class="highlight"><pre><code class="language-capnp" data-lang="capnp"><span></span><span class="k">struct</span> <span class="n">Map</span> {
     <span class="n">entries</span> <span class="nd">@0</span> <span class="nc">:List(Entry)</span>;
     <span class="k">struct</span> <span class="n">Entry</span> {
       <span class="n">key</span> <span class="nd">@0</span> <span class="nc">:Text</span>;
       <span class="n">value</span> <span class="nd">@1</span> <span class="nc">:Text</span>;
     }
   }</code></pre></figure>
 
   {% comment %}
   Highlighter manually invoked because of: https://github.com/jekyll/jekyll/issues/588
   Original code was:
     struct Map {
       entries @0 :List(Entry);
       struct Entry {
         key @0 :Text;
         value @1 :Text;
       }
     }
   {% endcomment %}
 
   Can change to:
 
   <figure class="highlight"><pre><code class="language-capnp" data-lang="capnp"><span></span><span class="k">struct</span> <span class="n">Map</span>(<span class="n">Key</span>, <span class="n">Value</span>) {
     <span class="n">entries</span> <span class="nd">@0</span> <span class="nc">:List(Entry)</span>;
     <span class="k">struct</span> <span class="n">Entry</span> {
       <span class="n">key</span> <span class="nd">@0</span> <span class="nc">:Key</span>;
       <span class="n">value</span> <span class="nd">@1</span> <span class="nc">:Value</span>;
     }
   }</code></pre></figure>
 
   {% comment %}
   Highlighter manually invoked because of: https://github.com/jekyll/jekyll/issues/588
   Original code was:
     struct Map(Key, Value) {
       entries @0 :List(Entry);
       struct Entry {
         key @0 :Key;
         value @1 :Value;
       }
     }
   {% endcomment %}
 
   As long as all existing uses of `Map` are replaced with `Map(Text, Text)` (and any uses of
   `Map.Entry` are replaced with `Map(Text, Text).Entry`).
 
   (This rule applies analogously to generic methods.)
 
 The following changes are backwards-compatible but may change the canonical encoding of a message.
 Apps that rely on canonicalization (such as some cryptographic protocols) should avoid changes in
 this list, but most apps can safely use them:
 
 * A field of type `List(T)`, where `T` is a primitive type, blob, or list, may be changed to type
   `List(U)`, where `U` is a struct type whose `@0` field is of type `T`.  This rule is useful when
   you realize too late that you need to attach some extra data to each element of your list.
   Without this rule, you would be stuck defining parallel lists, which are ugly and error-prone.
   As a special exception to this rule, `List(Bool)` may **not** be upgraded to a list of structs,
   because implementing this for bit lists has proven unreasonably expensive.
 
 Any change not listed above should be assumed NOT to be safe.  In particular:
 
 * You cannot change a field, method, or enumerant's number.
 * You cannot change a field or method parameter's type or default value.
 * You cannot change a type's ID.
 * You cannot change the name of a type that doesn't have an explicit ID, as the implicit ID is
   generated based in part on the type name.
 * You cannot move a type to a different scope or file unless it has an explicit ID, as the implicit
   ID is based in part on the scope's ID.
 * You cannot move an existing field into or out of an existing union, nor can you form a new union
   containing more than one existing field.
 
 Also, these rules only apply to the Cap'n Proto native encoding.  It is sometimes useful to
 transcode Cap'n Proto types to other formats, like JSON, which may have different rules (e.g.,
 field names cannot change in JSON).