Overview: Schemas and Extensions

To define the structure the main input ought to have, Nyarna uses a schema. Schemas define the root type of the documents they are used for, and additionally may define backends with which those documents can be processed.

To create a schema, you first create a \SchemaDef. This can contain definitions of types, functions, and backends, but doesn't make the symbols visible. A \SchemaDef can be implicitly or explicitly instantiated, which will yield a \Schema. The instantiation will establish all symbols defined in the schema. A \SchemaDef remembers its instance, and will always return that instance on subsequent instantiations.

You can define schemas in their own standalone modules, by using the predefined schema named Schema. This is best practice, as it allows the Nyarna processor to provide an internal backend in the predefined schema that simply validates your schema – semantic errors would otherwise only be visible when the first instantiation happens. Other backends could be provided for the predefined schema, e.g. for autogenerating code that loads Nyarna input into native types.

If your schema defines backends, you declare a variable with which you can access the currently processed document. A backend can define variables, functions, and a body which gets evaluated when the backend is used. The body is to emit a concatenation of \Output, each of which creates a document.

Backends do not need to follow a certain style, but it is best practice to implement them via matchers. This enables us later to extend the schema.

\standalone(schema=\import(Schema))

\SchemaDef(root=\Concat(\Animal)):
  Cat = \Record:
    name : \Text
    lives: \Natural
  \end(Record)

  Dog = \Record:
    name : \Text
    breed: \Text
  \end(Record)

  Animal = \Intersection(\Cat, \Dog)
:backends:|\doc|
  info = \backend:
  :funcs:
    writeInfo = \matcher:
    :(\Cat):|\c|
      The cat \c::name has \c::lives lives.\
    :(\Dog):|\d|
      The dog \d::name is of the breed \d::breed.\
    \end(matcher)
  :body:
    \Output(name=info.txt):
      \map(\doc::root, \writeInfo)
    \end(Output)
  \end(backend)
\end(SchemaDef)

Extending a schema works by creating a \SchemaExt and later using it as additional input when instantiating the \SchemaDef. A \SchemaExt can define additional types and backends, but can also extend certain entities. Currently, this works for intersections, backends, and matchers.

By extending an intersection, an extension can allow additional content in the document. By extending a matcher, an extension can make it process values of types that could perviously not be processed. By extending a backend, an extension can extend the matchers defined within the backend, thus allowing an extended intersection to be properly processed by a backend.

A backend must be set up appropriately for it to be extended. This is why it's best practice to implement backends with matchers. If an extension extends an intersection, but does not extend a backend that processes values of this intersection, the backend is rendered unusable.

A \SchemaDef remembers the extensions it has been instantiated with. A subsequent instantiation allows fewer extensions to be given, but any extension given that has not been seen in the initial instantiation will result in an error. Extensions are designed so that any document that is valid for a schema is also valid for any schema that is the base schema with some extensions. With these rules, you can use a schema in a module, and import that module somewhere else where you use a richer version of the schema.

This concludes the overview. You may take the tour for a more detailled discussion of language features and to experiment with the language.

\fragment(\SchemaExt)

\SchemaExt:
  Rat = \Record:
    name  : \Text
    sniffs: \Natural
  \end(Record)

  Animals |= \Intersection(\Rat)
:backends:|\doc|
  info = \backend:
  :funcs:
    writeInfo |= \matcher:
    :(\Rat):|\r|
      The rat \r::name has sniffed \r::sniffs times.
    \end(matcher)
  \end(backend)
\end(SchemaExt)