XForms 2.0: What's New?

The author Steven Pemberton, CWI, Amsterdam

1957: The first municipal computer (Norwich, UK)

The First Computer in Norwich

Just one of 21 cabinets making up the computer.

2015: The Raspberry Pi Zero

Raspberry Pi in Norwich

The first computer so cheap that they gave it away on the cover of a magazine

How do they compare?

The Elliot ran for about a decade, 24 hours a day.

How long do you think it would take the Raspberry Pi Zero to do the same amount of computing?

How do they compare?

The Elliot ran for about a decade, 24 hours a day.

How long do you think it would take the Raspberry Pi Zero to do the same amount of computing?

The Raspberry Pi is about one million times faster...

Compare

The Raspberry Pi is not only one million times faster. It is also one millionth the price.

A factor of a million million.

A terabyte is a million million bytes: nowadays we talk in terms of very large numbers.

Want to guess how long a million million seconds is?

A million million seconds

Is 30,000 years...

In other words, a really big number...

Moore's Law

In fact a million million times improvement is about what you would expect from Moore's Law over 58 years.

Except: the Raspberry Pi is two million times smaller as well, so it is much better than even that.

One reason for that is that originally computers were priced in an interesting way.

Meet my grandfather

AC Wheeler as a child

Born in 1880, a middle child in a family of 20(!) children.

1880: nearly no modern technologies.

In such a large household, as is to be expected, each child had a task, and it was my grandfather's to ensure that the oil lamps were filled.

It must have been indeed an exciting time, when light became something you could switch on and off.

This may well explain my grandfather's fascination with electricity, and why he set up a company to manufacture electrical switching machinery.

The Price of Electricity

It is interesting to note that when Edison first introduced electrical lighting, he priced the electricity not based on the cost of manufacture, but on the cost of the same amount of light provided by traditional means. THe light cost the same, but with the advantage of ease of use, and comparative safety.

(And indeed there are many reports of the deaths of well-to-do women because of candles.)

The Price of the Original Computers

Similarly, the first computers were priced based on the cost of the people they were replacing.

Let's go back to 1957

In the 50's, computers were so expensive that nearly no one bought them, nearly everyone leased them.

To rent time on a computer then would cost you of the order of $1000 per hour: several times the annual salary of a programmer!

When you leased a computer in those days, you would get programmers for free to go with it.

Compared to the cost of a computer, a programmer was almost free.

Programming in the 50's

TypingIn the 50's the computer's time was expensive. A programmer would:

Why? Because it was much cheaper to let 3 people check it, than to let the computer discover the errors.

The design of programming languages

The first programming languages were designed in the 50s.

They were designed with the economic relationship of computer and programmer in mind.

It was much cheaper to let the programmer spend lots of time producing a program than to let the computer do some of the work for you.

Programming languages were designed so that you tell the computer exactly what to do, in its terms, not what you want to achieve in yours.

Why even bother?

You might wonder why programming languages were even considered, since they did nothing to improve computer time.

And indeed, there were people right into the 70's who thought that programming languages were a waste of computer time.

The answer seems to be that programs were then portable between different computers (which were all different), at the marginal cost of having to compile them once before running them many times.

Back to now

It happened slowly, almost unnoticed, but nowadays we have the exact opposite position:

Compared to the cost of a programmer, a computer is almost free.

I call this Moore's Switch.

Moore's Switch

Moore's Switch illustrated
Relative costs of computers and programmers, 1957-2017

But, we are still programming in programming languages that are direct descendants of the languages designed in the 1950's!

We are still telling the computers what to do.

Declarative programming

A new way of programming: declarative programming.

This describes what you want to achieve, but not how to achieve it.

XForms

XForms is a declarative system that lets you define applications. BBC Sport App

It is a W3C standard, and in worldwide use.

Example: 150 person years becomes 10!

A certain company makes one-off BIG machines (walk in): user interface is very demanding — traditionally needed 5 years, 30 people.

With XForms this became: 1 year, 10 people.

Do the sums. Assume one person costs 100k a year. Then this has gone from a 15M cost to a 1M cost. They have saved 14 million! (And 4 years)

Example: Insurance Industry

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application.

Example: Insurance Industry

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application.

[Two days later]

Programmer: I'll need 30 days to work out how long it will take to program it.

Example: Insurance Industry

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application.

[Two days later]

Programmer: I'll need 30 days to work out how long it will take to program it.

XFormser: I've already programmed it!

Example: NHS

The British National Health Service started a project for a health records system.

Example: NHS

The British National Health Service started a project for a health records system.

One person then created a system using XForms.

A Review

"Please don't assume that the audience already knows XForms"

XForms

XForms 1.0 was designed for online Forms.

After some experience it was realised we had followed HTML too slavishly, and with some slight generalisation, it could be more useful.

So was born XForms 1.1, a Turing-complete declarative programming language.

Implementations from Belgium, France, Germany, NL, UK, USA.

XForms 2.0 is in preparation.

And I am here today to tell you about it.

Form and content

"The term form refers to the work's style, techniques and media used, and how the elements of design are implemented.

Content, on the other hand, refers to a work's essence, or what is being depicted."

Wikipedia

This is a nearly-perfect description of XForms

XForms, an overview

XForms is all about state. (Which means it meshes well with REST - Representational State Transfer).

The data can be internal or come from external sources.

You describe the data: properties and relationships.

You can display any selection of values in the content.

Initially the system is in a state of stasis.

When a value changes, by whatever means, the system updates related values to bring it back to stasis.

This is like spreadsheets, but much more general.

The result is: programming is much easier, since the system does all the administrative work for you.

Example

I've got a position in the world as x and y coordinates, and I want to display the map tile of that location at a certain zoom.

My data:

x, y, zoom

Openstreetmap has a REST interface for getting such a thing:

http://openstreetmap.org/<zoom>/<x>/<y>.png

Map interface

The coordinate system changes at each level of zoom.

As you zoom out, there are in each axis half as many tiles, so there are ¼ as many tiles. And the interface indexes tiles, not locations.

So to get a tile:

  1. You have to know how big a tile is
  2. you have to calculate the correct index using this plus the zoom.

Declarative Example

The data: x, y, zoom

scale = 226 - zoom
tilex = floor(x/scale)
tiley = floor(y/scale)
url = concat("http://tile.openstreetmap.org/", zoom, "/", tilex, "/", tiley, ".png")

That is really all that is needed (modulo syntax, which looks like this:)

<bind ref="tilex" calculate="floor(../x div ../scale)"/>

That's the form. Now the content:

<input ref="x" label="x"/>
<input ref="y" label="y"/>
<input ref="zoom" label="zoom"/>
<output ref="url" mediatype="image/*"/>

and the tile will be updated each time any of the values change.

Live tile with working zoom

Source

Map

Source

Map

Source

XForms, an overview

Separation of data from UI, similar to separation of style from content with CSS, with similar advantages.

Instances of data + properties.

<instance src="sale.xml"/>
<bind ref="something" property="something"/>

Properties include:

Whenever a value changes, the related values are automatically updated, in spread-sheet style.

Example properties: relevant and required

<bind ref="address/state"
      required="../country = 'USA'"
      label="State"/>

This means that state is only required for the USA. The field will always be visible.

<bind ref="address/state"
      relevant="../country = 'USA'"
      required="true()"
      label="State"/>

This means that state will only be visible for the USA, but once visible will be required.

Controls in the UI then bind to data nodes, inheriting their properties.

<input ref="address/state"/>

Relevance

Here is an example (try it). If you select USA as country, the control for state appears.

Source

Relevance

Similarly, the billing address is only relevant if it is different from the delivery address:

<bind ref="address[@type='Billing']"
      relevant="../@different=true()"/>

Relevance

Source

Events and Actions

Typically XForms works automatically.

You can hook into the processing model to respond in special ways.

Events announce changes in the state;
actions
effect changes to the state.

E.g. xforms-ready announces that the system has initialised (and is at stasis). You could respond to this by recording today's date and time:

<action ev:event="xforms-ready">
   <setvalue ref="today" value="now()"/>
</action>

Events and Actions

Other events announce when a value changes, or when it changes validity, relevance, etc.

Other useful actions include activating a button, and inserting and deleting elements and attributes from data.

In fact the only way to get a vanilla button to do anything is to listen for the activation events, and then respond with an action.

<trigger label="Restart">
   <action ev:event="DOMActivate">
      <setvalue ref="score" value="0"/>
   </action>
</trigger>

What's New

Data

Expressions

Controls

Actions and Events

Submission

Document Structure

Data

XForms now also accepts JSON as external data.

{"father": {"given": "Mark", "family": "Smith"}, 
"mother": {"given": "Mary", "family": "Smith"}}

As seen by XForms:

<json type="object">
   <father type="object">
      <given>Mark</given>
      <family>Smith</family>
   </father>
   <mother type="object">
      <given>Mary</given>
      <family>Smith</family>
   </mother>
</json>

A value would be selected with an XPath expression such as mother/given.

Data

An array such as

{"load": [0.31, 0.33, 0.32]} 

is transformed to

<json type="object">
  <load type="array">
    <_ type="number">0.31</_>
    <_ type="number">0.33</_>
    <_ type="number">0.32</_>
  </load>
</json>

and accessed with an expression such as load/_[1].

JSON roundtrips.

Data properties

Originally a property for a data node had to be defined in one go in a single bind element. Now it may be extended over several binds, with rules for how they are combined.

The new whitespace property defines how whitespace is treated in values when a user inputs a value: trimmed, compressed or removed entirely.

For instance, if the user inputs spaces when entering a credit card number, they are automatically and silently removed.

Types

There are a number of new types:

The definition of validity has been slightly improved by allowing non-required values to be considered valid if empty. This simplifies the definition of controls for optional values.

Expressions

XForms 2.0 uses XPath 2, instead of XPath 1; this allows amongst other things atomic values and typed values.

However, the expression language is defined as a separate module allowing future versions of XForms easily to switch to newer versions of XPath.

AVTs: Earlier versions of XForms only allowed data values to be injected into element content of controls (using the output control); calculated values can now also be injected into attributes, enabling the deprecation of many child elements.

For instance, where you previously had to say:

<output ref="maptile">
   <mediatype value="site/type"/>
</output>

you can now write:

<output ref="maptile" mediatype="{site/type}"/>

AVTs also create many new presentation options; for instance class values can now be calculated dynamically:

<output ref="balance" class="{if(. &lt; 0, 'negative', 'positive')}"/>

Custom functions and variables have been added.

New functions

bind()

Source

Controls

Earlier versions of XForms distinguished whether a control expected a single value, such as

<input ref="height"> 

or a sequence of values, such as

<repeat nodeset="person">

However, it was observed that users did not benefit from this distinction, so the nodeset attribute has now been deprecated in favour of ref everywhere.

If a control expects a single value, and gets a sequence, it selects the first in the sequence (which is what also happened in earlier versions).

Labels etc

Most controls have label, hint, help, alert as sub-elements. For example:

<input ref="age">
   <label>Age</label>
   <alert>Must be a non-negative whole number</alert>
</input>

They may now alternatively be defined as attributes:

<input ref="age" label="Age" alert="Must be a non-negative whole number"/>

but also they may be attached to the node itself with a bind:

<bind ref="age" type="nonNegativeInteger" 
      label="Age" alert="Must be a non-negative whole number"/>
 ...
<input ref="age"/>

This has, amongst other things, the advantage of allowing the condition and matching message to be kept close to each other:

<bind ref="age" label="Age">
   <bind type="integer" alert="Must be a whole number"/>
   <bind constraint=". &gt;= 0 and . &lt;= 120" alert="Must be between 0 and 120"/>
</bind>

The label elements and attributes are consequently now optional on controls.

Actions and Events

All actions now have an iterate attribute to allow them to iterate over a range of values:

<setvalue iterate="item[@selected=true()]" ref="." value=". + 1"/>

Recalculation

The XForms recalculation algorithm is described in four steps, and to match this, in previous versions of XForms there were four actions: rebuild, recalculate, revalidate, and refresh ; however, it turns out that all most users want to ensure is that everything is up-to-date A new action <update/> subsumes the previous actions: . The update action worries about the details.

It is now possible to call script (such as Javascript) from actions; this includes passing parameters.

In all versions of XForms the state of all instances is saved at start-up to enable the reset action. Now it is possible to define at which point the instances are saved with the retain action, and also to reset one or more instances rather than all of them.

It is now possible to add properties to dispatched events. Event handlers are no longer disabled when their associated controls are disabled.

Submission

It is now possible to override the mediatype of an incoming document. This is useful for cases where the server uses the wrong, or a too-broad, mediatype, and for cases where the mediatype doesn't expose that the document is an application of XML.

The default submission method is get (previously there was no default).

A new submission attribute nonrelevant replaces the existing relevant, adding new options (relevant was a boolean, but there are now more than two options for relevance processing).

Document structure

XForms has always been designed to be integrated in another markup language, whether XHTML, SVG, ODF, or whatever. To that end, XForms has not needed a root element. However, for those occasions that an implementation needs a free-standing usage of XForms, there is now an optional root element <form>.

Progress

Converging

More or less complete rewrite of spec to make it more readable (10% smaller result).

Test-suite started.

Implementations adding the new features.

Test suite

Test suite is written in XForms.

See Markup UK paper.

Source

Conclusion

Updating an existing language has the extra problem that you have to deal with legacy: you want to make changes without invalidating existing documents.

This sometimes blocks the most obvious change, and demands creativity to find a suitable expression of the required functionality.

XForms 2.0 is a surprisingly expressive language. Declarative programming requires a different mindset, but once acquired allows the definition of applications in an unexpectedly small number of lines of code.

XForms course at XML Summer School in Oxford this September.