Using XForms for interfaces to XML data

Steven Pemberton, CWI, Amsterdam

XForms

Some history

Some background

What we would have done differently, if only we'd known

What is still to be done

May 1998

W3C Workshop "Shaping the Future of HTML"

This workshop decided there should be a new WG to create a modularised XML version of HTML, with new features for forms.

"T.V. Raman: stronger typing of fields and the ability to specify at least simple constraints using declarative means. now it's being done with javascript and there's no point to using javascript just to verify that the content of a field is a number".

HTML WG

Dec 1999: HTML 4.01 ready, and XHTML 1.0.

Works starts on XHTML 1.1, a cleaned up version, and future versions.

There is a very large contingent of people only interested in Forms: Forms WG is spun off.

Analysis

Forms had been added to HTML in 1993. Very simple, mostly presentation-oriented, but effective, and widely used.

Searching

Buying

Logging in

Configuring hardware

Reading mail

Composing email

Etc etc

• Tracking packages
• calculating currencies
• submitting taxes
• banking
• expenses
• calendars
• blogging
• wikis
• ...

Problems with HTML Forms

• Presentation oriented, mixing data and presentation
• No types, Ping-ponging to the server
• Reliance on scripting
• Problems with non-Western characters
• Accessibility problems
• Hard to make cross-device for single authoring
• Impoverished data-model, no integration with existing streams
• Hard to manage, hard to see what is returned
• No support for wizards and shopping carts etc.

The WG

Consisted of many Forms companies; people representing accessibility; usability; abstraction, etc.

We all had our own requirements, and we vaguely knew it had to be with XML, but not how.

Requirements

Obvious things that were needed included:

• restrictions on the values you could enter;
• more aid in entering values (for instance dates);
• constraints on values based on other values (e.g. outward journey is earlier than return journey);
• automatic calculation of some values;
• structured data.

A Timeline, XForms 1.0

1999

2000

2001

2002

2003 *

=Requirements document

=Working draft

=Last call

=Candidate Recommendation

=Proposed Recommendation

=Recommendation

=The month we got Opera and Apple's comments

* = implementors' workshop

XForms 1.0

At the time, the most implemented W3C spec ever (25 known implementations).

In February 2003 we had an implementors' workshop, where a sizeable number of implementors turned up.

We had client-side implementations, server-side, and a mix.

We had native, and plug-ins, desktop and mobile.

XMLDOMCSSJavascriptXHTMLXPathXForms

    =Regular XHTML Browser

Time to build all of above: 3 programmers, 4 months

Total footprint (on IPAQ implementation): 400K (above Java VM)

The Oracle implementation was particularly impressive.

Some of the things we had explicitely designed for

Device independence

Freedom of implementation choices (server/client, and mix)

Don't let implementation drive design decisions.

We also knew there were other things we wanted, but you have to ship it at some point.

The strengths of the design

Structured data

Separation of data and presentation

Integrated support of external data

Much is declarative, and there are hooks to allow actions when it isn't.

Forms are tractable.

Support of use cases missing from HTML in a generalised manner

A large degree of generality

The major weakness (in hindsight)

Too slavish following of HTML features, in particular using fixed strings where HTML used them.

Industry briefing 2004

Adobe forms

Microsoft infopath

Low hanging fruit

Several implementations had added extensions, not always compatibly.

We wanted to fix that as soon as possible.

We saw a number of low-hanging fruits we could pluck.

We also saw the chance, after some experience with the language, to generalise further.

Applications

XForms 1.1 was more general than XForms 1.0, and fixed a number of the worst cases.

XForms 1.1 had input, output and a (Turing-complete) processor.

All of a sudden, people were producing applications with XForms, things that didn't look like forms at all.

XForms 2.0

XForms 2.0 is now in preparation, and very close to last call.

One change is the use of XPath 2.0.

But maybe the most effective change is the addition of attibute value templates

<output ref="total" class="{presentation}"/>

This in fact removes the need for some of the things added to XForms 1.1.

Usability

Some changes are for usability of authoring. They add no new functionality, but make life easier.

For instance XForms used to allow only one bind per node. Now you may have several binds, and they are combined in an obvious way.

<bind ref="i" type="integer"/>
<bind ref="i" constraint=". &gt; 0"/>
<bind ref="i" constraint=". &lt; 100"/>

This type of change helps both human authors as well as the automatic construction of forms, by the way.

Usability

Similarly, XForms used to distinguish between bindings to single nodes

<output ref="total"/>

and to multiple nodes

<repeat nodeset="todo">...

but it turned out that users weren't helped with this distinction. They thought in terms of the particular binding and not the general case:

<bind nodeset="total" .../>

might bind to all elements called total, but there is only one, and so they would 'incorrectly' use "ref" instead of "nodeset".

Open-source implementations

There are a number of open-source implementations:

XSLTForms is an open-source client-side implementation based on XSLT 1.0 to compile XForms to vanilla (X)HTML and Javascript.

Orbeon Forms, a server-side implementation

The betterFORM project, server-side.

Xfolite is a light-weight XForms client for the J2ME platform. It was originally created at Nokia Research Center.

Enterprise implementations

Many companies use XForms either internally or externally, and have their own implementations, either for internal use or for licensing. For instance:

• Xerox
• IBM
• OpenOffice
• Jadu
• EMC
• Inventive Designers
• Cordys
• Seneca

Users

Examples include.

• The entire British Insurance industry,
• The US Navy (in submarines),
• NASA (Jet Propulsion Laboratories),
• UK Government (Planning Inspectorate), many UK local government sites
• Verifone - a payment company, for configuring petrol pumps,
• Xerox, for an Enterprise Content Management system, to implement dynamic forms and to do XML binding to editing and adding for Wikis, Blogs, and content management, and another use in active development.
• Yahoo for several internal applications, and now part of their mobile platform
• NACS - the National Association of Consumer Stores for configuring and accessing data from a range of devices,
• Vancity - A Canadian Credit Union, with public XForms-driven pages,
• Daiwa - a Japanese Bank, for a transaction system,
• German Shipbuilders, for configuring ships,
• Fraunhofer (known for MP3) for configuring websites,
• Bristol-Myers-Squibb (pharmaceutical),
• Remia - a major Dutch food manufacturer,
• KDDI: embedded in a Japanese mobile phone,
• US Center for Disease Control: for disease control after hurricane Katrina

Interesting uses

Cordys: legacy

NACS: devices

Experience

The DoD did some research and discovered that 90% of the cost of software production was in debugging.

Interestingly, Fred Brookes in his book "The Mythical Man Month" reported that the number of bugs in a program is not linear with the length of a program but quadratic:

b ∝ L^1.5

Which means: if a program is ten times as long, it has 30 times as many bugs, which means it costs 30 times as much to make.

Conversely, a program that is 10 times smaller costs 3% of the larger program.

Experience

Because you specify what you are trying to achieve and not how to achieve it, there is far less administration to worry about. This means: shorter programs.

How much shorter?

One correspondent who was converting a large collection of apps for a company from Javascript to XForms reported that the XForms were about ¼ the size.

So that means we should expect the production time and cost to reduce to one eighth, about an order of magnitude. And indeed this seems to match experience.

Data point: 150 person years becomes 10!

A certain company makes BIG machines (walk in): user interface is very demanding — traditionally needed 5 years, 30 people.

With XForms this became: 1 year, 10 people.

Do the sums. Assume one person costs 100k a year. Then this has gone from a 15M cost to a 1M cost. They have saved 14 million! (And 4 years.)

Data point: A true story

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application

Two days later:

Programming man: I'll need 30 days to work out how long it will take to program it

XForms man: I've already done it.

The point

The point is, a declarative approach takes a lot of the work out of your hands.

This is a big win.

What can still be improved?

More data-sheet (select labels; help, hint, alert texts)

Data as an abstraction: not just XML

More abstracted accessors

Subforms

Idioms: repetition

Reducing the need for actions (more declarative-ness)

Generic XML editing (generating controls according to the structure).

Conclusions

XForms has more than met its requirements, and has turned from a forms language into a declarative applications language.

The advantages of a Turing-complete, declarative programming approach have still got to hit home in the wider community.