The author

XForms – The Big Picture

Steven Pemberton, CWI and W3C, Amsterdam

About me

Researcher at CWI in Amsterdam (first non-military internet site in Europe - 1988, whole of Europe connected to USA with 64kb link!)

Co-designed the programming language ABC, that was later used as the basis for Python

At the end of the 80's built a system that you would now call a browser.

Organised 2 workshops at the first Web conference in 1994

Chaired the first style and internationalization workshops at W3C.

Chaired HTML WG for the best part of a decade.

Co-author of HTML4, CSS, XHTML, XML Events, XForms, RDFa, etc

Forms co-chair at W3C

XForms

XForms, as the name suggests, was originally designed as a replacement for HTML Forms.

The first version did roughly that, but thanks to generality in the design we realised that with small changes it could do much more: you could use it for other applications as well.

The resultant design is XForms 1.1.

A new version XForms 2.0 is in preparation.

How can a Forms Standard be used for Applications?

XForms has input, output, and a processing engine. That's all you need for an application.

Although an application may not look like a form, all it is doing is taking some input from the user, and responding to it.

The Bottom Line

Experience with projects defined with XForms has shown:

Around a ten-times saving in costs, compared with traditional methods.

Data point: 150 person years becomes 10!

A certain company makes BIG machines (walk in): user interface is very demanding — traditionally needed 5 years, 30 people.

With XForms this became: 1 year, 10 people.

Do the sums. Assume one person costs 100k a year. Then this has gone from a 15M cost to a 1M cost. They have saved 14 million! (And 4 years)

Data point: A true story

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application.

Data point: A true story

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application.

[2 days later]

Data point: A true story

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application.

[2 days later]

Javascript man: I'll need 30 days to work out how long it will take to program it.

Data point: A true story

Manager: I want you to come back to me in 2 days with estimates of how long it will take your teams to make the application.

[2 days later]

Javascript man: I'll need 30 days to work out how long it will take to program it.

XForms man: I've already done it.

How can this be?

A brief history of the development of programming languages.

1950's

In the 50's, when (so-called) high-level languages first started emerging, computers cost in the millions. Nearly no one bought computers, nearly everyone leased them.

When you leased a computer in those days, you would get programmers for free to go with it. Programmers were essentially free (in comparison with the cost of the computer).

(Nowadays it is exactly the reverse of course. Computers are essentially free. With a programmer maybe costing 100k a year, using a computer that costs €1000 spread over 3 years, it is the programmers who are expensive.)

The design of programming languages

What this meant was that the computer's time was expensive.

TypingSo a programmer would write the program, copy it to special paper, give it to a typist, who would type it out, then give the result to another typist who would then type it out again to verify that it had been typed correctly the first time.

Why all this extra work? Because it was much cheaper to let 3 people do this work, than to let the computer discover the errors for you.

The Design of Programming Languages

And so programming languages were designed around the needs of the computer, not the programmer. It was much cheaper to let the programmer spend lots of time producing a program than to let the computer do some of the work for you.

Programming languages were designed so that you can tell the computer what to do, in its terms, not what you want to achieve in yours.

Moore's Law

Since then Moore's Law has been doing its work, and computers have been getting exponentially faster.

Laptop speed 1988-2013

In 1988 my laptop had a power of 800. My present one has a power of more than 2M. That is 18 doublings! (So I've done better than Moore's Law!)

What exponential growth really means to you and me

Often people don't understand the true effects of exponential growth.

A BBC reporter recently: "Your current PC is more powerful than the computer they had on board the first flight to the moon". Right, but oh so wrong (Closer to the truth: your current computer is several times more powerful than all the computers they used to land a man on the moon put together.)

Take a piece of paper, divide it in two, and write this year's date in one half:

Paper

2013

Now divide the other half in two vertically, and write the date 18 months ago in one half:

Paper

2013
2012

Now divide the remaining space in half, and write the date 18 months earlier (or in other words 3 years ago) in one half:

Paper

2013
2012
2010

Repeat until your pen is thicker than the space you have to divide in two:

Paper

2013
2012
2010
2009
2007
2006
2004
2003
01
00
98
97
95
94

This demonstrates that your current computer is more powerful than all other computers you have had put together.

In other words

Since the 50's computers have become incredibly cheap and powerful, and yet we are still programming them with programming languages that were designed to save the computer work.

In fact most computers spend most of their time idle. Here is my computer as I write these slides. The bump at the end is caused by taking the screen shot.

A computer idling

What we need to do (and XForms does) is take the load off the programmer, and put more of it on the computer. Use some of that spare power!

1970's

So by the 1970's computers were becoming two orders of magnitude cheaper, and programmers weren't: the cost of software was starting to hurt. The term "software crisis" was even coined.

The American DoD did some research and discovered that 90% of the cost of software production was in debugging.

Interestingly, Fred Brookes in his book "The Mythical Man Month" reported that the number of bugs in a program is not linear with the length of a program but quadratic:

b ∝ l1.5

Which means: if a program is ten times as long, it has 30 times as many bugs, which means it costs 30 times as much to make.

Conversely, a program that is 10 times smaller costs 3% of the larger program.

The verbosity gap

What a quadratic increase looks like

The XForms advantage

And this is the main reason that XForms can produce such savings.

Because (as you will see) you specify what you are trying to achieve and not how to achieve it, there is far less administration to worry about in programs. This means: shorter programs.

How much shorter?

One correspondent in Denmark who was converting a large collections of apps for a company from Javascript to XForms reported that the XForms were about ¼ the size.

So that means we should expect the production time and cost to reduce by one eighth, about an order of magnitude. And indeed we do.

What does it mean to be an order of magnitude more productive?

A program that you now would write in a week, you could write in a morning.

A program that now would take a month you could write in two days.

A program that would take a year to write, you could produce in a month.

Some History

May 1998

W3C Workshop "Shaping the Future of HTML"

This workshop decided there should be a new WG to create a modularised XML version of HTML, with new features for forms.

"T.V. Raman: stronger typing of fields and the ability to specify at least simple constraints using declarative means. Now it's being done with Javascript and there's no point to using Javascript just to verify that the content of a field is a number".

HTML WG

Dec 1999: HTML 4.01 ready, and XHTML 1.0.

Works starts on XHTML 1.1, a cleaned up version, and future versions.

There is a very large contingent of people only interested in Forms: Forms WG is spun off.

Analysis

Forms had been added to HTML in 1993. Very simple, mostly presentation-oriented, but effective, and widely used.

Searching

Google

Buying

Amazon

Logging in

Yahoo

Configuring hardware

Linksys router

Reading mail

Reading mail

Composing email

Composing email

Etc etc

Problems with HTML Forms

The WG

Consisted of many Forms companies; people representing accessibility; usability; abstraction, etc.

We all had our own requirements, and we vaguely knew it had to be with XML, but not how.

Requirements

Obvious things that were needed included:

A Timeline, XForms 1.0

1999

2000

2001

2002

2003

=Requirements document

=Working draft

=Last call

=Candidate Recommendation

=Proposed Recommendation

=Recommendation

=Implementors' workshop

XForms 1.0

At the time, the most implemented W3C spec ever (25 known implementations).

In February 2003 we had an implementors' workshop, where a sizeable number of implementors turned up.

We had client-side implementations, server-side, and a mix.

We had native, and plug-ins, desktop and mobile.

XMLDOMCSSJavascriptXHTMLXPathXForms

    =Regular XHTML Browser

Time to build all of above: 3 programmers, 4 months

Total footprint (on IPAQ implementation): 400K (above Java VM)

The Oracle implementation was particularly impressive.

Some of the things we had explicitely designed for

Device independence

Freedom of implementation choices (server/client, and mix)

Don't let implementation drive design decisions.

We also knew there were other things we wanted, but you have to ship it at some point.

The strengths of the design

Structured data

Separation of data and presentation

Integrated support of external data

Much is declarative, and there are hooks to allow actions when it isn't.

Forms are tractable.

Support of use cases missing from HTML in a generalised manner

A large degree of generality

The major weakness (of XForms 1.0) (in hindsight)

Too slavish following of HTML features, in particular using fixed strings where HTML used them.

Low hanging fruit

Several implementations had added extensions, not always compatibly.

We wanted to fix that as soon as possible.

We saw a number of low-hanging fruits we could pluck.

We also saw the chance, after some experience with the language, to generalise further.

A Timeline, XForms 1.1

2004

2005

2006

2007

2008

2009

=Working draft

=Last call

=Candidate Recommendation

=Proposed Recommendation

=Recommendation

Applications

XForms 1.1 was more general than XForms 1.0, and fixed a number of the worst cases.

XForms 1.1 had input, output and a (Turing-complete) processor.

All of a sudden, people were producing applications with XForms, things that didn't look like forms at all.

The Approach: Data

The essence is to separate what is being returned from how the values are filled in. Compare this with CSS style sheets: in the head you specify the styling, and in the body, you have the content to be styled.

The data may be pre-populated from local or external sources, and repopulated at any time in the form-filling process.

The Approach: Controls

An essential difference with HTML is that XForms controls are intent-based rather than presentation oriented.

Rather than specifying that a control consists of radio buttons, or a menu, XForms specifies what the control does, for instance that it selects one item from a list of items. CSS or similar can be used to provide the necessary presentation.

This approach allows the same XForm to be used across different devices without change.

The labels may be populated from local or external sources (good for localisation and multi-lingual environments)

Intent-based Controls

Controls are intent-based, by expressing what the control should do, rather than how it should look. So a control can be represented in different ways depending purely on styling.

Here are three identical controls, just styled differently

(Source)

You can output values as well as input them

(Source)

Controls adapt

In this example, nothing has been said about the type of the data being collected (so it gets the default string type):

(Source)

Controls adapt

Here, the control is exactly the same, but the data has been defined as being a date. The control adapts to the type:

(Source)

Controls can be pre-populated

Many forms have a field asking which country someone comes from.

In XForms such data can be kept in a separate file, and updated once, so that all forms that use it get updated simultaneously.

See next slide for an example.

Forms can be multilingual

The labels and values can be initialised from external files. SO adding a new language to a form is just a case of adding the labels for that language to the external file. You don't have to change the form.

(Source)

Improving the user experience

XForms has been designed to allow much to be checked by the browser, such as

It also allows you to calculate values, such as the sum of a group of figures.

This reduces the need for round trips to the server or for extensive script-based solutions, and improves the user experience by giving immediate feedback on what is being filled in.

Authoring and maintaining complicated forms

Because XForms uses declarative markup to declare properties of values, and to build relationships between values, it is much easier for the author to create complicated, adaptive forms, and doesn't rely on scripting.

An HTML Form converted to XForms looks pretty much the same, but when you start to build forms that HTML wasn't designed for, XForms becomes much simpler.

XML

XForms is properly integrated into XML: it is in XML, the data it collects in the form is XML, it can load external XML documents as initial data, and can submit the results as XML.

By including the user in the XML pipeline, it at last means you can have end-to-end XML, right up to the user's desktop.

It can talk to web services, using SOAP or XML-RPC; however, it still supports 'legacy' servers.

Use of XML technologies

Rather than reinventing the wheel, XForms uses a number of existing XML technologies, such as

This has a number of benefits:

Integration with existing data streams

Data can be pre-loaded into a form from external sources.

Existing Schemas can be used.

It integrates with SOAP and XML RPC and JSON.

Doesn't require new server infrastructure.

Device independence

XForms on a PDAThanks to the intent-based controls, the same form can be delivered without change to a traditional browser, a PDA, a mobile phone, a voice browser, and even some more exotic emerging clients such as an Instant Messenger.

This greatly eases providing forms to a wide audience, since forms only need to be authored once.

Internationalisation

Thanks to using XML, there are no problems with loading and submitting non-Western data.

Accessiblity

XForms has been designed so that it will work equally well with accessible technologies (for instance for blind users) and with traditional visual browsers; this means that XForms are accessible out of the box.

New use cases

Apart from regular HTML forms uses, XForms supports:

Implementation cost

XForms is rather easy to implement

XMLDOMCSSJavascriptXHTMLXPathXForms

    =Regular XHTML Browser

Time to build all of above: 3 programmers, 4 months

Total footprint (on IPAQ implementation): 400K (above Java VM)

It is royalty-free and unencumbered

So there can be no vendor lock-in! (If you think this is a good idea, join W3C!)

Deployment

There are several methods of deployment:

Native (eg Openoffice)

Client-side (eg XSLTForms)

Server-side (eg Orbeon, Betterforms)

Open-source implementations

There are a number of open-source implementations:

XSLTForms is an open-source client-side implementation based on XSLT 1.0 to compile XForms to vanilla (X)HTML and Javascript.

Orbeon Forms, a server-side implementation

The betterFORM project, server-side.

Xfolite is a light-weight XForms client for the J2ME platform. It was originally created at Nokia Research Center.

Enterprise implementations

Many companies use XForms either internally or externally, and have their own implementations, either for internal use or for licensing. For instance:

Users

Examples include.

Interesting uses

Cordys: legacy

NACS: devices

A Final Example

Maps

Conclusions

XForms is a flexible solution for forms and related applications.

It is an open standard.

Its use saves a lot of money.

XForms resources

A tutorial: http://www.w3.org/MarkUp/Forms/2010/xforms11-for-html-authors/

For an overview of all features, elements and attributes of XForms 1.1, see the XForms 1.1 Quick Reference.

It's not easy reading, but the final arbiter in questions of doubt is the XForms 1.1 Specification.

Living XForms 2.0 Draft: http://www.w3.org/MarkUp/Forms/wiki/XForms_2.0

The implementation used for the examples in this talk is XSLTForms.