Abstraction and extraction: in praise of

Steven Pemberton, CWI and W3C, Amsterdam

About me

Researcher at the Dutch national research centre CWI (the first European internet site)

Co-designed ABC, the programming language that Python is based on

Wrote part of GCC (float.h is generated by what I wrote)

In the 80's co-designed what you would now call a browser

Organised 2 workshops at the first Web conference in 1994

Co-author of CSS, HTML4, XHTML, XML Events, XForms, etc

Chair of XHTML and Forms working groups

Once editor-in-Chief of ACM/interactions.

Programming languages

Algol 68: Little known now but very influential

Much of C and Unix was influenced by Algol 68

If you use the word 'dereference' then so are you.

It was an interesting language, and much directed to making the programmer's life easier.

One of the nice things about programming languages is that they abstract away detail, like how data structures are implemented, how procedures are called, and do a lot of the administration for you.

Unix

I went to a talk in the 70's by Kernighan and Ritchie where they were introducing Unix

Funnily enough, they were selling it from a usability point of view: look how easy it is to do these things compared with other operating systems. And they were right.

And by the way: there was an abstraction error in C, that shows all the way through Unix as a result - they conflated characters with units of store. Those of us in the programming business at the time saw it coming.

UTF-8 only exists because of that mistake. If they hadn't, the ordinary programmer would never have to be confronted with character encodings (because we would just be able to say 'char c;' and the system would deal with the details).

Termcap was a similar abstraction error, though with less far-reaching effects.

The battle

It is difficult now to realise that there was a battle then between assembly language and 'high-level' programming language.

The argument for high-level languages was: Computers are becoming more powerful: let them do the work!

There is evidence that a high-level programming language (of the level of C or Pascal) can reduce effort by an order of magnitude (a program that would take you a week can take you an afternoon).

We live in an exponential world

To demonstrate Moore's LawTake a piece of paper, divide it in two, and write this year's date in one half:

Paper

2007

Now divide the other half in two vertically, and write the date 18 months ago in one half:

Paper

2007
2005

Now divide the remaining space in half, and write the date 18 months earlier (or in other words 3 years ago) in one half:

Paper

2007
2005
2004

Repeat until your pen is thicker than the space you have to divide in two:

Paper

2007
2005
2004
2002
2001
1999
1998
1996
95
93
92
90
89
87

This demonstrates that your current computer is more powerful than all other computers you have had put together (and the original Macintosh (1984) had tiny amounts of computing power available.)

Usability: What is it?

Designing things (software/programming languages/cookers) to allow people to do their work:

Efficient, Error-free with Enjoyment or
Fast, Faultless and Fun

Don't confuse usability with learnability. They are distinct and different.

Usability is important

What are the features of websites that you go back to regularly, that differentiate them from websites with the same purpose that you don't go back to?

Differentiating features

Forrester did some research on this:

(the rest is noise: 14% and lower)

Applied to software?

Scratching Someone Else's Itch

(Why Open Source can't do Usability)

The itch

Every good work of software starts by scratching a developer's personal itch.

Eric Raymond, The Cathedral and the Bazaar

Someone else's itch

There's a closely related issue, however that I don't know how to solve yet without a big player with a lot of money, which is doing systematic user interface end user testing. We're not very good at that yet, we need to find a way to be good at it.

Eric Raymond, Why Open Source will Rule

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

When Intuitives drive to work, they watch the tectonic plates, deep in the earth's crust, rubbing together...

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

When Intuitives drive to work, they watch the tectonic plates, deep in the earth's crust, rubbing together. They run into the cow.

Programmers

Open Source software is produced by programmers.

Programmers are different from most people.

For instance, a far greater proportion of programmers are intuitives than in the general public. They like the command line.

This means that when programmers produce open source software, since they are largely scratching their own itch, they will tend to produce the software for themselves, and in particular be perfectly content with the (programmer-oriented) user interface.

As an example, a recent review of photo editing software by the Dutch consumer reports magazine placed The Gimp as last because of its lack of ease of use.

Non-programmers

If Open Source software is to make inroads into the general public, something will have to be done about this.

The typical response to a complaint about a piece of open source software is "I'll show you where the source is".

But by definition the people with this particular itch won't be programmers, and they won't know how to fix the problem:

The challenge

This means that if Open Source Software is to appeal to people that are not producing it

As Raymond remarked "a big player with a lot of money" would do it (and see Ubuntu), but that is not scalable.

ABC

So there we were in the early 80's designing a programming language based on principles of usability

Usability for the programmer

A typical project meeting

A typical project meeting for the ABC group

ABC

Designed with a 10 year range.

Interpreted

Very high level (only 5 data types)

Gave more work to the computer, reduced the programmer's time by an order of magnitude

Views

So there we were in the late 80's and we decided to build what we called an application environment.

Designed with a 10 year range.

Designed to increase usability for the programmer and the end-user.

The system had an extensible markup language, vector graphics, style sheets, a DOM, client-side scripting...today you would call it a browser (it didn't use TCP/IP though).

It ran on an Atari ST (amongst others).

Clocks

Four clocks in the Views system

Programming Clocks

The shortest code I could find of an analogue clock was something over 1000 lines of C (the longest was over 4000 lines):

1000 lines of C code

Clock

Here is the essence of the code used for the Views clock example.

type clock = (h, m, s)
displayed as 
   circled(combined(hhand; mhand; shand; decor))
   shand = line(slength) rotated (s × 6)
   mhand = line(mlength) rotated (m × 6)
   hhand = line(hlength) rotated (h × 30 + m ÷ 2)
   decor = ...
   slength = ...
   ...
clock c
c.s = system:seconds mod 60
c.m = (system:seconds div 60) mod 60
c.h = (system:seconds div 3600) mod 24

About the length of code

The US DoD discovered that 90% of the cost of software production is debugging

Fred Brooks of IBM discovered that the number of bugs in a program doesn't grow linearly with the size of the program, but quadratically: S1.5

In other words a program that is 10 times longer takes more than 30 times the effort/cost

Or put another way: a program that is one tenth the size costs 3%

Compare this

In the middle ages, 1545 to be exact, the mathematician Cardan wrote in his Ars Magna (in Latin, thus my thanks to Lambert Meertens for this translation):

Raise the third part of the coefficient of the unknown to the cube, to which you add the square of half the coefficient of the equation, and take the root of the sum, namely the square one, and this you will copy, and to one {copy} you add the half of the coefficient that you have just multiplied by itself, from another {copy} you subtract the same half, and you will have the Binomium with its Apotome, next, when the cube root of the Apotome is subtracted from the cube root of its Binomium, the remainder that is left from this is the determined value of the unknown.

What he was trying to say was that one root of x3 + px = q is calculated so:

d = (p/3)3+(q/2)3
c = sqrt(d)
b = c + (q/2)
a = c - (q/2)
x = cuberoot(b) - cuberoot(a)

a calculation that any reasonably trained schoolchild can even prove nowadays.

And so to the Web

The term Ajax was invented by Jesse James Garrett, a Usability Expert, to promulgate the techniques as a method of improving usability, primarily by lowering latency times for the end user (Usability=speed+correctness+fun)

But producing an Ajax-empowered page is a lot of work!

Data-point: Google maps, poster child of the Ajax Generation, more than 200k of code.

Does it have to be so hard?

Let the computer do the hard work

A dual CPU being twice as idle

Declarative programming

While everyone is able to write a Web page, almost no one can write an application (programmers are different)

Except: spreadsheets

Why? Declarative programming. The computer does the administration.

XForms

Although XForms comes from designing a replacement for HTML Forms, it is really an application language: it has input, output and computations.

The data is abstracted away into several 'instances' which can be loaded and saved asynchronously over the net.

The 'controls' are really abstract: they only say what the control is supposed to do, not how they look.

The computational model is constraint-based, i.e. like spreadsheets.

Using techniques similar to stylesheets (i.e. XBL) you can define the presentation separately, and even have different presentations for different circumstances (e.g. for different devices).

Datapoint: Google Maps

As a pilot, someone implemented (a version of) Google Maps using XForms.

Google Maps done with XForms

Result: 25k bytes of code.

(Actually had satellite option before Google Maps did)

Datapoint: Machine interface

A company that builds huge walk-in machines with complicated user interfaces needed 30 people working for 5 years to build the user interface.

With XForms 10 people needed a year.

Conclusions

In 40 years computers have become some 25 orders of magnitude faster.

That's: 10,000,000,000,000,000,000,000,000 times faster.

Programmers have managed 2 or maybe 3 orders of magnitude in that time.

In the 1960s if you bought a computer from IBM, you got free programmers in the deal.

Nowadays it is the hardware that is free (comparatively).

If we want to win, rather than just play catch up all the time, we need to use our available resources wisely.