Open Source is Not Enough!

Steven Pemberton, CWI , Amsterdam

Contents

About me

1609

VOC Logo

The Dutch East India company, the world's first multinational, and the first company to issue stock, is fed up with the 8 to 12 months it takes to travel.

Routes

Routes to the east indies

Henry Hudson

So the VOC decide to do some research, and hire an Englishman, Henry Hudson, to discover "the Northeast route".

He is contracted to do only that.

But he doesn't.

Instead he turns the other way, and instead discovers Manhattan.

Hudson's Voyage

Then, and for centuries after, all trans-Atlantic shipping was done with sailing ships.

Hudson and the Halve Maen in the Hudson River

Steam ships

An early steam ship towing a sailing ship in a painting by TurnerWhen steam ships were introduced they were not reliable enough to travel trans-Atlantic distances, they couldn't travel far without breaking down, and they were inclined to blow up.

The Fighting Temeraire tugged to her last Berth to be broken up, by Turner, 1838:

Niche

But steam ships were able to find a niche in lake and river transport, where the distances were short, and where they had the advantage of being able to travel against the wind and on wind-still days.

Once in the niche they could improve reliability until they were able to travel trans-Atlantic.

Once that happened, all shipping switched to steam, and all the companies producing trans-Atlantic sailing ships went out of business; not one survived into the 20th century.

Sustaining improvements

Sustaining improvementsTypically, technologies improve gradually and continually over time. These are sustaining improvements. An existing technology fulfils or exceeds the market's needs.

Disruptive improvements

A disruptive technology is introducedSteam ships on the other hand are an example of a disruptive technology,a term introduced in the excellent book The Innovator's Dilemma by Clayton M. Christensen.

A disruptive technology when introduced is able to do similar things as an existing technology, but has one or more disadvantages that make it less desirable for the existing market, such as higher price, lower performance, lower reliability etc.

Niches

A disruptive technology improves to beat extant technologiesIf that were all, then of course no one would buy it, but a disruptive technology always has at least one advantage over existing technologies that despite its disadvantages allows it to find a niche, and therefore survive.

Whilst in this niche it is able to improve, typically more rapidly than the existing technology, until it performs equally with the existing technology, or at least sufficiently for the market needs, but with the added advantages that allowed it to survive.

As a result the existing market steps over to the new technology and the old one dies (typically taking the companies producing the technology with it).

Examples

There are huge numbers of examples of disruptive technologies.

Christensen's book documents many other examples, such as pneumatic diggers, 14 inch disks being replaced by 8 inch disks, being replaced by 5¼ inch disks, being replaced by 3½ inch disks, and the change from mainframe computers to minicomputers to PCs.

Other examples

You can probably identify lots of examples of disruptive innovations just by thinking of things that have essentially disappeared:

We can see lots of disruption now as a result of the internet (newspapers, books, music, film, travel agents, shops, ...)

Why don't the existing companies just shift to the new technology?

Is Open Source a Disruptive Technology?

Take Linux:

Can it be even better? It needs to do more than just imitate.

Making it better: Usability

Usability comes from designing things (software/programming languages/cookers) to allow people to do their work:

Efficient, Error-free with Enjoyment or
Fast, Faultless and Fun

Don't confuse usability with learnability. They are distinct and different.

Usability is important

What are the features of websites that you go back to regularly, that differentiate them from websites with the same purpose that you don't go back to?

Differentiating features

Forrester did some research on this:

(the rest is noise: 14% and lower)

Applied to software

Why Open Source is Bad at Usability

Every good work of software starts by scratching a developer's personal itch.

Eric Raymond, The Cathedral and the Bazaar

There's a closely related issue, however that I don't know how to solve yet without a big player with a lot of money, which is doing systematic user interface end user testing. We're not very good at that yet, we need to find a way to be good at it.

Eric Raymond, Why Open Source will Rule

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

When Intuitives drive to work, they watch the tectonic plates, deep in the earth's crust, rubbing together...

Psychology

Different people have different psychologies.

This seems almost too obvious to be true, but it is surprising how many people don't properly understand it.

My favourite description of how people – particularly programmers – differ is in chapter 15 of Bruce Tognazzini's book Tog on Interface.

When Sensories drive to work, they are aware of the birds, the trees, the hills turning green. They notice a cow lowing in the field. [...]

Intuitives live in their own private universe, depending on an internal model of external events. [...]

When Intuitives drive to work, they watch the tectonic plates, deep in the earth's crust, rubbing together. They run into the cow.

Programmers

Open Source software is produced by programmers.

Programmers are different from most people: a far greater proportion of programmers are intuitives than in the general public. They like the command line.

When programmers produce open source software, since they are largely scratching their own itch, they tend to produce the software for themselves, and in particular are perfectly content with the (programmer-oriented) user interface.

As an example, a review of photo editing software by the Dutch consumer reports magazine placed The Gimp as last because of its lack of ease of use.

Non-programmers

If Open Source software is to make inroads into the general public, something will have to be done about this.

The typical response to a complaint about a piece of open source software is "I'll show you where the source is".

But by definition the people with this particular itch won't be programmers, and they won't know how to fix the problem:

The challenge

If Open Source Software is to appeal to people that are not producing it

As Raymond remarked "a big player with a lot of money" would do it (which explains the success of Ubuntu and Android), but that is not scalable.

Robert Metcalfe

Invented Ethernet, for which he won the Turing award last year

Founded 3Com

Predicted the collapse of the internet in 1996 (which clearly didn't happen)

Has a law named after him

Metcalfe's Law

Metcalfe proposes that the value of a network is proportional to the square of the number of nodes.

v(n)=n2

Visual demonstration of the mathsSimple maths shows that if you split a network into two, it halves the total value:

(n/2)2 + (n/2)2 = n2/4 + n2/4 = n2/2

In other words, you have the same number of nodes, but only half the value.

This is why it is good that there is only one email network, and bad that there are so many Instant Messenger networks. It is why it is good that there is only one World Wide Web.

The Value of things joined

Internet: distance vs cost - the true cost of communication

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

For instance Wikipedia: 100,000 hours work

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

For instance Wikipedia: 100,000 hours work - the number of hours the US population spends watching adverts on TV in a weekend.

The Value of things joined

Internet: distance vs cost - the true cost of communication

The web: everyone sharing information shows us the true cost of information

For instance Wikipedia: 100,000 hours work - the number of hours the US population spends watching adverts on TV in a weekend.

For instance: Open street map.

Open source

Also an example of Things Joined

gcc → kernel → apps

The Open Source ecosphere just keeps getting better and better.

But applications are islands

In the days before graphics interfaces, almost all programs could be piped together to produce something more useful. They could be joined.

Unfortunately we have lost that ability.

Applications are islands that cannot be joined.

This means that a collection of applciations is of value n, instead of n².

Application Monoliths

A layered architectureEach application contains a functional part, a data accessing part and an interface part.

We need an architecture where those parts are separable, so that a program's data is available to other applications (think mashups); a program's functionality is available to other applications (think pipes), and where the user interface is replaceable without rewriting the whole app (think usability).

Open source is not enough: we need open data, open functionality and open interfaces as well.

Example: Image processing

A layered architectureThe Gimp image processing software has a data layer that can read and write many types of image, a functional layer that can edit, improve, and convert images, and a user interface layer that gives you access to that functionality.

If the layers were separable, other applications could use them to read or write images; to crop or resize images; having read the criticisms of the usability of Gimp, someone could write a new interface layer to make it easier to use, or make easy and expert user interfaces, without rewriting the underlying application.

An application ecosphere

With all the parts of applications in separate pieces, applications become a Metcalfe network of parts that can join together: the value of the whole increasing quadratically with the number of applications.

When you write a new application, you have done more than just added an application to the pool, you have added new functionality and new sources of data to the whole.

So instead of going from n independent applications to n+1 independent applications, where each new application adds value 1, you go from a network of cooperative applications of value n², to a network of value (n+1)², which is n²+2n+1.

So each new application adds a value of 2n+1. As the network gets bigger, so does the value of adding a new node.

Conclusion

Open Source is probably a disruptive technology.

Part of being disruptive is having advantages, and about getting rid of the disadvantages.

Usability needs serious attention.

One way of improving is to design an architecture that allows us to plug things together more easily, and in the process increasing the Metcalfe value of the whole.