Date: November 19 2009

Currently: Reading F1 Racing (October 2009 edition)

Mood: Contemplative !:l

Warning! Long entry alert!

This blog entry started as a comment in Sidepodcast's discussion about trustworthy websites. During the discussion, there had been a fair bit of stick thrown at GMM (Global Motorsport Media) for stories being sent through their network without proper checking. What nobody had expected was for GMM itself to reply - at length.

I started typing a response, then came to the conclusion that nobody was going to stand reading the length of that reply in someone else's blog. It is worth saying that I'm only replying to the part of GMM's comment that requires a long response - quite a bit of it made perfect sense and merits little more than an "I agree". (If a GMM representative goes so far as to read this blog entry, then I thank them for their effort in doing so :) ).

The Whipped Cream Model of The Effects of Information Accuracy Expectation

Part of the GMM representative's comment queried how GMM could get so much flak when it was ultimately one of their sources that made the error. It is an interesting question. There definitely is a tendency to send blame for errors throughout the part of the system preceding the one which caught the error. This is because information is not like a water main system, where fixing the leak makes the system run correctly again (usually). Information typically goes through several minds before reaching the end of its influence, which means there are many checking points. It is part of human nature to check information. Therefore, the expectation is that information spread is as accurate as the provider of that information knows it can be. That is why information reliability has a reflection on everyone who carries that item of information.

The quality of an information provider is affected (among other things, most of which are not even touched upon by this discussion) by how closely the items of information provided link to reality. Accurate information increases the quality and the reverse is true. In fact, given how the human mind works, negative experiences stand out more than positive ones (and, in general, have more influence). One erroneous item of information, if in the wrong place, can throw askew a lifetime of understanding.

One could call it the "whipped cream" principle - in the same way as whipped cream transported from a cartridge to a bowl of pudding leaves cream stuck to the edges of dispenser and pudding as well as the cartridge, so the effects of information stick to the conduits and recipients of that information as well as the source. If the cream is in-date, then nice cream will stick to everything. If the cream has gone off before it's piped through, then everything will stink and, unless washed down properly, all the cream that comes out will taste funny even if the source was subsequently good.

This is why scientific journals have a process of peer review and reputable news journalists have checking systems. I have no idea what checking systems the German Focus magazine might have, because the only Focus magazine I'd heard of before yesterday was the UK science title. Focus should have checked the information before publication, but by the same token everyone down the chain should have checked the information as best as they were able as well.

As to why the messenger is getting shot, that is because blame is considered to attach to every part of the system that delivered the false information (the reverse is also true). So when the vote story was found to be inaccurate, Focus got some of the blame, GMM got some of the blame, the sites that spread the story thereafter got some of the blame and (in some cases) the individuals who went on to tell their friends about it got some of the blame.

However, previous track records are taken into account when the total effect of an information error is calculated:

- Focus is not the source of many stories we know about. While the readers of the vote story probably wouldn't trust it again, they are also unlikely to be in a position where that is an issue.

- GMM has had quite a few erroneous or misleading stories in the past. The effect of the vote story was quite small in terms of perception, it just so happens that it was the story that caused things like this discussion to take place (owing to a number of things over which GMM has limited/no control, such as the lack of substantial news that interested Sidepodders at that moment, the discussion of media quality issues in previous weeks prompted by the crossing of cut-off points (discussed in the "Information Accuracy, Freedom and Authority" section) and the tendency of Sidepodders to make lists - "Websites You Can Trust" being a very easy activity to turn into a list).

- The sites spreading the story often do what Keith does and moderate their feed. This reduces their informational error rate. While the individual erroneous story will have had more impact on them than GMM (and the more accurate the site usually is, the bigger the impact), the higher starting point of trust means that the net trust level is still higher than GMM's.

- The effect on individuals is varied and difficult for me to comment on personally, since I only told other people about it after the story was revealed to be false. Everything from zero effect to a complete loss of trust is possible, depending on the history of the relationship, the track record of the individual information provider and the receptivity of the recipient.

Granted, not everyone has the phone number to the right person in Mercedes to establish whether a vote on the F1 programme happened or not. I suspect that the nature of GMM may preclude it from having the right contacts to absolutely prove or disprove every story that crosses its desk (sometimes even the specialist press struggle). But a web monitor role implies that there is a certain amount of checking for accuracy.

Asking the source of each news item "What is the background for this item?" where it is not immediately obvious would be a good way of reducing the error rate while remaining within the powers GMM has. I suspect the answer received for the vote story would probably have allowed GMM to stop the story in its tracks, and maybe even issue a cautious denial itself. In the latter case, it would even have a (valid) scoop, which would increase the quality of the information conduit. In turn, this would increase the trust people had in GMM and increase GMM's ability to monetise that trust (which ultimately helps keep GMM going and growing).

Information Accuracy, Freedom and Authority

Next, the GMM representative wondered how the denial of the story invalidated the GMM story.

The explanation of this is summarised in yesterday's blog entry. What happened was that James Allen, a journalist most noted for his ITV commentary but who also writes books, mentioned in his blog that he had phoned Mercedes and they had denied the story. The key here is "he had phoned Mercedes". This immediately gave his take on the matter a more solid basis than the previous existing information, including that from GMM. Also, whatever gripes people may have had over James' commentary, he was not known for giving inaccurate information out (and when he did he usually got corrected before any damage got done, usually because the error had been committed in the high-pressure arena of the commentary box - one of several reasons why many high-speed sport broadcasters have two commentators). Taken together, this meant the information was most likely reliable.  This is known as information authority.

Information with authority tends to be believed over information lacking in authority. In an environment where pre-censoring is not practised (as in a place with a free press), this becomes important. The more censoring of information is delegated to individual intelligences, the more important information authority is. Note that information authority does not come from it having been approved by some (psuedo)political institution (otherwise we'd take everything the FIA said at face value), but through the likelihood of the story's proximity to the truth, the clarity and detail of the information, the track record of the information provider and how well the information fits in with other items of information the recipient possesses (or believes he/she possesses).

A free media tries to meet whatever is considered the most important value in the society in which it is placed. In the capitalist West, this often means being as profitable as possible. However, whatever other value(s) the press may chase have to be balanced by at least a modicum of what the people want, otherwise who will read their material? It so happens that when it comes to information, what most people want is authority. Many are satisfied by a low level of authority, but many others want the highest level of authority possible. Yes, it is possible for a news source to be wrong with authority, but if it is frequently wrong it will not retain its authority for long. The less authority an information source has, the less likely it is to be believed. "The Boy Who Cried Wolf" demonstrates this principle simply but brilliantly.

As for the analogy entailing our reaction to getting fifteen good strawberries and one bad one, there will be a variety of behaviours in that situation. Again, these vary according to the expectations of the buyer, the quality of the previous strawberries from that vendor and the quality of strawberries from other vendors. If fifteen out of sixteen is relatively good in the strawberry market, then people will be happy to continue with the vendor and will easily forgive the bad one. If other vendors routinely get sixteen good strawberries out of sixteen (and, to improve the analogy, charge more - with the strawberries, the price is currency, the other outlets' price is time waiting for the items), then responses get more mixed. Some will want the cheap strawberries regardless of quality, some want the consistently reliable strawberries regardless of the cost. The majority will consider both types. The more frequently bad strawberries appear, the more likely the high-quality ones will be chosen instead of the cheap ones, but everyone has different cut-off points. The vote story may well have crossed a few key people's cut-off points.

Speculation

The GMM representative made an interesting point about speculative stories. He/she says that 6 out of 10 "highly speculative" stories are completely false, 3 out of ten are partially true and 1 out of 10 is completely true. GMM's position is pro-speculation. A lot of the response is anti-speculation. I would be somewhere in between.

I don't mind speculation if it is clearly marked as such. If someone providing information has reason to doubt its accuracy, then that tag helps put the information in its proper context. This is especially true if a) the provider has specific grounds for that doubt and b) those grounds are provided along with the information. While it would be awkward for GMM to say it doubts an item because its source has previously provided dodgy information (for one thing, the source might refuse to provide further information to it!), other causes of doubt could be provided.

This is especially important given the failure rate of speculative stories. If we take the "6 out of 10" figure as true (while I have no proof, it's the most accurate working figure I've seen and the GMM representative is more likely to know the figure than I am), then that means the majority of such stories are false. That is a serious failure rate for information of this type, even if speculative stories form a small proportion of the total output. If an investment had a 6 out of 10 chance of losing money, the government would demand some sort of warning. While it would be wrong for a free press to have such a thing imposed on it from the government, the general public would benefit from wanting/expecting/requesting it of their information sources. GMM should be capable of it - the statistic provided by the GMM representatives proves that it is capable of recognising a speculative story, at least to an extent where it is confident of knowing the boundaries for the purposes of providing that statistic. If degrees of uncertainty can be indicated, so much the better, but one can't have everything.

Speculative stories presented as fact have no place in my diet of information. There's enough bias and disinformation-at-source to filter out without knowingly getting more from the intermediaries separating me from the source. Granted, GMM is not my brand of vodka (I don't drink vodka, but that's another story entirely...), but quite why anyone would want speculation to look like fact is a mystery. Of course I want to know as much as possible about F1. But taking in inaccurate information that has to be unlearned and replaced in short order gets in the way of that goal, especially since I only have twenty-four hours in a day*.

As for "wondering ahead of time"... ...humans are good at that when left to their own devices. All the likes of GMM can do is direct the path of such wondering. Asking for the direction to be as valid as possible seems reasonable.

De-bookmarking is a problem because of the ubiquity of GMM content across the internet (and, at various times, other erroneous stories). The interconnectedness of the internet can be wonderful at times, but a simple de-bookmark wouldn't remove the content in its entirety. What needs to happen is a better appreciation of how to spot a good story from a bad one, improved story moderation from everyone and a preference for accurate information over inaccurate stuff.

* - Could be worse. In the Triassic period, there were only twenty-three hours per day...