From Tony Tue Aug 29 12:31:16 2006 Content-Transfer-Encoding: quoted-printable Subject: RE: Isoft fiasco Date: Tue, 29 Aug 2006 12:31:10 +0100 To: "Aaron Sloman"
Dear Aaron, I enjoyed reading your analysis of large-scale government procurement. But I don't think Lynne Jones will: it is far too long. Politicians think in soundbites these days. Think what you actually want her to do. Is it to ask for a commission of enquiry into government high-tech procurement? Then say so; and perhaps write a 'terms of reference' for the enquiry. Yours, Tony. From George Coulouris Tue Aug 29 14:09:38 2006 Subject: Re: Isoft fiasco Date: Tue, 29 Aug 2006 14:08:55 +0100 To: Aaron Sloman Aaron, > Some of those developments might have happened faster if the near > monolithic power of IBM in the commercial world had not prevented > deployment of much better alternatives to IBM PCs (e.g. products > from Sun, Apple, Apollo and other smaller companies using non-Intel > based cpus, and non-Microsoft software). It's not easy to tell > whether there would have been the same drop in costs because of > growth of numbers and the existence of clone makers. (Sun > Microsystems from the start allowed others to build hardware to run > their operating systems, unlike Apple, for instance.) It might be worth mentioning the existence of UK-based alternatives (Acorn, Whitechapel Computers - there's quite a bit on the net about the former, but apparently nothing on the latter, though its designers are still extant), none of which received any significant government recognition or support, unlike their US competitors (through defence contracts). > Some of the history of the ten years from 1900 can be found here. Should be 1990. Best wishes, George George Coulouris Visiting Professor in Residence Computer Laboratory Cambridge University [I replied mentioning Bleasdale too.] From C.Tully Tue Aug 29 15:37:21 2006 Date: Tue, 29 Aug 2006 15:36:20 +0100 From: Colin Tully Subject: RE: Isoft fiasco To: Aaron Sloman Dear Aaron Super read. I only hope your MP finds the time to read it! You say: "If there is a national problem that seems to require the development of a large and complex system, then governments must find a way to grow our understanding of the problem as we grow the solution, and thereby grow our understanding of all the detailed sub-problems, in small steps with a lot of parallel exploration of options." Does this seem to be a modern restatement of Karl Popper's position in "The open society and its enemies", about which Bryan Magee wrote as follows? "Because he regards living as first and foremost a process of problem-solving he wants societies which are conducive to problem-solving. And because problem-solving calls for the bold propounding of trial solutions which are then subjected to criticism and error elimination, he wants forms of society which permit of the untrammelled assertion of different proposals, followed by criticism, followed by the genuine possibility of change in the light of criticism. Regardless of any moral considerations ... he believes that a society organized on such lines will be more effective at solving its problems, and therefore more successful in achieving the aims of its members, than if it were organized on other lines." I'm aware of the mixed and lively reactions that Popper evokes, but I hope I'm not just being mischievous. Yours, Colin. ===== From Aaron Sloman Tue Aug 29 17:40:24 BST 2006 Date: Tue Aug 29 17:40:24 BST 2006 Subject: Re: RE: Isoft fiasco Responding to Colin Tully Thanks Colin > Super read. I only hope your MP finds the time to read it! I know it is rather long (one of my faults), and more than one person has said so, but I wanted to provide examples to liven it up and also make it fairly easy to read, which ruled out some types of compression. Lynne Jones has a PhD in biochemistry and is capable of reading and writing about quite technical topics. I agree with her on most things -- but not all. > You say: "If there is a national problem that seems to require the > development of a large and complex system, then governments must find a > way to grow our understanding of the problem as we grow the solution, > and thereby grow our understanding of all the detailed sub-problems, in > small steps with a lot of parallel exploration of options." > > Does this seem to be a modern restatement of Karl Popper's position in > "The open society and its enemies", about which Bryan Magee wrote as > follows? > > "Because he regards living as first and foremost a process of > problem-solving he wants societies which are conducive to > problem-solving. And because problem-solving calls for the bold > propounding of trial solutions which are then subjected > to criticism and error elimination, he wants forms of society which > permit of the untrammelled assertion of different proposals, followed by > criticism, followed by the genuine possibility of change in the light of > criticism. Regardless of any moral considerations ... he believes that a > society organized on such lines will be more effective at solving its > problems, and therefore more successful in achieving the aims of its > members, than if it were organized on other lines." > > I'm aware of the mixed and lively reactions that Popper evokes, but I > hope I'm not just being mischievous. Not at all. I was not consciously echoing Popper. I am a great admirer of much of his work (though not the over-simplified versions often attributed to him, and I think he got some things wrong, including the emphasis on the discovery of laws as opposed to the discovery of possibilities -- a key feature of computer science). I had completely forgotten the bit you quote, though I have often recommended Magee's little book on Popper (in the Modern Master's series) as an excellent introduction to Popper's ideas, and to philosophy of science. Perhaps I'll add a footnote. Thanks also to the others who have sent me comments. Jon wrote: > perhaps we could all send copies or versions to our MPs Since I have made it public it can be used in any way anyone desires, though sending a link may be better than sending a copy, since it may evolve. One thing I am not yet clear on is what action can be taken and at what level. One suggestion I've had is to propose a commission of enquiry into government high-tech procurement. I am not sure I know enough to do that well. Aaron http://www.cs.bham.ac.uk/~axs/ From jon.crowcroft Tue Aug 29 16:06:07 2006 To: Aaron Sloman Subject: Re: Isoft fiasco Date: Tue, 29 Aug 2006 15:24:13 +0100 From: Jon Crowcroft you're leter is spot on (if a bit long for a non techie, but ought to be ok if you MP Is clueful) perhaps we could all send copies or versions to our MPs i assume you've read the missive by ross anderson et al i think is shocking they fell into this trap! they had several (even government) projects that set good examples... cheers j. From Aaron Sloman Tue Aug 29 18:17:42 BST 2006 To: Jon Crowcroft Subject: Re: Isoft fiasco Thanks Jon, > you're leter is spot on (if a bit long for a non techie, but ought to be > ok if you MP Is clueful) > > perhaps we could all send copies or versions to our MPs See previous comment. > i assume you've read the missive > by ross anderson et al I've heard of it, but haven't found it yet. It doesn't seem to be accessible via his web page. I'll try asking google another way. Or if you have a link you could send me I'll be grateful. I guess you have now seen his response to me. > i think is shocking they fell into this trap! > they had several (even government) projects that set good examples... I fear this is a government with too many stupid fanatics with a little dangerous learning near the top. Aaron From owner-cphc-membersTue Aug 29 17:27:26 2006 Date: Tue, 29 Aug 2006 16:37:21 +0100 From: Ross Anderson Subject: Re: Isoft fiasco Aaron > Comments and criticisms welcome I've wondered for many years why so many government IT projects fail, and I'm not sure complexity arguments suffice. They explain some of the failures, but not why the public sector is so manifestly worse than the private sector. I believe you have to consider factors from organisational theory and public-choice economics. Civil service departments behave the way they do because of the incentives facing civil servants and ministers. There is a vast research literature on this; entry points include http://en.wikipedia.org/wiki/Public_choice_theory http://en.wikipedia.org/wiki/New_Institutional_Economics http://en.wikipedia.org/wiki/James_M._Buchanan This body of literature was mined diligently by the scriptwriters of 'Yes, Minister'. Briefly: people optimise their own utility as well as that of their employer. Ministers go for soundbites, while civil servants try to expand their bureaucratic empires. Getting projects approved often involves log-rolling - putting in features to appeal to colleagues or political constituencies, rather than for strictly functional reasons (think of all the things the ID card was going to do at various times during its journey through Parliament). Dodging liability, and dumping risk, are deeply entrenched. Ministers often optimise their career chances by acting as advocates for their department rather than for the public that elected them. This is all well-explored by many shelf-feet of books on politics and economics. There's other stuff on the systems side. Why is it, for example, that everything from Windows to mobile phones ends up with too many features, at the cost of both usability and dependability? Well, features get added until the marginal cost equals the marginal benefit. The benefit of each feature accrues to a small vocal group of customers; the cost is a slightly increased probability of a blue screen of death or whatever, spread across the whole user base. The result is a 'tragedy of the commons' in which every programmable device at equilibrium has so many features that the median user is just about to throw it at the wall out of frustration. Dependability is very far from the social optimum: the time spent each year by people waiting for Windows to reboot may exceed Microsoft's market cap. For more, see for example: http://www.cl.cam.ac.uk/~rja14/econsec.html How do we combine the insights from political economy and the analysis of system dependability? This would make a nice PhD thesis for someone who didn't mind becoming seriously unpopular in Whitehall! There are various starting points. First, politics in a democracy is about reconciling conflicting interests, so it often seems wise to keep the specification of a system vague. Second, as ministers crave publicity, they need frequent announcements - so they keep changing the spec, just like the tax code. Third, as ministers rotate every few years, promising a system that will solve all current problems, in five year's time, is a risk-free way of 'doing something'. It would be nice to have the time to develop this further - there are no doubt dozens of other factors. But I'm sure the core of it is this: the way ministers work is exactly at the other end of the spectrum from how a good project manager works. If you're building a system to a fixed-price contract, you race to close down options as fast as you can, to remove uncertainty, to get the client to make the hard decisions about design tradeoffs and sell them to his colleagues, to agree not to tinker with anything until you deliver the software, and to take ownership of the system as you build it. All of these are at odds with the incentives facing an ambitious minister. Now, the dependability literature teaches us that large software project failures are mostly due to overambitious, vague and / or changing specifications, coupled with poor communications and an inability to acknowlegde the signs of failure early enough to take corrective action. Sound familiar? This is not anybody's fault. It's just how things are, like gravity or fog. Government departments can't develop systems well because of the sort of organisations they are. But that's OK. We don't want to live in Stalin's Russia or even in Singapore. We've got the democracy we have because, although it's awful, everything else that people have tried has been worse. Ross From Aaron Sloman Tue Aug 29 22:52:20 BST 2006 To: cphc-members Subject: Re: Isoft fiasco Responding to: Ross Anderson Thanks Ross, (various bits of your message snipped): > I've wondered for many years why so many government IT projects fail, > and I'm not sure complexity arguments suffice. I am sure you are right that there are different factors. Similar points to yours about how governments and their employees work were made by some of the commercial/industrial people on the local linux users group mailing list, on which I invited comment. I suspect, however, that even if all the factors you mentioned were somehow fixed there would still mostly be disastrous failures in decade-long multi billion £/$ IT projects, whether paid for by public or by private funds -- for the reasons that I gave, namely reasons concerned with the inherent complexity and unpredictability of the problem-solving/design/development processes, and the impossibility of producing in advance sets of *requirements* that will be seen to be good ones at the end of the project -- partly because of the new insights that will have been gained from the project, whether successful or not, and because of other changes in technology, user knowledge and expectancies, etc. > They explain some of > the failures, but not why the public sector is so manifestly worse > than the private sector. One factor may be the relative infrequency of such large long term expensive IT projects in the private sector? > I believe you have to consider factors from organisational theory and > public-choice economics. Civil service departments behave the way they > do because of the incentives facing civil servants and ministers. > There is a vast research literature on this; entry points include ... Thanks for all the pointers. > Dodging > liability, and dumping risk, are deeply entrenched. Yes: I see that in decisions taken at university level by administrators and central computing system managers, who take decisions where the needs of teaching and research are given lower priority than avoiding some remote legal risk, for which they might be blamed. ('Covering backs'.) > Ministers often > optimise their career chances by acting as advocates for their > department rather than for the public that elected them. This is all > well-explored by many shelf-feet of books on politics and economics. What I've seen in recent years suggests that an additional factor can operate: blinkered fanaticism of ministers (and prime ministers) who see themselves as the saviours of the nation (or something bigger), and think they really do know the truth whereas the voters, judges, commentators, house of lords, etc. have all got it wrong, but will eventually see the light... etc. > There's other stuff on the systems side. Why is it, for example, that > everything from Windows to mobile phones ends up with too many > features, at the cost of both usability and dependability? Well, > features get added until the marginal cost equals the marginal > benefit. The benefit of each feature accrues to a small vocal group of > customers; the cost is a slightly increased probability of a blue > screen of death or whatever, spread across the whole user base. The > result is a 'tragedy of the commons' in which every programmable > device at equilibrium has so many features that the median user is > just about to throw it at the wall out of frustration. Dependability > is very far from the social optimum: the time spent each year by > people waiting for Windows to reboot may exceed Microsoft's market > cap. I would not disagree with your general point, and I don't often defend Microsoft, but having seen and cursed windows 98 and after that windows 2000 over the years, running on my wife's machine (because there's a map-making program she can't do without which requires windows), I noticed a huge change for the better when she got a new machine running XP. Of course there may be some disaster in the offing as she has only had it for a few months. (I still wouldn't use Windows myself as it's far too rigidly constrained for me, and I prefer other ways of dealing with newly detected deficiences, risks, etc. than MS update.) > For more, see for example: > > http://www.cl.cam.ac.uk/~rja14/econsec.html > > How do we combine the insights from political economy and the analysis > of system dependability? If my argument is right it cannot be done in *any* 10 year monolithic project whose goals and even approximate costs and timetable are fixed in advance. >... > It would be nice to have the time to develop this further - there are > no doubt dozens of other factors. But I'm sure the core of it is this: > the way ministers work is exactly at the other end of the spectrum > from how a good project manager works. But no project manager, however good, could overcome the problems I listed, especially if some of them can only be solved by switching development teams and companies (and project managers?) in unpredictable ways -- to take advantage of some newly developed expertise. > If you're building a system to a fixed-price contract, you race to > close down options as fast as you can, to remove uncertainty, to get > the client to make the hard decisions about design tradeoffs and sell > them to his colleagues, to agree not to tinker with anything until you > deliver the software, and to take ownership of the system as you build > it. All of these are at odds with the incentives facing an ambitious > minister. > > Now, the dependability literature teaches us that large software > project failures are mostly due to overambitious, vague and / or > changing specifications, coupled with poor communications and an > inability to acknowlegde the signs of failure early enough to take > corrective action. Sound familiar? If my arguments are right, among large, long term IT projects, only trivial, non-innovative ones can possibly succeed, and even most of those will fail simply because they produce something as out of date as their original requirements specification. > This is not anybody's fault. It's just how things are, like gravity or > fog. Government departments can't develop systems well because of the > sort of organisations they are. But that's OK. We don't want to live > in Stalin's Russia or even in Singapore. We've got the democracy we > have because, although it's awful, everything else that people have > tried has been worse. I don't know of any government that has tried the approach I sketched (which Colin pointed out is very Popperian), which treats every large long term project as inevitably being not just a design and implementation project, with pre-specified requirements, but also a major national, or possibly international, *learning* process, in the course of which requirements, knowledge, technologies, possibilities, opportunities and constraints will all change in unpredictable ways, alongside whatever designs and implementations are produced, and will change partly *because* of that production process. However, you may be right that the institutional and cultural pressures in governments and the civil service will prevent the benefits of such an alternative approach being achieved. Since the recommended approach allows for a substantial subset of projects to be axed relatively early, while the knowledge gained remains publicly available, it may be that (a) the total amount of wasted money will be much smaller (b) and all those failures may nevertheless indirectly contribute to other successes, in the way that the internet grew out of learning processes and many failures. Aaron http://www.cs.bham.ac.uk/~axs/ From Aaron Sloman Tue Aug 29 23:02:50 BST 2006 To: cphc-members Subject: Re: Isoft fiasco [PS to Response to Ross] A thought has just struck me. Perhaps we could reverse the old idea of requiring an *industrial* uncle for academic research projects, and require an *academic* nephew/niece for every government-funded project placed with a private company? The main task of the academic would be to analyse and report publicly on what was learnt through that expenditure, in addition to contributing in other ways, if appropriate. He she would have to be paid *entirely* from public funds and debarred from owning shares in or receiving any financial benefits from the company for several years! Of course, there are messy details that could undermine this. Something similar was suggested by someone on the SB linux users list commenting on my letter. He also wrote > Private Eye No.1164 17th August page 6 has some serious comment > on this and other wastes of taxpayers money. I have not seen that. Aaron http://www.cs.bham.ac.uk/~axs/ From owner-cphc-members Wed Aug 30 17:14:19 2006 Date: Tue, 29 Aug 2006 18:21:34 +0100 From: "Michaelson, Greg" Subject: Re: Isoft fiasco Comments: To: Ross Anderson Of course we don't know about failures of similar sized projects in theprivate sector because they're not subject to the same degree of publicscrutiny. So we don't know whether it's the project size or the publicsectorness that is significant. And given the shared Tory & Labourobsession with outsourcing we don't know whether in-house groups woulddo any better or worse than private consultants. I suspect that theprofit motive is a major factor in out-source failure as it forcescontract holders to minimise their own costs by cutting corners in orderto keep their shareholder sweet. Best wishes Greg ________________________________ From owner-cphc-members Tue Aug 29 21:23:20 2006 Date: Tue, 29 Aug 2006 20:34:57 +0100 From: Yorick Wilks Subject: Re: Isoft fiasco Comments: To: "Michaelson, Greg" Im sure Greg has put his finger on it: the failure of companies driven by a profit motive with shareholders (Google, Microsoft etc ) compared the successes of all those good old nationalised corporations without either. Are we the old two left still mourning the loss of Post Office Telephones and the 6 month wait for a home line? Yorick Wilks On 29 Aug 2006, at 18:21, Michaelson, Greg wrote: > Of course we don't know about failures of similar sized projects in > the private sector because they're not subject to the same degree of > public scrutiny. So we don't know whether it's the project size or the > public sectorness that is significant. And given the shared Tory & > Labour obsession with outsourcing we don't know whether in-house > groups would do any better or worse than private consultants. I > suspect that the profit motive is a major factor in out-sourcefailure > as it forces contract holders to minimise their own costs by cutting > corners in order to keep their shareholder sweet. > > Best wishes > > Greg > From G.Michaelson Wed Aug 30 09:25:17 2006 Subject: RE: Isoft fiasco Date: Wed, 30 Aug 2006 09:23:29 +0100 From: "Michaelson, Greg " To: "Aaron Sloman" Aaron My email was exactly as quoted by Yorick. Best wishes Greg -----Original Message----- From: Aaron Sloman [mailto:A.Sloman@cs.bham.ac.uk] Sent: 29 August 2006 21:55 To: Michaelson, Greg Subject: Re: Isoft fiasco Hi Greg, I have seen your message as quoted by Yorick, but for some reason did not get the original. Is the bit quoted by Yorick complete? > Of course we don't know about failures of similar sized projects in > the private sector because they're not subject to the same degree of > public scrutiny. So we don't know whether it's the project size or the > public sectorness that is significant. And given the shared Tory & > Labour obsession with outsourcing we don't know whether in-house > groups would do any better or worse than private consultants. I > suspect that the profit motive is a major factorin out-sourcefailure > as it forces contract holders to minimise their own costs by cutting > corners in order to keep their shareholder sweet. > > Best wishes > > Greg If not could you re-send me the whole message please? It's possible that it's temporarily jammed somewhere on Janet and will find its own way here later, but I thought I should make sure. Incidentally, there are some known commercial failures, though not of the same size. One example was the GEC-63 computer in the 1980s. Thanks. Aaron From owner-cphc-members Wed Aug 30 06:21:20 2006 Date: Wed, 30 Aug 2006 04:22:00 +0100 From: Jon Crowcroft Subject: Re: Isoft fiasco Comments: To: "Michaelson, Greg" it is worth remembering that two major software houses with track record in very large scale systems integration did NOT bid on this project (IBM Is the obvious one) - the contract was placed in such a way as to put them off (from what I;ve learned talking to people there) - microsoft didnt bid even though some aspects of the GP and ward systems would have been something they could have done plausbly (though the image archive/exchange system might have been beyond them since data bases aint there thing) unfortunately, from talking to medics on the frontline in testing these systems, I have to say that some of the failures start to smell of something worse than simple large systems project failures through incompetence, and look more like bordering on actual dodgy behaviour (for example, legacy databases on immunisation were discontinued before accepteance tests for replacements had been passed, despite complaints - amidst finger-pointing amongst the various sub contractors - I doubt anyone will get done for criminal incompetenece (leading, possibly to unnecessary deaths) but they ought to be... I think a more constructive thing for us to do would be to point (as some of Aaoron's letter does) at ways not only to do this right, but to repair the programme of work that is, at least, partly getting there - federation was used in the Welsh system, as well as some smart social engineering using incentives for local services to use their national databases (as disaster recovery backup for example) - I never understood why the UK system was divided into such large pieces (and why it wasnt grown out of existing trusts' bottom up attempts to build IT for their own management) there are other pieces that may never work (e.g. proper privacy with the way the cradle to grave paitent record is envisaged) but the components like the x ray and other image stuff, and the prescription system ought to be seperable tractable projects even at the scale pictured (actually, i dont see why they are a whole lot harder than the dvla and passport systems, which are (after some teething problems) ok, let alone the inland revenue which is slowly getting there with online tax returns etc etc of course the merger between inland revenue and customs and excise has yet to play out, and should provide some "amusement"... I'm not quite sure why databases with 60M objects keyed on one value, with a few kbytes entry per person should be considerd dificult in these days where laptops ship with 100GByte drives, but there you go... yes, private projects fail too - lots. and they dont have to be IT based, just complex....anything (as ross so rightly says) with a lot of humans stakes held is subject to all sorts of gremlins and pookahs... cheers jon From rjp Wed Aug 30 09:22:38 2006 Date: Wed, 30 Aug 2006 09:22:22 +0100 From: Rob Pooley Subject: Re: Isoft fiasco [PS to Response to Ross] Absolutely, Aaron. I have touted the idea for several years that the best way to support software engineering research, which always has a problem with being able to observe and experiment on realistic systems, would be exactly the requirement for all large public sector projects to award funding to academics for this purpose. It is, in effect, how some research gets grafted on to EU projects. Rob From owner-cphc-members Wed Aug 30 10:18:26 2006 Date: Wed, 30 Aug 2006 10:16:46 +0100 From: Steve Linton Subject: Re: Isoft fiasco Many excellent points from Ross. I especially like: > But that's OK. We don't want to live > in Stalin's Russia or even in Singapore. We've got the democracy we > have because, although it's awful, everything else that people have > tried has been worse. > Interestingly, from what I have picked up from a superficial reading the NHS IT stuff has, by government standards, been put together relatively well. There is a publically visible non-political long term project manager, whose personal utility is fairly directly aligned with the success of the project. He has let the primary contracts to a couple of large, well-financed world-class integrators (IBM, Accenture and CSC are in there, I forget the details) who get to make most of the medium-level requirements decisions, carry most of the financial risk and get to structure the contracts with the actual software providers. More or less equivalent software is (or was) being developed in parallel by two companies (iSoft and someone else) for these contractors, so if one developer goes belly-up they can just buy the other package, and it's STILL in danger of falling apart (essentially because both software developers may fail). Steve -- Steve Linton School of Computer Science & Centre for Interdisciplinary Research in Computational Algebra University of St Andrews Tel +44 (1334) 463269 http://www.dcs.st-and.ac.uk/~sal Fax +44 (1334) 463278 From owner-cphc-members Wed Aug 30 12:12:06 2006 Date: Wed, 30 Aug 2006 11:00:19 +0100 From: Ian Marshall Subject: Re: Isoft fiasco Aaron I am firmly convinced that almost none of the issues are unique to IT However, there is one significant issue that does appear to be software specific - it is the widespread belief of people who do not develop software, that software can be changed at no significant cost. One easy way forward is to remind politicians and managers that any creation requiring human effort, including programming, will require additional human effort (on a scale at least comparable to what was required to create it in the first place) to make significant changes. We do not expect plasterers to work for free if we change the design for the mouldings after they have made them, and nobody should expect software builders to be any different. Poor estimation is the characteristic of all artisan based activity (think builders, plasterers etc) and is more associated with poor understanding of hidden risks than with size or complexity. Greater use in software projects of estimation and risk techniques widely used in the construction industry would doubtless be useful. Your recommendation to build understanding by use of small scale solutions that get incrementally improved and integrated is good. There still needs to be an occasional large scale refactoring (analagous with IPv6?), but this should only happen in the light of operational experience with a more cobbled together solution, and with the involvement of the end users. IMHO too much of what we build (as a society) is designed by "experts" who are divorced from the reality of the environment their creations will inhabit. The result is products that do not take account of place (geography), culture, human occupants/cohabitants/users, etc when designing solutions. As this "brutalism" is a generic problem (affecting much more than software) we will not fix it by offering solutions to IT issues alone. I personally think planners and beureaucrats have promoted brutalism because it is easier than the (usually) messy business of consulting the people who will have to live with the end reults. As people are starting to realise easier does not necessarily mean cheaper. One approach (an effective use of academic expertise) to enlightening the decision makers further might be an authoritative quantitative study on whole life costs comparing products created with a user centric approach to products created with a brutalist approach. Actually this seems so obvious it must exist already (but possibly not for software). Does anyone have pointers to studies of this kind, that could be precised for governmental audiences? Regards Ian From owner-cphc-members Wed Aug 30 13:58:56 2006 Date: Wed, 30 Aug 2006 13:57:45 +0100 From: "Bernard Cohen" Subject: Re: Isoft fiasco Responding to Ian Marshall's message. Nobody in this discussion group seems to have noticed that the issues being raised are precisely those associated with 'systems of systems', particularly in the miltary and aerospace fields. I realise that drawing a comparison between military and healthcare demands may be anathema to some, but I suggest that texts such as D. C. Alberts' 'Power to the Edge' describe dilemmas faced by both military and healthcare enterprises, and by their suppliers. In both, there is a need to design both products and organisations so that the latter can 'orchestrate' the disparate services provided by the former, in response not only to fantasies elaborated by those at the 'centre', or 'top' (politicians, staff officers heathcare administrators, etc.), but to unanticipatable demand situations encountered by embodied individuals at the 'edge' (soldiers and clinicians as well as non-combatants and patients). Systems of systems do not admit analysis using classical systems theoretic (nor, therefore, computer science) models (see, e.g., Doug Norman of Mitre Corp). They need a framework in which the pragmatics of individuals with different desires and the semantics of systems with different capabilities may be expressed, composed and critically analysed so as to reveal the 'holes' that they will have to deal with if their enterprise is to be sustainable. See www.brl.com for a thorough discussion of these issues and proposals for their resolution. Prof Bernard Cohen, Dept of Comp Sc, City Univ, Northampton Sq. London EC1V 0HB tel: ++44-20-7040-8448 fax: ++44-20-7040-8587 WWW: http://www.soi.city.ac.uk/~bernie "Patterns lively of the things rehearsed" From George Coulouris Wed Aug 30 14:06:36 2006 Subject: Re: Isoft fiasco Date: Tue, 29 Aug 2006 19:59:37 +0100 Aaron, I ought to have said in my previous message that I agree with almost every word of your analysis and wish I had thought of it myself, it's been too easy to just sneer at the many government project failures without thinking the problem through as you have now done. Here are a few further thoughts: Some very large scale software developments in industry have succeeded (banks, airlines, ...). The ratio of successes:failures is much higher for large industrial projects. These successes are part of the reason for the politician's misplaced confidence in the software industry. The reason for them ought to be explained. [I wrote that before seeing Ross's comment along similar lines.] Your reference to the history of the internet is excellent, but it almost makes it sound like an isolated case, whereas we know that without ARPA + industry -funded open research and development, other key developments such as timesharing, interactive graphics, distributed systems and personal computing would have been stunted or drastically delayed. Of course there were plenty of ARPA-funded projects that never saw the light of day. And such examples don't address a key difference between the US and UK. In the US the electorate has enough faith in technology to allow large amounts of taxpayers money to be spent on advancing it without clear applications. My intuition is that you're right about the need for incremental and open development, but more thinking needs to go into the process by which it can be structured to satisfy guardians of the public purse. The term 'deliverables' comes to mind, but we all know how wrong- minded that is. Working, useful systems at each stage in the process, even if they don't address a very ambitious set of requirements seems more like it. But then how would big leaps ever be taken? Maybe only in research projects. Best wishes, George George Coulouris Visiting Professor in Residence Computer Laboratory Cambridge University From owner-cphc-members Wed Aug 30 15:32:55 2006 Date: Wed, 30 Aug 2006 14:36:59 +0100 From: Ross Anderson Subject: Re: Isoft fiasco Aaron I don't think it would ever occur to a private-sector company to do something like NPfIT. I have been involved in private death-march projects too, the worst being a 200-person, 3-year project to move a bank's core systems from ICL to IBM mainframes which ended up taking 400 people 5 years and helped cost the bank its independence. That's about as big, and as bad, as it gets in business. In man-year terms XP was slightly bigger, and Vista bigger still, but the user lock-in gives Bill some assurance of commercial success, even if the product ships a year late. Also, when Microsoft ships a new product it's the result of an evolutionary process which gives them a huge amount of specification and masses of test cases from day 1. NPfIT is one or two orders of magnitude bigger; the only comparably sized private-sector engineering project of which I'm aware was when Boeing gambled their company on the development of the 747. Given that they were already making the 707, 727 and 737, that was a much more tractable problem. They knew how to make fuselages, wings, engines, hydraulics and so so; they had test pilots and airworthiness certification procedures. It was evolution not revolution. NPfIT has many other problems. It's not driven by the users but by civil servants, who fear doctors and envy the systems they have built for themselves - all of which are to be ripped and replaced. The complexity of these systems was not understood, and the users' support (or even acquiescence) has not been won. NPfIT is predicated on the idea (and driven by the desire) that all medical records should be gathered in from the multitude of GP PCs, hospital servers and filing cabinets where they currently lie, and made accessible to the centre for a multitude of secondary purposes (Sir H says 'research' but he means audit, cost control and various kinds of surveillance). This is rightly resisted by doctors because of patient confidentiality and professional autonomy. I did a survey for the National Audit Office which compared the health IT spend in the UK with that in the USA and a number of European countries: http://www.cl.cam.ac.uk/~rja14/Papers/nao-report-final.doc which showed that the UK is a real outlier. Other developed countries are building federated systems: the centre limits itself to developing and promulgating interoperability standards, which then get built into local systems as they're upgraded or replaced. Over time, things like electronic prescribing start to work over more and more of the country - without huge disruption, expenditure and risk. And given that the UK is 5% of GWP (and heading towards 1% as India and China develop) there is no choice but to use the market leading systems. The UK can't deevelop the best systems for cardiologists, diabetologists, and all the other specialities. We will end up using international standard software just as the NHS already uses Microsoft Word. We will thus get health services that use standard, federated systems, like it or not. NPfIT will thus have the same sort of effect as the attempt by BT and Racal ten years ago to get doctors to use X.400 rather than SMTP: it held up the adoption of email in hospitals by five years. For more in this vein see http://www.theregister.co.uk/2006/06/13/letters_1306/ http://politics.guardian.co.uk/publicservices/story/0,,1860168,00.html In short, the NHS is building the wrong system for the wrong reasons and it deserves to fail. Ross From owner-cphc-members UK Wed Aug 30 17:40:19 2006 Date: Wed, 30 Aug 2006 10:13:56 +0100 From: "Michaelson, Greg" Subject: Re: Isoft fiasco In a similar vein, we still don't know what effect if any the Millenium Bug had in the private sector, again because of secrecy. It's curious that we don't demand the same degree of transparency from the private sector as from the public sector, even though they are now easily the largest provider of services and beneficiaries of public expenditure. In particular, no one seems to point fingers at the makers of these truly awful one-size-fits-all systems. Instead they just get given more time and money, and the public sector victims of PFI get the blame. For example, why are the endemic inadequacies of University finance and student record systems always the fault of poor old Computer/Corporate Information Services rather than of the software suppliers? Like Yorick, I miss decent nationalised industries. Yes they were wasteful, but that's probably the price of mass provision at a uniform standard for all. I'm just back from a long weekend on the Atlantic fringes of the South West Highlands: terrible mobile phone coverage but lots of red telephone boxes. One of the reasons nationalised industries gave better service was because they did things in house. My first employers, the old Scottish Gas, had fleets of engineers in vans who seemed happy to stay until a problem was fixed. Now you get a time slot and an arbitrary contractor of highly variable competence, and when you complain to the energy provider they disclaim responsibility. Again, many Universities, having privatised their cleaning, have gone back to in-house provision because it's cheaper, more flexible and gives better service. Rant, wheeze... Greg PS privitisation = turning into a hedge fund From Jim Hunter Mon Sep 04 13:38:05 2006 Date: Mon, 04 Sep 2006 13:27:24 +0100 To: Aaron Sloman Subject: Re: Isoft fiasco Aaron Good on you! - interesting to see the others' comments. This may be anecdotal, but I've been told that the management of IT change in the NHS here in Scotland (which of courses is devolved) is much more incremental (and hence more successful). This was from an IT manager in NHS Grampian. Jim Prof. Jim Hunter Department of Computing Science University of Aberdeen King's College ABERDEEN AB24 3UE UNITED KINGDOM http://www.csd.abdn.ac.uk/~jhunter/
My own contributions are licensed under a
Creative Commons Attribution 2.5
If you use or comment on my ideas please include a URL if possible, so that readers can see the original (or the latest version thereof).
School of Computer Science
The University of Birmingham