healthcare.gov
Estimating The Magnitude of Future Problems With healthcare.gov
Last week I ran across a post by Paul Hammant’s blog, Testability and Cost of Change, that reminded me of another software engineering author, Barry Boehm, from my time. Paul included a quote which is the primary reason I believe the www.healthcare.gov web site will continue to have problems in 2014.
“Bugs are cheaper to find/resolve in a design stage, which is cheaper than finding them in development, which is cheaper than finding them in QA, which is cheaper than finding them in production”
Paul had a nice diagram of this relationship which I have reproduced here.
While looking up information on Barry Boehm’s writings on software engineering I ran across a slide from Stevens Institute of Technology that listed some of the reasons software project fail. Not surprisingly many of these reasons are being used to describe the development and management of the www.healthcare.gov site.
| Lack of User Input | 12.80% |
| Incomplete Requirements and Specs. | 12.30% |
| Changing Requirements and Specs. | 11.80% |
| Lack of Executive Support | 7.50% |
| Technology Incompetence | 7.00% |
| Lack of Resources | 6.40% |
| Unrealistic Expectations | 5.90% |
| Unclear Objectives | 5.30% |
| Unrealistic Time Frames | 4.30% |
| New Technology | 3.70% |
| Other(Poor Management Dominates) | 23.00% |
So how far are we away from a failed software project? Although I applaud Mr. Zients efforts to fix the front end bugs affecting the web site by November 30th , I think those of us who like to see the system fixed so we can move on to the unfinished business of health care reform are likely to be disappointed. Considering the multitude of security and back end problems along with admission that 40% of the system is not developed yet, it is easy to envision the public’s support for exchanges and the Affordable Care Act will wane further in 2014. The front end problems were not the most critical part of the system that needed fixing. They were the easiest to fix. At some point in 2014 we will look around and realize that when the average person believes the project has failed and they are not going to change their mind. Then the project has officially failed and the only question is when does the media recognizes the project has failed. So where does health care reform go if we have a failed web site? This is not the end of the world but it does make the remaining Affordable Care Act ideas that have not already failed much harder to implement. If you agree with me that the Affordable Care Act was primarily a political rather than a health care reform achievement then it is not unreasonable to argue that the web site has delayed health care reform for several years and that less partisan, incremental changes would have been more successful. Maybe this angst that we still have not done anything to rein in health care costs after four more years will bring clearer heads to the table. It is the elephant in the room. This partisan ring to rule them all has to go back to the mountain which bore it if we want “real” health care reform. A non partisan approach modeled after the procedures used to do identify which military bases to close is probably the best course of action. Although the Affordable Care Act supporters hate to admit this, the old system even with its problems continues to look much better than the new system with all of its problems. The biggest problem between the old system and the new system is that most of the designated “losers” in the new version have been arbitrarily chosen. This is never good politics and for some reason they continue to insist on saying that the Republican party is the stupid party. I wish! The new and improved version of the Affordable Care Act needs to have a whole lot less of these arbitrary “losers”. If you are designated as a Affordable Care Act “loser” then it should be something both parties can agree upon. Hopefully we can finally fulfill one of the President’s talking points and actually show people with “substandard” plans that there is a better and lower cost health care plan on the exchange. Picking arbitrary people as “losers” to fulfill your political ambitions for wealth distribution does not cut it!
RE: Bad Managers Ruined Obamacare
I guess it is no surprise that I agree with Glenn Reynolds of Instapundit fame who wrote in his USA Today column that Bad Managers Ruined Obamacare. Engineers respect the consequences that the Mythical Man Month is probably a fact of life you cannot skirt around. Politicians ignore it at their own peril.
All appearances to the contrary, the managers involved in this debacle aren’t dumb. But they come from a background — law and politics — where arguments often take the place of reality, and plausibility can be as good as, or better than, truth.
What engineers know that lawyers and politicians often don’t is that in the world of things, as opposed to people, there’s no escaping the sharp teeth of reality. But in law, and especially politics, inconvenient facts are merely inconvenient, something to be rationalized away.
When our country has accomplished great things in the past, there has usually been a great engineer running the program: Hyman Rickover with the nuclear submarine program, or Wernher von Braun with the Apollo space program, for example. Rickover and von Braun were famously stern taskmasters, but they did not substitute wishes for reality.
Which may be why they were able to launch submarines, and rockets that astounded the world. While today, we can’t even launch a website.
TheHealthSherpa and the HMO Debate
Kudos to a couple of programmers who decided to create a demonstration site of the shopping functions that should have been included in www.healthcare.gov. I have a minor quibble with the article since I believe that this demonstrates management failure rather than an architecture failure since the shopping function could have been completed by an independent team. It is the old project management tactic, divide and conqueror.
Three San Francisco programmers in their 20s proved just how inept HealthCare.gov’s architects were, taking just a few nights to design a simplified version of the glitch-ridden contraption.
Michael Wasser, Ning Liang and George Kalogeropoulos, who share office space with other Bay-area techies, built an alternative website,TheHealthSherpa.com, that resolves HealthCare.gov users’ main complaint ”“ easy access to plan descriptions, according to CBS News. And it didn’t take them $600 million to do it.
“They got it completely backwards in terms of what people want up front,” Liang said. “They want prices and benefits, so that they could make the decision.”
Unfortunately as I research the health insurance issue further I am still struggling to find the details on the plans. TheHealthSherpa.com does not help in this area. This weekend I did some research into copays after some comments by Juan Williams last week and into the decision health insurance customers will have to face between HMO and PPO insurance plans. Copays are unlikely to save me any money but customers who have a lot of doctor’s appointments and drugs might find a benefit depending on their premiums. Depending on your point of view the HMO style of health care was made famous or infamous in the 1990’s HMO debate. This sounds like déjà vu all other again. The HMO option is the lowest cost plan for me and the lowest cost PPO plan is more expensive even when I include the subsidy than my current plan. I am tentatively planning on writing a future post that explores the value of the Affordable Care Act plans compared to my current insurance. I will have to remind myself to make sure I am comparing PPO plans or I will be comparing apples to oranges.
Things that make me go hmm… Up to 40 percent of the technology needed to run the new Obamacare health insurance marketplace has not yet been built
Since I previously voiced my skepticism of management’s competence in administering the www.healthcare.gov project, I am not be surprised that there was an assortment of uncompleted work. However the magnitude of Mr. Chao’s claim astounds me. Here is his quote:
Up to 40 percent of the technology needed to run the new Obamacare health insurance marketplace has not yet been built and will not be ready when insurance companies start sending in bills when coverage begins January 1, the project manager of HealthCare.gov told the U.S. Congress on Tuesday.
Uh, what was the management team doing over the last three years when the developers were falling behind and why is this management team suddenly so confident that they can build and test these missing business functions in sixty days? The missing business functions sound like management reports that would have been essential to managing a successful implementation and integration of the insurance marketplace and to answer all of those pesky questions from Congress and the media!
Things that make me go hmm… Fixing www.healthcare.gov by the End of November
I have been critical of www.healthcare.gov since I first used the web site in 2010. It sure looked like I was the only person who had tried to use it and I can say that it did not improve with age. Two weeks ago I had to chuckle when Mr. Zients announced that “by the end of November, HealthCare.gov will work smoothly for the vast majority of users.” Since his claim violates much of my personal experience and knowledge of software design and development, I was skeptical that they could fix www.healthcare.gov by November 30th. When a project like www.healthcare.gov makes the transition from development mode to maintenance mode, fixing problems on the fly while customers are using the system is difficult and risky. I am not saying the healthcare.gov folks cannot do this but experienced IT guys will remind anyone still listening that they have been there, done that, and have the cuts and bruises to show for it. I hate to say I told you so but CBS is reporting in “Memo warned of "limitless" security risks for HealthCare.gov” that:
Chao said he was unaware of a Sept. 3 government memo written by another senior official at CMS. It found two high-risk issues, which are redacted for security reasons. The memo said "the threat and risk potential (to the system) is limitless." The memo shows CMS gave deadlines of mid-2014 and early 2015 to address them.
I can think of only one reason that Mr. Chao was unaware of the memo, somebody did not want Mr. Chao to know the extent and gravity of the problems. Management failure is a dish best served cold. Good luck, Mr. Zients!
Mr. Zients Versus The Mythical Man Month
Last week I had to chuckle when Mr. Zients announced that “by the end of November, HealthCare.gov will work smoothly for the vast majority of users.” I am one of the few long time www.healthcare.gov visitors and have been anxiously looking forward to improvements since 2010 when I first complained the insurance finder was useless. Although I admire his chutzpah the two things I can say for sure is that there will be a touchdown dance on November 30th and there will still be a lot of serious problems to fix. The touchdown dance is the easy part of his task. Unfortunately the American people are married to this software. Like a bad Las Vegas wedding in which we hate to admit our mistake, we will trudge onward for the sake of the children.
The first problem facing Mr. Zients is that he is up against the software engineering and project expertise of Fred Brooks, whose central theme in his book, “Mythical Man Month”, is that “adding manpower to a late software project makes it later” has been ignored by the administration. They have already announced their plan to hire QSSI to come in and fix the problems with the web site in 30 days. Adding more people and thinking this will fix the problem is a big problem. Saying that it has to be done in 30 days has me in alternating fits of laughing and crying. As a person who has made his living fixing “other people’s code” for thirty years, this solution is a recipe for disaster and no seems to be listening. So let me frame the problems facing this system with a diagram from the book, Mythical Man Month.
Using the analogy from the book software products start out in the “Program” quadrant and are transformed via generalization, testing, documentation, maintenance, and system integration into a “Programming System Product”. The “Programming System Product” in our case is www.healthcare.gov and the final acceptance test is whether the American people can use it to purchase subsidized insurance. In 1974 Mr. Brooks asserted that a “Programming System Product” costs nine times as much as the “Program” so the vast majority of the cost and effort is spent generalizing, testing, documenting, and integrating the interfaces. Unfortunately for Mr. Zients this part of software engineering has not changed over the years.
From the reports I have read there has been very little testing and the specifications for the programming interfaces did not go out until eight days before the launch. It looks like most of the money and effort was spent in the “Program” quadrant and very little was spent in the areas that would actually result in a successful “Program System Product”. This reeks of management failure. As part of 1% who successfully got through the application process far enough to download a copy of my potential insurance plans I can say that the site has a lot of serious problems. It brings a whole new meaning to the term, “bad beta site”. Although I have no doubt that this new contractor, QSSI, can clean up the code discussed in this Reddit thread, the other problems that have been reported are more daunting and time consuming. Here is a short list of problems in no particular order.
- The usability problems pointed out by the NN group
- The back end problems pointed out by Dan on marginal revolution.
- The 834 problems pointed out by Sarah Kliff on the Wonkblog
- Identity theft problems pointed out at MotherJones.
I think both the Affordable Care supporters and detractors agree that despite the fact that the web site is a clusterfark of monumental proportions, it will get fixed eventually. The question is whether it will be sufficiently complete and secure in time. Since they ignored my old web development adage, “copy the best and ignore the rest”, maybe they should start looking at an exit plan that involves joining forces with the “best in the business”. There is still time for letting www.eHealthinsurance.com and its six competitors finish a smaller, less politicized version of the the job and minimize the impact of a failed www.healthcare.gov.
Cross posted at wehuberconsultingllc.com
I Finally Figured Out Why The Health Insurance Rates Increased So Much in 30 Days
CBS News has an article, HealthCare.gov pricing feature can be off the mark, which is a good explanation of the price increases I wrote about in the post, The Affordable Care Act ”” The Fix Is In. The rates on Healthcare.gov’s “window shopping” page and the downloadable spreadsheet were using the rates for a 40 year old couple and that price is dramatically different than the rate for a 59 year old couple. So if we look up the 2nd lowest silver plan for 2014 on www.ehealthinsurance.com we find that the monthly cost for a 40 year old couple is $632 compared to the September 24th estimate of $515. This is a more “palatable” 23% increase. I still do not understand how some people are reporting that the rates are going down in Ohio. Here are some of the 2013 plans for a 40 year old couple available at www.ehealthinsurance.com.
For those who are curious the 2nd lowest silver plan monthly cost for a 50 year old couple is $884 in 2014 and $1,288 for a 59 year old couple. For a person who is paying $391 a month for a plan that whose actuarial value is almost a silver plan, I am still trying to deal with a 229% price increase.
The Affordable Care Act — The Fix Is In
**Revision – See I Finally Figured Out Why The Health Insurance Rates Increased So Much in 30 Days for the explanation. In this case I think it is fair to say that the Department of Health and Human Services was incompetent rather than lying or in collusion with insurance companies.
For some time I wondered why the Kaiser Subsidy Estimator said I was not due a subsidy. The estimator said I should be able to find a plan for less than $4,750 a year but when I priced plans at www.ehealthinsurance.com it said the lowest bronze plan would cost a little over $12,000. This is not close! Although it is hard to find the web page on www.healthcare.gov, there is a web page that allows you to “window shop” the available insurance plans without logging in. I wrote about my experiences with the prices available in the post, Ohio Health Insurance Rates ”“ Curiouser and Curiouser. Once again I was puzzled why they had such low rates compared to www.ehealthinsurance.com and www.anthem.com. Then I saw an interesting report, Enrollment in Obamacare Exchanges: How Will Your Health Insurance Fare?, that said that the health care insurance for Ohio are going down compared to last year. If www.ehealthinsurance.com and www.anthem.com are correct then where were these people getting this erroneous data? Last week I solved the problem.
The source of the error was the Department of Health and Human Services. On September 24th, 2013, HHS released a table that “lists all health plans available in states where the federal government is operating the Marketplace”. So I downloaded the spreadsheet and looked up my data. It matched what the “window shopping” page said. For kicks I copied the prices from www.ehealthinsurance.com over to the spreadsheet and calculated how much the plans had increased in less than 30 days. Here is my report. It was 111%. When you round to the nearest integer every Anthem plan had gone up exactly 111%. Despite all of the differences between the plans the price increase was 111%. Hmm… that’s odd!
So I logged into www.healthcare.gov and copied the available policies for my family into the spreadsheet and calculated the price increase. Yeah, I got in. It helps if you have 30 years experience in IT. The first thing I noticed was that the price increase was 104% across twenty plans from five insurance companies. Wow, what a coincidence!
So what went wrong with insurance rates in Ohio in less than 30 days? I realize that climate scientists have set a pretty low bar for scientists but 104% in less than 30 days is a large, significant error. Why is the 104% price increase the magical number that all of the insurance companies agreed upon? Even though I am a skeptic I was at least hopeful that the Affordable Care Act would result in a more honest, bottom up pricing mechanism that would have resulted in some variation around 104%. Instead the price increase appears to be a top down decision with a rather large fudge factor added in. This top down decision making is what I see as the primary difference between a high cost state like Massachusetts and the rest of the country. Now we are left to ponder if this price increase is the risk premium that the Affordable Care Act brings to the table or Washington cronyism at its worst? It seems almost too convenient that many perfectly fine health insurance plans are being cancelled only to be replaced by a much more expensive plan with a higher deductible. Once again I am stuck with the question,
Was the Department of Health and Human Services incompetent, lying for political reasons, or something else when they released the insurance data one week before the exchanges opened?
Re: Obamacare Cost and Health Insurance Providers for an Individual,59 Years Old, in Batavia, OH
I think www.healthcare.gov has a problem with their rates for Ohio. I continue to think they are showing 2013 rates in their sample data. Here is some summary data from www.legalconsumer.com, Obamacare Cost and Health Insurance Providers for an Individual,59 Years Old, in Batavia, OH, which uses data it got from www.healthcare.gov. You can see that it says the 2nd lowest silver plan costs $515.13. This cost happens to be higher than the cost I guessed at in the post, Ohio Health Insurance Rates ”“ Curiouser and Curiouser. When I go to www.ehealthinsurance.com or www.anthem.com I continue to find their prices for 2014 to be about 100% higher than those posted in the sample data.
Projected Obamacare Rates for an Individual, Age 59, in Batavia, Ohio
- Lowest Catatrophic Plan = $367.15/mo
- Lowest Bronze Plan = $443.13/mo
- Lowest Silver Plan = $485.75/mo
- Second Lowest Silver Plan* = $515.13/mo
- Lowest Gold Plan = $608.16/mo
When I look up the price for the "Anthem Silver DirectAccess w/HSA – cbey" plan on www.healthcare.gov it says it will cost $611.82 for a couple. When I first looked up the price on www.anthem.com it says it cost $1246.26 for a 5000/5000/10%. I checked again today and the price is $1288.32 for a 4000/8000/10%. The closest 2013 plan offered by Anthem to this plan is the Premier Plus 0%(5000/10000/0%) at $704.90 per month which has lower coinsurance, 0% versus 10%. I am guessing but I expect that if the Premier Plus 0% was re-priced for 10% coinsurance the price would be pretty close to the $611.82 shown on www.healthcare.gov.