in review

Okay – oddly – I have this habit of thinking that people deserve an unbiased look at anything that’s not explicitly labeled “opinion” in the news.

If you pick up a newspaper and turn to the Editorials section – you are reading opinions. That’s all they are. Opinions. They may be well-informed opinions; they might match or amplify your opinion. But at the end of the day, you are reading opinion. Likewise, television news used to separate news (the unbiased delivery of information pertaining to recent events of interest to the public) from opinion (what people think about that). It seems that, since the advent of the Internet as the medium we now know, the line between news and opinion has become so thick and so blurry that we can no longer make it out as a line.  It’s just a big gray blob of “news-pinion” out there.

So when the news goes on an on about the new Government Web site – talking ceaselessly about how bad and horrible it is, but providing no real specifics among volumes and hours of commentary – I have no way of knowing how much of that is news, and how much of that is opinion (and whether or not that opinion is well-informed or reasonable). The longer I hear how bad this Web site is without hearing what, specifically, is wrong – just that “it’s bad” – the more I tend to suspect that these reports are mostly opinion, and that most of that opinion is uninformed, or worse, opinion that was not recognized as opinion, being repeated over and over again by people who think it is news.

At the same time, I am a software engineer and an experienced Web developer. I am also an American who is eligible to use the exchange – and I live in a state that elected not to set up its own healthcare exchange. So I’m fairly well-suited to go and look at the Web site myself and develop an opinion based on what I can observe. So… That’s precisely what I did.

Up front – I have no special knowledge of this Web site or its design. I had nothing to do with building it. And what I’m reporting here represents only my own experience using the Web site. At the end I’ll share some opinions about what I was able to observe, but until then – what you see is simple observation.

Background and Test Conditions
I used a computer in my home with moderate specifications to access the Web site. The basic properties of the computer are shown below. It’s a reasonable, nothing-special, mid-range computer purchased a couple of years ago, with no upgrades at all (integrated Intel graphics, no additional RAM or hard drive space, same OS that came with the computer – it is more or less the out-of-the-box computer running the out-of-the box software).

Properties of the computer used to access the Web site

Properties of the computer used to access the Web site

My Internet connection is provided by Time Warner Cable.  I’m using their Turbo Internet service.  According to their speed test app – I’m currently getting about 15 Mbps download speed, with a little over 2 Mbps upload speed. See below.

Internet speed test results (Time Warner Cable)

Internet speed test results (Time Warner Cable)

I typically use Google Chrome for general Web browsing, and decided the most realistic way to determine how the Web site works is to do what anyone else would do – use the browser you always use. Chrome reports its current version is 30.0.1599.101 m and that it is currently up-to-date. I did not go out of my way to update it prior to accessing the Web site; Chrome updates itself (like most modern software).

I accessed the Web site this morning (Sunday, 3 NOV 13), between about 0845 and 0945. The entire session reported here took about an hour (with several interruptions, including a break to heat up a leftover piece of chicken for breakfast, and note-taking as I encountered things I found noteworthy or captured screen shots for later use).

Also, as a matter of conditions – every bit of information I entered into the Web site was real. I used my own information, on the assumption that I would actually buy health insurance through the exchange – thus I entered actual addresses, phone numbers, email accounts, social security numbers, and so on, as I went through the process. Before today I had only ever accessed once (from my mobile phone – very briefly) and I had not created an account or provided any information to anyone. I also did not have anyone with me or otherwise assisting me through the process.

I am providing a list here of observations by exception. I will not list those activities that proceeded exactly as I expected them to proceed. This means that, other than as noted here, the Web site looked, responded, and behaved in ways that are not noteworthy in any way.

General: The only general observation I’ll offer here is that some news accounts have talked extensively about lack of responsiveness, error screens, dead links, and so on. I experienced none of these things. Throughout my interaction with the Web site it remained responsive and did not produce any general error indications.

The following observations seemed noteworthy (per the above description):

Security: Identity verification included information I’d previously entered.
There is a step during the process of signing up for an account that attempts to confirm your identity, apparently using information drawn from other sources. This is similar to the experience you’d have signing up to get access to your credit report online. However, one of the questions used to validate my identity asked me to select the last four digits of my cell phone number. I had just entered my cell phone number on a previous screen. The choices presented did not include the last four digits of my cell phone number (thus I selected the “none of the above” option), and the Web site did confirm my identity. However, this seems like a weird choice, given that a screen just prior to this one had prompted me for the same information, just a moment prior.

Integration: I was asked what type of phone my mobile phone was at least twice, but possibly three times if you count the security question noted above.  I would count this as an artifact of poor integration.

User Experience: I chose to receive notices electronically, and used a checkbox to indicate that I would like to use the email address that I’d previously entered to receive those notices.  The UI forced me to re-enter that email address – even though, by this point in my experience, I had entered, confirmed, and received an email from this system at that address – AND clicked a link from that email to return to the system. This is just a bad choice on the part of the developer. It was a useless thing to do and an annoyance to the user.

User Experience / Security: I indicated that no one was helping me to fill out the forms in the online system. Yet, the system forced me to select a new, fourth security question, after I indicated that no one was helping me right at that moment.  A message on the screen indicated, paraphrasing, that just because no one was helping me now did not mean that no one would be helping me later – so I still had to pick a new security question and answer it just in case someone was going to help me later (so that someone could prove that they were sitting with me or represented me). This is just silly. By this time I’d entered a slew of personal information – things no one but me should be able to know absolutely. So if there is a need for this type of feature, it could be implemented without creating a new piece of information for me to track. Moreover, I have to question how this system is implemented based on this ham-handed approach to security. In principle, if there is someone else who is helping me – that person should have an account of their own in this system. The basic security principles of authentication, authorization, and non-repudiation apply to whoever is helping me, just as much as me. In that case I would expect that the “helper person” has his or her own account; uses his or her own information to log in; conducts all of his or her activities using that account, with all actions taken traceable to that account;p and, that his or her activities on my behalf are authorized by me (since I also have an account) before they become effective.

Inconsistency: The middle name is not shown on the “Who needs coverage” screen.

Poor UI design: There is an optional set of fields toward the bottom of the screen for each family member’s data entry, used to indicate ethnicity / race. The first such field is a radio button indicating whether or not the family member is of Hispanic origin. Once I select Yes or No – I cannot unselect my answer (because of the type of control used and how it operates by default). In principle, since it’s an optional question, it has three choices: Yes, No, and no answer. Once you give an answer – that’s it – you can’t un-give that answer.

User Experience: When entering family members, I was asked what seemed like a slew of unnecessary questions about people’s relationships to one another. I assume this was because my older daughter, still living with us and going to college, is over 18 years old. Even so, the number of questions asked was positively silly. I ended up having to indicate twice what my / my wife’s relationship was to our older daughter, then to our younger daughter; then, I had to answer individually for each person in the household how they are related to each other person in the household.  This was six distinct sets of questions covering four people. It seems that the developers wanted to understand family units within the household, in order to later recommend groups of people to include in insurance coverage. With that said, the whole thing seems pointless. If you need to know this, you could find out much more efficiently; and if the only reason to know this is to form the recommended groups of people, there is a better way to do that (simply let the user form groups out of the people he or she has already identified).

User Experience: The system asks a positively crazy number of optional questions at the end of the application process. These questions all revolve around events that may have occurred in the last 60 days, and include such things like whether or not anyone in your household has moved, gotten married, been adopted (or adopted someone else), had an immigration status change, or been incarcerated. I have no idea what purpose these questions serve, other than to annoy the user. If they’re optional – why would I answer?

Security: Granted, most people in America don’t have a hard electronic token like a CAC to prove their identity. However, at the last stage of the application process you’re asked to sign, attesting under penalty of perjury that the information given is true and correct to the best of your knowledge. There is a text box in which you must type “your electronic signature” affirming this. No guidance is given on the form as to what to do with this box. I typed “I AM ME” – and that, apparently, is my electronic signature. It seems lazy; it seems like the designers simply phoned that in. A better approach might be the one used for the IRS or student aid Web sites, where you self-select a pin in the early part of the process, and enter that pin at the end of the application. Another approach would be to have the user re-enter his or her password. At least one of those approaches would have something to do with who is behind the keyboard as this legal affirmation is given.

Now – at this point – I had submitted my application. I almost immediately got a notice indicating that I, and all members of my family, were eligible for coverage and needed to select insurance through the marketplace (lest I be called a liar – see below).

Eligibility notice

Eligibility notice

Great – right?  Well – here the Web site is again not all that clear.  When I return to the Marketplace I see this screen, after logging in:


Screen shown when I log back in

Screen shown when I log back in

Looks like there is some problem, right? So I click on the Application Details link in the big orange “there is some type of problem” box, and that takes me to this screen:

Application Details screen

Application Details screen

Everything suddenly looks rosey and peachy-keen.  So then I click on “View Eligibility Results” – thinking there must be some problem there (else why would I have gotten that big orange “SOMETHING IS CLEARLY WRONG” message on the last screen) – and I see this:

View Eligibility screen

View Eligibility screen

And if I click the only thing that seems obvious in this screen – RESUME ENROLLMENT – I am back to the first screen – the one with the big orange SOMETHING IS CLEARLY WRONG message. So – these three screens exist in an endless loop. It seems something is wrong – but it’s not wrong. What’s wrong is that you haven’t yet selected a plan. And that is not wrong – that is expected.

The way out of this do-loop is to click the big green “SET” area on the first screen (the one with the big orange obnoxious message). I call this an area because it is not any type of control that I would recognize as a Web developer. It’s not a traditional link. It’s not a button. It’s not anything. When you put your cursor over it, the cursor changes to a text cursor (the one that looks like a capital letter “I” – as though the way you’re supposed to interact with “SET” – whatever it is – is to copy text from it). But, if you click it – you continue the process.  The very next thing that happens is that you get a list of plans available for you and your family in your area.

I stopped short of actually selecting a plan (there are 11 to choose from in South Carolina), but it seems like some other developer took over at this point, and things work okay. I can choose coverage level (Bronze, Silver, Gold, or Platinum – noting that there are no Platinum plans here, at least where I am in South Carolina). I can filter by coverage levels and easily see what is, and is not, covered.

To be fair – the lower-level plans appear to be mostly crap. One in particular was just shy of catastrophic coverage only. They would work, sort of, in combination with a healthcare savings account. Sort of. At least they would prevent bankruptcy due to medical bills, but they seem to have such high deductibles that they wouldn’t be useful for most people. These are plans that cost about half of what my employer-sponsored coverage costs (both my portion and my employer’s portion).

The higher-level plans cost about the same as what my plan at work costs, roughly. They aren’t as good as the plans I have – but they are pretty good, and while I would pay more for those plans in the marketplace, that’s only true because my employer wouldn’t be paying for a portion of the coverage I get anymore. I wouldn’t do it this year – but it would be interesting, in a year or two, to see how many people would elect a plan out of the marketplace voluntarily, if their employers simply paid the employees (directly) what they used to pay to the insurers as their portion of the cost of health insurance.

So – there you have it. Reports of sucking are true in some ways. But, broadly speaking, I was able to get on, create an account, apply, get an answer, and select from available plans (and see what all those plans covered) in just shy of an hour (with breakfast and note-taking / screen-grabbing thrown in). Many of the problems I saw make me question whether there are deeper integration or security problems, but many of them were just annoyances that are properly characterized as superficial.

In short – it does suck, even in some important ways – but, it also does work.

This entry was posted in Uncategorized and tagged . Bookmark the permalink.

5 Responses to in review

  1. rushings says:

    Very thorough review. It is nice to see that it is working for someone. Unfortunately, I think you should have added a couple words to the end of your review: for me. It does not work for me. It does not work for 4 individuals I have tried to help, and who knows how many other Americans.

    I am also an experienced web developer, and from what I have seen from your review and others, I suspect that if I could create a new account and start over, then it would probably work for me. Unfortunately for me, I was trying to enroll in those hectic first few days. I absolutely experienced the crazy error messages and hang ups and resets that the news was talking about. In fact, the only difference between how the website works for me now and how it worked for me then is that it appears that the developers have simply deleted the error messages so they cannot be seen. I still get the hangs and breaks, just no error message.

    I suspect that as I was trying to enroll thru the errors and breaks and software patches that seemed to come hourly, my application got backend commands from too many versions of the software, and cannot be read by the current version. Unfortunately there is no recourse for me. I cannot create a new account because my ssi number already has an account. I have called and called the phone support, but they have no ability to help with website problems. They cannot reset a password or delete an account or an electronic application. They exist solely for two reasons: to apologize about the website and ask users to have patience, and to fill out applications over the phone for users who cannot use the website. I am stuck in limbo. So no, the website does not work (for me).

    • I’m sorry it’s not working for you.

      It seems clear to me (and is probably obvious to everyone) that, if news reports of the first month of operation are at all accurate, and others’ early experiences were as you have indicated – then the Web site should not have launched when it did. I have heard several news reports to the effect that testing of the overall integrated Web site began two weeks prior to launch. That is obscenely irresponsible if true. Like you, I suspect that almost everyone would be better off if all of the enrollments that started, say, prior to 1 NOV that have not been finished – and the corresponding accounts – were simply deleted wholesale.

      I intend to follow up and check performance and accessibility of the Web site during different hours (since this review was done on a Sunday morning) and will update this blog entry with what I find. Best of luck to you.

    • The Web site now features a “RESET” button, allowing you to remove any old application data that might be causing problems. I recommend that you try this. If you do, please report back on whether or not this substantially resolves the issues you’ve experienced to date.

      • I googled and googled, and browsed the healthcare site, and while I have found articles on this “reset” button, I cannot find it. As of right now I cannot log in at all. If I use my correct username/password credentials, it will reload the page. Maddening loop. If I use my correct username, but an incorrect password, it will notice and kick back and error message saying my password is incorrect. This happens regardless of OS or browser (tested chrome in windows and OSX, and the latest IE in windows).

      • I imagine that it is necessary to log in, before you can find or use the Reset button.

        If you can’t log in, there’s nothing else to do but call customer support.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s