As the Obama administration’s weekend deadline for a smoothly functioning online marketplace for health insurance arrives, more than a month of frantic repair work is paying off with fewer crashes and error messages and speedier loading of pages, according to government officials, groups that help people enroll and experts involved in the project.
But specialists said weeks of work lie ahead if the $630 million site, Healthcare.gov, is to accommodate the expected flood of people seeking to buy health insurance, including a major reconfiguration of much of the computer hardware. Without the additional changes, experts predict, the website may continue to crash during periods of peak usage.
Beyond the prospect of potential delays for consumers, insurers warn that problems remain in the invisible “back end” that transmits enrollment information to them. That data has been plagued by inaccuracies, insurers say. Administration officials have been unwilling to disclose the error rate.
As late as Wednesday, the site still continued to slow down when 30,000 users tried to log on simultaneously, according to project specialists. A batch of hardware upgrades and software fixes scheduled for this weekend, administration officials say, will allow the site to handle 50,000 simultaneous users, as promised, by Dec. 1, which is Sunday.
The Health and Human Services Department announced that the site would be shut down for 11 hours Friday night in order to put those upgrades into place, on top of the usual four-hour timeout for maintenance tonight.
Although the administration has postponed a December marketing campaign, fearful that the site would collapse under a surge in traffic, five weeks of repair work have clearly made the exchange better. From last Sunday to Tuesday, nearly 20,000 users managed to enroll in insurance plans, the most for a three-day period, according to people familiar with the project. By comparison, fewer than 27,000 users picked an insurance plan on the federal site in the entire month of October.
And pages that once took an average of eight seconds to load now show up in a fraction of a second. The rate at which a user sees an error message has dropped from about 6 percent to 0.75 percent.
But the pace of enrollment must pick up drastically if the administration is to meet its target of signing up 7 million people by the end of March, the number that insurers say they need to spread risks and keep prices down. While some states that built their own sites are making better progress enrolling people, applicants in 36 states, with two-thirds of the nation’s population, depend on the federal site.
At this week’s rate of enrollment, those enrolled through the federal exchange would total fewer than 1.1 million by the March deadline. Few insurance executives expect alternative options for enrolling, including by phone, mail or in person at counseling centers, to make up that gap.
The administration has already spent more than $9 million beefing up the system’s computing power with additional servers and other hardware. The reconfiguration of the data center — the website’s computer brain — is expected to cost millions more and require up to another month of work, specialists said.
Experts involved in the repair work say the overhaul is necessary because bursts of traffic beyond the designed capacity could bog down the site, forcing users into an electronic queue until emails notify them that they can return.
The only solution, several experts said, is to reconfigure many of the site’s computer servers so that they are dedicated solely to HealthCare.gov’s tasks. Currently, most of the servers juggle demands from other clients as well.
One expert said the site needs to be able to handle 100,000 simultaneous users to provide a safe margin of error.
“Think of it as Version Two,” he said.
Tests conducted this week for The New York Times by a California-based company that evaluates websites for major commercial clients found that the site remains too complicated for many users and is still prone to errors and delays.
Federal officials continue to try to create alternative ways to enroll. The latest is a shortcut called EZ-App, developed over the past month, which will allow consumers to more easily estimate how much federal assistance they are eligible to receive.
The concept is similar to the Internal Revenue Service’s 1040EZ form, the shortest and simplest way to file a federal income tax return. But even getting this slimmed-down process to work online has proved difficult, one person said, so it will initially only be available to individuals who seek to enroll by telephone.
The overhaul of the system’s hardware foundation is intended to address deep concerns about the site’s stability and horsepower.
Because of the system’s fragility, one subcontractor, Oracle, has delivered its own server to a data center near Washington, so that its software, crucial for users to create accounts, runs faster and more reliably.
Gary L. Bloom, the chief executive officer of another vendor, MarkLogic, said his firm is also moving its software to differently configured servers. MarkLogic provided the technology for the database that serves as the system’s internal filing cabinet and index.
“I am picking up my house and moving it to a better foundation next door,” he said in an interview.
He said that MarkLogic is performing up to standard but that “the network and the storage systems are not properly sized and not properly run.”
Another critical problem involved the specifications for a major computer switch that connects the computer servers through a security firewall to the Internet. Bloom said it has been upgraded from 4 gigabytes a second to 60. He said the earlier speed was the equivalent of employing four security staffers to screen Heathrow Airport’s passengers.
“The line to get through,” he said, “would go back to the city of London.”
Other critical components of the site also remain to be finished, including a financial management tool to handle payments to insurance companies for customers eligible for subsidies.
Some officials involved in the project have expressed disappointment that the redesign work has yet to start. But several technicians warned that rushed upgrades are risky.
“Everything is just spotlight on, high pressure, has to be done live, without a net,” one said.