WebCT Sporadic Errors

We received the following late last night from our WebCT Administrators:

We’ve received confirmed reports of sporadic system exceptions within WebCT, not confined to specific tools (quizzes, file download, etc). We believe this may be related to database problems, which are currently being investigated.

We have determined the source of the WebCT database problems, and are taking steps to repair it. A data recovery process will be running overnight, and should be completed in the morning. We’ll have further status updates then.

Please email GMBA Support if you are still having problems this morning. Thank you for your patience.

Update (11:25am): We are hearing from students and faculty that problems still exist. We just received an update from our admins a few minutes ago letting us know that they are still working on the issue.

Update (4:17pm): Our administrators are working directly with Blackboard/WebCT to fix the problem. They are hoping to have this issue resolved today; however, they say it could be as late as tomorrow early morning before the server is running normally. We apologize for any inconvenience this causes. We will post more updates as we receive them.

Update (9:55am, Sept.1 : We do not yet have any update from our WebCT Admins; however, we do know that the problem still exists. We will post any information we receive. Thank you for your patience.

Update (10:41am): Just received this from our admins:

WebCT is still experiencing the System Exception Errors that manifested on Sunday. We are still in the process of repairing the database error. We are in contact with WebCT’s technical support to expedite solution discovery. We are working to resolve this issue as quickly as possible with no further data loss.

Update (5:54pm): We just received an email from our administrators. The email was sent out to all university faculty, staff, and students. To read the complete email, which includes issues that have been verified as directly related to this problem, please check your @utdallas.edu email account.

We are near the end of a second major data recovery effort towards resolving the system errors affecting WebCT. This evening, we should conclude the course of action WebCT Technical Support believes will repair the damage WebCT has suffered.

Update (8:21am, Sept.19): WebCT is currently down due to emergency maintenance. The current WebCT site has a link to the Server Outage website for updates. We will continue posting updates on this blog as we get information.

Update (9:31am): Received the following from our WebCT admins:

The data recovery effort completed last night has not brought a solution to the ongoing problems with WebCT. We are now attempting an expanded data recovery effort, which is currently requiring the database to be down. Based on the progress of the last three hours, this recovery will take at least another six hours to run. We are attempting to resolve this issue with speed as well as caution, in order to prevent or minimize data loss.

Update (6:44pm): Our administrators are now working with Blackboard/WebCT and Oracle on the database issues that are the cause of our server problems. They have determined the reason why previous attempts to restore WebCT to normal functionality have failed. Unfortunately, we are told, the recovery process they have just started is to take another eight hours to completed; however, they are very confident that this will fix our server issue and restore WebCT to full functionality. As always, we will post any further information we receive.

Update (6:56pm): I just noticed on the Server Outage website that they have posted the following:

WebCT outage is expected to last through 8:00am, 20 Sep 2007.

Update (7:23pm): Just received the following from our admins:

We’ve been working with the top tiers of support at both WebCT and Oracle today, and after hours of collaborative investigation, we have discovered the root cause of errors we have been running into with our data recovery efforts. Those errors have been rectified, and we are now cleared to perform another data recovery. This will run overnight, and WebCT will continue to be unavailable while this process runs. At this point, we are expecting to bring WebCT back online by 9am tomorrow morning, with no loss to data within the system. We appreciate your continued patience through this extremely frustrating situation.

Update (8:45am, Sept.20): Just received the following from our admins:

I am relieved to report that WebCT is up and functioning again as of 8:00am this morning. Initial testing shows that the errors encountered since Sunday have been resolved. There does not appear to have been any data loss in the system recovery, but we do encourage faculty and students to verify any changes or submission made recently. We thank you for your patience shown during this ordeal.

WebCT Possible Interruption Today at 5:15pm CST

We just received the following from our network administrators:

The HTTP load balancers, which serve WebCT as well as general web traffic for www.utdallas.edu, netid.utdallas.edu, etc, have become unstable and need to be restarted at 5:15pm CDT, 10 September 2007. This may cause an interruption for web services, including WebCT, but is expected to last less than 5 minutes.

This should fix the issues some of you have been experiencing in which you get the “system exception” error when trying to access your WebCT courses. Please contact GMBA Support if you have any questions.

(6:19pm CST) Update: Just received the following update from our admins:

I believe we have resolved the problem that was causing sporadic System Exceptions over the course of Sunday, September 9th, and Monday, September 10th. You should be able to log in and use WebCT as usual now. If you experience further problems, please be sure to include the following information in trouble reports:

Your netid
Time of error
Error message
Action that led to error (logging in, clicking on Quiz 1, etc.) Course affected by error, if applicable, including section number

We apologize for the inconvenience caused, and thank you for your understanding and patience.

Library Resources for Online Students

Many of you guys might not realize this, but you have a wealth of information available to you through the UTD Library’s website. There are thousands of newspapers, magazines, journals, trade papers, and academic documents that are available to you at any time, no matter where you are in the world.

The link below has information on exactly how to access this information and even has numerous video tutorials that step you through exactly where to click. So whether you’re searching for sources for that research paper or wanting to read a recent magazine article, this is a resource you should definitely take advantage of. Let us know if you have any questions.

McDermott Library for Distance Education Students
http://som.utdallas.edu/globalmba/library/web/index.htm