Note: The contents of this page will continue to be updated based on feedback.......

(Feedback to be sent to Webmaster)

---

background

The OneSky4 - Global Scale ATC event took place Saturday 7 May 2005, between 1400z and 2300z (for VATEUR), as the fourth ONESKY event. But, this time the event was global !

A list of VATEUR FIRs/VACCs participating in the event is available here.

• Supervisors were responsible for announcements of VACCs, and monitoring of the servers.

• Staff members were tasked to provide screenshots during the event.

• There were no special regulations for OneSky4, except the ones announced on the main OS4 page, including some flight planning recommendations for pilots.

For the first time ever, six Regions cooperated in creating a situation never seen before. A Global VATSIM Event online, side to side, top to bottom. You were able to experience ATC and flights around the world you as in real life, non stop all the way!

For the VATEUR Region, the OneSky4 Event started at 1400 UTC, and ended at 2300 UTC. As hasbeenduly noted, we did experience some network (server) issues during the event, and this is covered under the Network section below. Except from this, no major conflicts and/or incidents have been reported so far.

---

Update: Re-checking the VATSIM Statistics today (21 May 05) the numbers have changed dramatically. The accuracy of these statistics have been disputed by Richard Critz, but we include the latest VATSIM Statistics for 7 May, on peak concurrent connections, as found on VATSIM:

At the same time, looking at Total Connections per Day, also for 7 May, it states the staggering number of 14478, twice of what is "normal", which may be explained by the network issues and where more members had to reconnect to stay online.

From VACC Portugal		From VACC Finland
(1546 UTC)		(1852 UTC)

Webmasters note: The following "snippets" are picked "as-is", and does not necessarily reflect the views of VATEUR !

Michal Rok:
- Maybe a global event like this is not the way to go, maybe it will be better to have more smaller ones? I know it's nice for PR reasons to be able to say that we will have everything from Australia to Alaska covered, but there's few people who make any use of that coverage. Conclusion: do small events, help your pilots get used to reading ATC booking in advance, improve communication of events. It's more work, but the result will be more fruitful and pilots will be happy.

Martin Georg / VATEUD1:
- We don´t know exactly yet what happened during the OneSky 4 evening, but for sure we now know that the statements made before the event, that out network could cope easily with thousands of connections, were not true.
If there is any positive outcome of OneSky 4, then it is the insight, that we need fundamental changes in the way our network is managed, and how software for our network is developed and distributed.

Jan Naslund:
- If the technical guys cannot find the root cause (Webmaster: to the server issues) I think that it could possibly be a good idea to stage another mega-event like the OneSky4 but this time the technical guys would be ready and have their trace and
monitoring equipment and whatever running to try to pinpoint the problem if it occurs again.

Hans Utne:
- Online as BIRD FSS (Reykjavik Oceanic) between 1400 and 2300 UTC, and observed three periods where the servers struggled. First at around 1610 UTC, then 1905 UTC and finally around 2200 UTC. OS4 was a great event to start with, more ATCs online, increasing number of pilots joining as hours passed, but the network issues clearly took the initiative out of many as numbers dropped within Europe each time we had a server-crash.

Serdar Sualp:
- OS4 started nicely and gently. I happily took over the duty from my fellow controller around UTC 16:30 as there were about 5 traffic in LTBB FIR. By the time the number of traffic started to increase and I and all pilots in my FIR were happily enjoying the event. No lag, no voice problem, few rookies and few text mode pilots :-)
First some pilots disappeared from my scope, very shortly thereafter controllers followed. The last communication was with adjacent FIR LTAA. He said "What is happening ? I lost contact with all planes". I had the same problem and I decided to change server. I immediately shifted to UK-1 and saw some planes and controllers, but not LTTA_V_CTR. Then I called him on the phone and asked him to shift to UK-1 in the hope of seeing each other again. The answer was very simple and short ? "What UK-1 server ???" He was seeing only 2 US server. I asked him to close down ASRC and re-start again. This was a fatal mistake. Having done this, he never managed to connect to any server again. I phoned him again and asked him "keep trying".
I started to shift from one server to another, the result was very interesting. I saw different planes and controllers on different servers. I tried to control every plane by shifting from one server to the other for about 45 minutes.
Then.....
I shut down the computer and turned on the TV and enjoyed the football match of my favorite team....

Unfortunately, some network problems were encountered during this event as the servers failed at least 3 times, making things a bit difficult for both Pilots and ATCs.

---

Richard Jenkins / VATSIM12 has provided the following statement regarding the network crashes (as copied from the VATSIM Forum with his approval):

I think we can all agree things didn't go well Saturday and the network failures caused the fly-in to lose much of its momentum. VATSIM faces a different problem in regards to network connectivity than the other networks. Our network and the membership base associated with it is a truly global community. We have large populations of members on all continents, this is something that no other network can claim. With that in mind, the addition of new pilot clients, and the influx of many new members, we have encountered trouble dealing with large numbers of people online. There is one positive thing to come from the events Saturday. While most of us were aware that something has been going on with the network for the past weeks, it was difficult to pinpoint the problem because it was not always present. The fly-in made some of the problem extremely clear.

What Happened:

(First, this is very preliminary information and is subject to change. I have also removed the server names to keep the owners from having to answer a flood of mail)

1. Server-1 was acting as the main European hub. The connection between Server-1 and Server-2 degraded and became unusable. This caused the CPU on Server-1 to go to100% as the output queue filled. Server-1 was removed as the hub and Server-2 was designated as the European hub, responsible for Europe and the Oceanic connection to Server-3.

2. The Server-2 and Server-3 connection failed eventually. Server-2 does not have as good an Oceanic connection as Server-1 and this was deemed a temporary fix.

3. The connection was then changed back to Server-1 for the Atlantic hop. This worked for a period of time.

4. These multiple failures caused a ripple effect across the network which resulted in the crash.

As I mentioned above VATSIM faces a problem the other networks don't. We have members spread across the globe in higher concentrations and we must provide connection for those members. To service those members requires a stable oceanic connection between Europe and North America. The events from Saturday made it very clear that the connection is precarious at best. The problem appears to be random, but becomes very clear with intense traffic.

We realize that many people worked hard to organize the event and are extremely disappointed in what happened. It appears the fix is not going to be easy and may take some time if we have to acquire additional resources, but rest assured people are working on it.

Richard Jenkins
VATSIM12

We would like to thank all VACCs, VAs, Staffs and all who directly -, or indirectly -, were involved and participated in OneSky4.

On behalf of VATEUR, VATEUD, VATUK and VATRUS.