Problem has been solved by france telecom.

Hey guys,

actually their is a massiv connectivity issue, when your packets reach the french-opentransit (france telecom network).

The symptoms are massiv packet loss (up to 70% per HOP on french opentransit) or permanent disconnects.

The first report of that issue was in the same week as 1.3.1 was patched on the EU servers.

Since then, the situation get worse and now multiple EU-countrys are affected.

This issue seems to affect many EU-countrys, which try to connect with a service that is hosted by the france telecom or use their network.

Normally it occurs only while european prime time (6-10pm CET) and doesn't affect only warhammer.

There are reports of other games, hosted in french by france telecom.

GOA and france telecom are allready informed about this. GOA actually collect all the feedback of players (germany, england, danmark, ...).

They know that this situation isn't aceptable for many of you and try to fix it very fast!

To fix this issue, they need help from the users.

Which each trace route you publish here, you'll help to identify the problem(s).

In the meantime, stay calm and continue to post information about this problem to give GOA and/or france telecom the possibility to fix this issue.

And, in addition, take a look in here to get the latest news about that problem. I'll post them, if something is to mentioned.

As noted in the connectivity feedback post I have had serious problems with Warhammer Online lately, being a network engineer in the making (Cisco Acadamy Student) I took the time to run some more extensive testing on this matter. My problem has been that the server closes down my connection quite periodically, especially when the server is under some stress. During prime time the server closes my connection every 10-30 minutes and after a reconnect I end up on the character select screen. Just before these disconnects I often experience the game from a very personal view where everyone freezes while I can move around freely and my client allows me to perform actions but none of them affect my opponents, allies or the enviroment.

I tried to remedy my connection issues by changing the settings of my network adapters, changing my network equipment, changing DNS server and finally I did some traceroutes and found that the error was not on my part. From what you can see on the data below the issues start with the final server in cogentco's network, an ISP commonly used to purchase bandwith worldwide, but the problems do not stop there. As soon as I enter the opentransit.net the problems continue and even though GOA does now own that net it is the domain of France Telecom and hence the home of GOA (GOA is part of the Orange/France Telecom enterprise) which makes me file my complaint here. An average package loss of 10% on every(!) hop within the GOA controlled network performed over 7,5 hours with 5 second intervalls is simply not acceptable! The fact that it also takes my connection 8 hops to reach the final IP when tracing patcher.war-europe.com also leaves some questions about the routing in there, especially when I have 10% chance to loose a package.

I have already contacted my ISP about these issues, and they promised to do what they could but that is quite limited considering where the errors occur. I also intend to report to cogent co about the 20% package loss on the final hop in their network and here I am asking GOA to investigate what the problem could be on their end. 

Hopefully we can resolve this issue together, right now the game I enjoy and love is simply unplayable.

Data results from 2000 traceroutes with pingplotter:

 Target Name: patcher.war-europe.com
         IP: 80.12.102.30
  Date/Time: 9/30/2009 6:10:32 PM to 9/30/2009 11:04:52 PM

Hop Sent Err  PL% Min  Max Avg  Host Name / [IP]
 1  2000   3  0.1   0 5954   3  my.router [192.168.1.1]
 2  2000   3  0.1   0 5914   4  c-XXX-XXX-XXX-XXX.cust.bredband2.com [REMOVED]
 3  2000   3  0.1   4 5874  16  cr-se-mlm-kvg15-1-v199.bredband2.net [89.160.97.113]
 4  2000   3  0.1   4 2976  12  er-se-mlm-kvg15-1-v8.bredband2.net [83.233.160.19]
 5  2000   3  0.1   4 2976   8  te1-3.ccr01.mma01.atlas.cogentco.com [149.6.48.5]
 6  2000   3  0.1   5 5754  11  te1-2.ccr01.cph01.atlas.cogentco.com [130.117.0.49]
 7  2000   3  0.1  11 5714  17  te2-3.ccr01.ham01.atlas.cogentco.com [130.117.2.149]
 8  2000   6  0.3  20 5674  25  te1-8.mpd02.fra03.atlas.cogentco.com [130.117.49.237]
 9  2000   3  0.1  31 5634  36  te7-2.mpd01.par02.atlas.cogentco.com [130.117.0.102]
10  2000   3  0.1  31 5594  35  te2-2.mpd02.par01.atlas.cogentco.com [130.117.2.81]
11  2000 401 20.1  32 5554  38  te1-3.ccr01.par04.atlas.cogentco.com [130.117.2.78]
12  2000 191  9.6  43 5514  50  gi1-3.parse7.Paris.opentransit.net [193.251.254.241]
13  2000 176  8.8  43 5474  49  te2-2.parse1.Paris.opentransit.net [193.251.129.173]
14  2000 196  9.8  44 5434  49  tengige1-8-0-2.pastr1.Paris.opentransit.net [193.251.128.77]
15  2000 204 10.2  44 5394  49  pos0-11-0-0.lontr1.London.opentransit.net [193.251.242.113]
16  2000 187  9.3  48 5354  54  ge-2-2-0-0.nrlil202.Lille.francetelecom.net [193.252.100.225]
17  2000 195  9.8  49 5314  54  [80.12.201.6]
18  2000 211 10.6  49 5274  54  [193.253.67.14]
19  2000 179  8.9  49 5234  53  [80.12.102.30]

And a picture from the same sample:

Hello Kallisiti,

actually there is a connectivity/forwarding problem on the opentransit network.

I've allready tell it to the community managers, which will collect our feedback about that issue (germans are effected as well) and will send it to themselves (France Telecom). :)

So continue to post error reports for it so that the technicians can fix it soon.

Further analysis shows that its a ICMP Time To Live Exceeded error, as expected when tracerouting. None of the packages I analysed had fragmentation, so its probably a timeout somewhere. That made me change the time to wait for the pings to 30s but nothing really changed there either (standard wait time was 3s) so the lost packages are probably thrown away or routed to some dead end.

I also tried connecting through a VPN in russia and england, and had no errors whatsoever on those routes. If it would help your analysis I could also supply some wireshark logs even if I prefer not to post information like that on public forums. In other words, anything else I can do to help?

When you say, that your traces' was ok from england and russia, but your own connection (properly of your ISP) and the connections of some german ISPs not, what does that mean.

I can remember while I played WoW (2-3 years ago), there was a massiv overflow problem between the german-opentransit and the french-opentransit, but here aswell only for some ISPs.

Their had discuss to extend the network between this HOPs, but if they finished the project, don't know.

Maybe here it's the same problem, cause it only occurs while primetime 6-10pm (GMT+1).

What you can do is... simply make more different tests.

The CMs told me, that as more people - or better countrys - are effected, as fast there is a chance to get a fix.

You said, that you tested it via VPN in england/russia, maybe you can share the results (traceroutes and so on) or send it directly to one of the CMs.

I'll talk to them tommorrow, that your're allowed to send them emails.

When I play via a VPN tunnel my ip is moved away from my ISP to the network provider that hosts my VPN service so one conclusion to draw from that and the fact that the problems occur mostly on the France Telecom owned network would be that its some trouble with the routing with the broadcast agent between France telecom and German / Swedish ISPs. Either that or if GOA purchased bandwith on an international network like cogent co directly, in that case its that agent and the France Telecom network that needs to handshake a bit better! :)

When I had a similar problem in a more local network it was a router that was flooded so to speak, it was simply to slow to handle the traffic and threw away those packages it could not handle resulting in periodical package losses as the router's memory was full. I doubt this is the case here, but I thought it might be worth mentioning.

The fact that the problems only occur during prime time is probably the result of the load on the servers. From what I can see the package loss is occuring on most hours, but during high load the game servers simply does not have the time to tolerate and wait for lost packages and closes the connection. Atleast thats my theory! :)

Anyways, here is a traceroute log from my connection using a VPN tunnel to Russia. I can collect a log through England too if thats helpful and if your technical staff wants wireshark logs I would be more then happy to email them.

Target Name: patcher.war-europe.com
         IP: 80.12.101.250
  Date/Time: 10/2/2009 2:24:02 AM to 10/2/2009 2:40:38 AM

Hop Sent Err   PL% Min Max Avg  Host Name / [IP]
 1   200   0   0.0  36  97  40  [1.254.6.15]
 2   200   0   0.0  36  78  41  [1.254.6.11]
 3   200   0   0.0  37 168  44  [85.249.223.1]
 4   109   0   0.0  37 142  45  ae0-22.RT.V10.MSK.RU.retn.net [87.245.253.1]
 5    97   0   0.0  59 153  70  ae2-6.RT.TC1.STO.SE.retn.net [87.245.233.134]
 6   200   1   0.5  74 176  83  ae-5-5.ebr1.Dusseldorf1.Level3.net [4.69.140.198]
 7   200   0   0.0  85 134  92  [4.69.142.170]
 8   200   0   0.0  89 140  94  ae-2-2.ebr2.Frankfurt1.Level3.net [4.69.132.138]
 9   200   0   0.0  89 222 101  ae-92-92.csw4.Frankfurt1.Level3.net [4.69.140.30]
10   200   0   0.0  89 178  98  ae-4-99.edge3.Frankfurt1.Level3.net [4.68.23.203]
11   200   0   0.0  85 232  98  [4.68.111.234]
12   200   0   0.0  99 181 104  tengige0-2-1-0.lontr1.London.opentransit.net [193.251.131.134]
13   200 200 100.0   0   0   0   [-]
14   200 200 100.0   0   0   0   [-]
15   200   0   0.0 103 149 110  [193.253.67.14]
16   200   0   0.0 103 173 109  [80.12.101.250]
 

Another note, cogent co was aware of this problem. THis is what I said to them:

As you can see in the attached image and the statistics below you seem to
have some trouble with te1-3.ccr01.par04.atlas.cogentco.com [130.117.2.78],
im not sure if the error report should come from me but I thought I'd inform
you of this problem anyways. I sent this message to the support departments
responsible for Sweden and France since this tracerout goes between
bredband2.com (my ISP) and GOA (france telecom). Please check this problem
if you can or forward this message to the appropriate department.

Thanks in advance. 

And here is their reply

Hello,

Thanks for your mail, this is a known problem . Please, if you are a Cogent customer, send us your service ID and company name and we will inform you about this issue.

Best regards,

Cogent EU-NOC

Hi...

I have a laag and FT is the reason....

That is my tracert generated log....

1.   1ms  1ms <1ms  192.168.1.1   (my router)

2.  1ms 1ms 1ms 192.168.0.1 (my router 2)

3.   64ms     62ms    63ms    brhm-bam-2.network.virginmedia.net [194.145.148.3]

4.   60ms    55ms    59ms     brhm-core-1a-ge-114-0.network.virginmedia.net [213.106.231.85]

5.   62ms    60ms    63ms     bir-bb-a-ae1-0.network.virginmedia.net [213.105.103.97]

6.   68ms    61ms    63ms     168.ge-1-3-3.mpr1.lhr2 .uk.above.net [213.161.65.149]

7.   75ms    67ms    67ms     xe-1-1-0.mpr1.cdg11.fr.above.net [64.125.24.2]

8.   89ms    67ms    77ms     above-ft-3.cdg11.fr.abov.net [64.125.12.202]

9.   68ms    67ms    70ms     tengige1-1-0-0.pastr1.Paris.opentransit.net [193.251.128.69]

10. timeout

11. timeout

12. timeout

13. 74ms    73ms    78ms     80.12.201.6

14. timeout

15. 193.253.67.14  network not accessable. 

Any other sugestions........ What can I do to make it better.... choosing different ISP wont solve the problem....

Thx for mooving this Kaxzane... but basicly it is request to FT and your Technical guys....as problem is very painfull....

Assessment what to do from your side (GOA) would be beneficial.... as it affects not me only....

Regards.

PS. Waiting for CSR respond regarding this problem for last 1h.... (one ticket closed without answer to me or ingame mail ) ;/

EDIT:.... Second CSR ticket locked without answer : / (Making 3rd one then) :/

EDIT2: Where to complain about CSR guys ??!!

EDIT3: You have some problems with main page of War-europe.... (java script error)

So, you're figuring this is not a Virgin Media issue?

I've been on their case for a week trying to sort out my lag, blaming them for bad routing. The VM support forum has a few threads started by gamers suffering high pings, seems to have all started week and a half back. Though this also coincided with the last big patch (the 900mb+ one)

If it's not Virgin Media's fault I'll be eating humble pie.