Advanced search

Message boards : Server and website : Optimized bandwith

Author Message
Profile GDF
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist
Send message
Joined: 14 Mar 07
Posts: 1925
Credit: 629,356
RAC: 0
Level
Gly
Scientific publications
watwatwatwatwat
Message 54125 - Posted: 27 Mar 2020 | 22:37:54 UTC

We have optimized the network so that bandwidth to the server should double. Hopefully this will make the download/upload better.

gdf

Zalster
Avatar
Send message
Joined: 26 Feb 14
Posts: 208
Credit: 4,490,828,031
RAC: 11,006
Level
Arg
Scientific publications
watwatwatwat
Message 54127 - Posted: 28 Mar 2020 | 0:44:18 UTC - in response to Message 54125.

Thank you!!
____________

rod4x4
Send message
Joined: 4 Aug 14
Posts: 161
Credit: 1,860,712,884
RAC: 1,228,322
Level
His
Scientific publications
watwatwatwatwatwatwat
Message 54129 - Posted: 28 Mar 2020 | 3:47:14 UTC - in response to Message 54125.

We have optimized the network so that bandwidth to the server should double. Hopefully this will make the download/upload better.

gdf


I can confirm the site is far more responsive to browse via web.

Many thanks for your efforts!

kain
Send message
Joined: 3 Sep 14
Posts: 150
Credit: 567,520,285
RAC: 315,460
Level
Lys
Scientific publications
watwatwatwatwatwatwatwat
Message 54137 - Posted: 28 Mar 2020 | 14:53:59 UTC - in response to Message 54129.

We have optimized the network so that bandwidth to the server should double. Hopefully this will make the download/upload better.

gdf


I can confirm the site is far more responsive to browse via web.

Many thanks for your efforts!


Indeed, it's much faster now.

Thank you!

Aurum
Avatar
Send message
Joined: 12 Jul 17
Posts: 252
Credit: 9,791,563,847
RAC: 3,936,285
Level
Tyr
Scientific publications
wat
Message 54139 - Posted: 28 Mar 2020 | 15:32:18 UTC

I still woke up this morning to a long queue of GG WUs needing to move up & down. When they're moving the transfer rate seems faster.

ISPs choke the upload rate to be a mere 10% of your bandwidth and DL gets 90%. Since GG won't send another WU until it completely ULs the completed WU computers sit idle waiting, i.e. pregnant pauses.

What's wrong with increasing our ration to 3 WUs per GPU???

Profile ServicEnginIC
Avatar
Send message
Joined: 24 Sep 10
Posts: 198
Credit: 1,456,178,911
RAC: 932,972
Level
Met
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 54141 - Posted: 28 Mar 2020 | 17:55:59 UTC - in response to Message 54125.
Last modified: 28 Mar 2020 | 17:56:26 UTC

On March 27th 2020 GDF wrote:

We have optimized the network so that bandwidth to the server should double...

Since March 14th, due to Coronavirus crisis, here in Spain all we citizens are required by government's order for home confinement.
Gianni, Toni, and all GPUGrid's Team at backstage: Thank you very much for your continued support at current hard times!!!

Hoping everybody healthy,

Aurum
Avatar
Send message
Joined: 12 Jul 17
Posts: 252
Credit: 9,791,563,847
RAC: 3,936,285
Level
Tyr
Scientific publications
wat
Message 54590 - Posted: 6 May 2020 | 14:07:36 UTC
Last modified: 6 May 2020 | 14:26:05 UTC

I've been watching my BoincTasks Transfers page for the last several hours wondering when it will clear. When I got up there were almost no GG WUs actually running since the UL queue was full. I've been using these commands in my cc_config file for a year or so and at first they seemed to help but now I'm not so sure. Maybe if everyone used them or if they could be enforced by the server:

<max_file_xfers>9</max_file_xfers>
<max_file_xfers_per_project>3</max_file_xfers_per_project>


From https://boinc.berkeley.edu/wiki/Client_configuration
<max_file_xfers>N</max_file_xfers>
Maximum number of simultaneous file transfers (default 8).
<max_file_xfers_per_project>N</max_file_xfers_per_project>
Maximum number of simultaneous file transfers per project (default 2).

But it does not behave as described, maybe because things are actually happening faster in the computer than what's being displayed on the screen. But these commands lump CPU & GPU WUs together and treat DLs the same as ULs.

My Charter Spectrum ISP limits UL speeds to 10% of DL speeds so UL is always the choke point. I just ran a speed test with a GG transfer backlog trying to clear:
53.3 Mbps download and 4.66 Mbps upload.

It seems that it would be better if the GG server could limit the number of ULs from a given IP address.

For now I'm going to switch to 64 &1 and see how that runs through the next couple of days of server backups:
<max_file_xfers>64</max_file_xfers>
<max_file_xfers_per_project>1</max_file_xfers_per_project>

Note: I don't know the first thing about how servers work.

Erich56
Send message
Joined: 1 Jan 15
Posts: 697
Credit: 3,294,215,935
RAC: 399,269
Level
Arg
Scientific publications
watwatwatwatwatwat
Message 54591 - Posted: 6 May 2020 | 14:42:30 UTC

What I have noticed today, since this morning, was/is an obvious GPUGRID server outage a few times.
No access to the Homepage, and stalled uploads and downloads :-(

Ian&Steve C.
Avatar
Send message
Joined: 21 Feb 20
Posts: 67
Credit: 923,953,874
RAC: 6,428,825
Level
Glu
Scientific publications
wat
Message 54592 - Posted: 6 May 2020 | 16:31:59 UTC

transfers are really sluggish.
website is hit or miss.

something doesn't seem right
____________

Erich56
Send message
Joined: 1 Jan 15
Posts: 697
Credit: 3,294,215,935
RAC: 399,269
Level
Arg
Scientific publications
watwatwatwatwatwat
Message 54593 - Posted: 6 May 2020 | 16:36:46 UTC - in response to Message 54592.

something doesn't seem right

I am wondering whether the GPUGRID people are aware of the problem?
No comments here from their side so far :-(

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 508
Credit: 524,027,144
RAC: 1,662,925
Level
Lys
Scientific publications
wat
Message 54595 - Posted: 6 May 2020 | 17:27:01 UTC

Downloads are currently being limited by a single connection from the project to any host.

Toni
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 9 Dec 08
Posts: 947
Credit: 4,353,973
RAC: 71
Level
Ala
Scientific publications
watwatwatwat
Message 54596 - Posted: 6 May 2020 | 17:37:17 UTC - in response to Message 54595.

I see nothing obviously wrong, so I hope it's some international connectivity issue.

Ian&Steve C.
Avatar
Send message
Joined: 21 Feb 20
Posts: 67
Credit: 923,953,874
RAC: 6,428,825
Level
Glu
Scientific publications
wat
Message 54597 - Posted: 6 May 2020 | 17:46:29 UTC - in response to Message 54595.

Downloads are currently being limited by a single connection from the project to any host.


is there a source for this?
____________

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 508
Credit: 524,027,144
RAC: 1,662,925
Level
Lys
Scientific publications
wat
Message 54598 - Posted: 6 May 2020 | 17:47:51 UTC - in response to Message 54596.

I see nothing obviously wrong, so I hope it's some international connectivity issue.

Of course as soon as I post something about it, all the stalled uploads and downloads cleared out.

The only thing of consequence now is the project requested a 1 hour backoff.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 508
Credit: 524,027,144
RAC: 1,662,925
Level
Lys
Scientific publications
wat
Message 54599 - Posted: 6 May 2020 | 17:49:52 UTC - in response to Message 54597.

Downloads are currently being limited by a single connection from the project to any host.


is there a source for this?

No, just what I was observing and after I posted that, the rest of the connections picked up and all the stalled tasks moved to the project on both hosts.

Toni says he sees nothing wrong on his end. Thinks there might be international connection issues that we were seeing.

Aurum
Avatar
Send message
Joined: 12 Jul 17
Posts: 252
Credit: 9,791,563,847
RAC: 3,936,285
Level
Tyr
Scientific publications
wat
Message 54600 - Posted: 6 May 2020 | 18:31:07 UTC

I'm amazed that we've gone this long through an international crisis with connectivity being up. Should be no surprise that elements of the net start going down. Will probably get worse before it gets better.
My transfer queue cleared after 7 hours.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 508
Credit: 524,027,144
RAC: 1,662,925
Level
Lys
Scientific publications
wat
Message 54601 - Posted: 6 May 2020 | 19:24:28 UTC

Well I'm back to stalled up/downloads again that I can't persuade to get moving.
Hoping that posting about it works the magic again.

Keith Myers
Send message
Joined: 13 Dec 17
Posts: 508
Credit: 524,027,144
RAC: 1,662,925
Level
Lys
Scientific publications
wat
Message 54603 - Posted: 7 May 2020 | 1:44:21 UTC
Last modified: 7 May 2020 | 1:45:40 UTC

The problem seems to be that my hosts never receive an ACK from the project about successful uploads.

Half my pending uploads are sitting at 100% progress for the small files but never clear the list.

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1003
Credit: 2,522,838,291
RAC: 3,286,983
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 54605 - Posted: 7 May 2020 | 8:30:07 UTC - in response to Message 54596.

I see nothing obviously wrong, so I hope it's some international connectivity issue.
I don't know what you're able to look at, but it's been particularly bad at certain times of day for the last 24 hours.

Yesterday morning, most attempts at most connections were failing until about 10:00 UTC. Then, suddenly the floodgates opened, and I managed to get all tasks uploaded, reported, and replaced over about 20 minutes. I went out for the day, but when I returned in the evening, most machines were queuing again and were still in backlog when I went to bed.

Starting this morning at about 06:05 UTC, most machines were running, but two were in local backoff. A single 'retry', and both uploaded, reported, and downloaded at full normal speed.

There was a small glitch around 06:45 UTC, but the rot set in an hour ago, just after 07:00 UTC. A few tasks have crept through, but I now have 9 tasks uploading, and 3 tasks downloading. Each task requires 16 separate server connections: 6 to upload, 1 scheduler contact, and 9 downloads.

Most of the delays seem to be failures to connect, so I'm not sure whether they would show up in internal monitoring - possibly only in slower turnround and reduced research throughput.

With the mothballing of SETI@Home, you will have the opportunity (extra volunteers) to complete much more bioscience research. But I would urge you to, perhaps, commission a network traffic audit from a networking specialist to try to locate the cause of these problems. Otherwise, you may find that the additional volunteers float away as suddenly as they arrived.

One additional problem is that every type of connection has to pass through the same bottleneck. Now to try connection number 17, to post this message.

Failed - "This site can’t be reached. The connection was reset." Take 2...

Richard Haselgrove
Send message
Joined: 11 Jul 09
Posts: 1003
Credit: 2,522,838,291
RAC: 3,286,983
Level
Phe
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 54606 - Posted: 7 May 2020 | 9:12:50 UTC

Today's floodgates opened a little earlier. Just completed this morning's big exchange - I'm good for a few more hours.

Profile Retvari Zoltan
Avatar
Send message
Joined: 20 Jan 09
Posts: 2185
Credit: 15,823,750,346
RAC: 732,335
Level
Trp
Scientific publications
watwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwatwat
Message 54609 - Posted: 7 May 2020 | 12:01:56 UTC

I had connection problems with these DC projects:
GPUGrid
Einstein@home
folding@home
However I didn't have connection problems with:
TN-Grid
Rosetta@home (I set 24h workunits for Rosetta, so this project may be not affected by this).

Post to thread

Message boards : Server and website : Optimized bandwith