Important Announcement **UPDATED**

[Kirby]

It has come to my attention in a few ways that lately people seem to be having trouble with downloading maps in-game, myself occasionally included.

Seeing as how I'm the one who pays for this thing, this is a topic which obviously concerns me, in more ways than one.

I wanted to just make a little announcement on what's going on, and potentially why it's going on as well, just so if we have to be annoyed for a time, at least we know what's up.

So here's what *should* happen under the normal circumstances.

your client: - Hey server, I wanna connect.
Server: Ok, we're on map X.
Your client: I don't haz that. =(
Server: Ok, ask <Kirby's FTP> for mapname.bsp.bz2, tell them I sent you.
Your client: Ok!
Your Client: Hey Kirby's server, I need mapname.bsp.bz2, and <red/blue IP here> sent me.
My server: No problem homie, here ya go.
Your client: \o/
Your client: Yo game server, I'm back wif the map yo!
Server: WELCUM D00D

Now, here's what's happening, read closely or you'll end up looking silly, like Tardle.

your client: - Hey server, I wanna connect.
Server: Ok, we're on map X.
Your client: I don't haz that. =(
Server: Ok, ask <Kirby's FTP> for mapname.bsp.bz2, tell them I sent you.
Your client: Ok!
Your Client: Hey Kirby's server, give me mapname.bsp
My server: ...wtf is this I don't even...
Your client: <emo rage>
Server: lol fu

IF THIS HAPPENS TO YOU

I need to know about it!

Send me a PM here on the forums with some info, namely the time it happened, your timezone, what map you should have been downloading, and what happened from your side. Did you crash to desktop? Did you get booted saying your map differs from the server? Some other error? Nothing at all, just pop back to server browser? I need to know details. AND GIVE ME YOUR IP ADDRESS. All this info is useless to me without your IP address, as I have no idea who you are in my logs.

[Tricks]

I lol'd

[Turtle]

Your servers kind of a dick bro

[CaffeinePowered]

Your servers kind of a dick bro

More like people's games are being dumb by asking for the BSP, since valve has coded it into source to use BZ2 files instead

[Surf314]

Your server gives great head speed when it's working though. I was getting 2 mbps once I think.

[Kirby]

Your server gives great head speed when it's working though. I was getting 2 mbps once I think.

Yeah I've had that server for upwards of two years now, I've always had absurd speed compared to the price I pay for it. That's why I started bugging Caff months before we needed a new FTP server if he wanted to use mine in the first place.

[Kirby]

So just to stave off any possibility of this problem being my server, I've been in contact with my host throughout the day, and determined a few things.

1. My host has a limit on concurrent connections that will throttle any overly active website, as any major web host does.

1a. We can't come remotely close to being in the same timezone as triggering this auto-throttle with a 24 man server, we can generate a max of 24 requests at a time, web hosts can serve tens of thousands per second.


2. There have been funky malformed requests coming into rilwar.info with stunning regularity, all pointing to the maps directory.


So, to clarify this more;

The problems with map downloads is the fault of the tf2 client, it is not requesting files properly, with disappointing regularity.

This could be solved by simply uploading the raw .bsp map files instead of their zipped counterparts, as the source engine is designed to ask for a .bsp file if the webserver says it doesn't have the .bz2 file, but that would mean going from using a little over 4GB of space to using well over 15GB of space, and map downloads could potentially take 4x longer at times, not to mention the headache Caff and myself don't want to deal with there.


So yeah, we suck it up and wait for valve to fix this.

[CaffeinePowered]

So just to stave off any possibility of this problem being my server, I've been in contact with my host throughout the day, and determined a few things.

1. My host has a limit on concurrent connections that will throttle any overly active website, as any major web host does.

1a. We can't come remotely close to being in the same timezone as triggering this auto-throttle with a 24 man server, we can generate a max of 24 requests at a time, web hosts can serve tens of thousands per second.


2. There have been funky malformed requests coming into rilwar.info with stunning regularity, all pointing to the maps directory.


So, to clarify this more;

The problems with map downloads is the fault of the tf2 client, it is not requesting files properly, with disappointing regularity.

This could be solved by simply uploading the raw .bsp map files instead of their zipped counterparts, as the source engine is designed to ask for a .bsp file if the webserver says it doesn't have the .bz2 file, but that would mean going from using a little over 4GB of space to using well over 15GB of space, and map downloads could potentially take 4x longer at times, not to mention the headache Caff and myself don't want to deal with there.


So yeah, we suck it up and wait for valve to fix this.

You should email valve with the log files and your findings, they might be able to further look into the bug

[Kirby]

Already in my plan once I'm home.

[Dtrain323i]

Maybe you'll get lucky and get trolled by gabe newell  :p

[Badgerman of DOOM]

Yeah, shit like this happened to my friend when he was trying to join the server one night.
A possible solution might be to have a downloadable rotation .7z file under the "downloads" section of the main site, so if anyone is really adamant about joining they can just grab it from the site. I have the file set up and ready to go if you guys want me to send it to you.

[Kirby]

Yeah, shit like this happened to my friend when he was trying to join the server one night.
A possible solution might be to have a downloadable rotation .7z file under the "downloads" section of the main site, so if anyone is really adamant about joining they can just grab it from the site. I have the file set up and ready to go if you guys want me to send it to you.

Except we rtv off rotation how often....?


The entire FTP is availiable for anyone at

http://www.rilwar.info/www.brbuninstalli.../tf2/maps/

or

www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/[url=http://][/url]

[Badgerman of DOOM]

[quote author=Ensign Epic link=topic=4712.msg159251#msg159251 date=1277007713]
Yeah, shit like this happened to my friend when he was trying to join the server one night.
A possible solution might be to have a downloadable rotation .7z file under the "downloads" section of the main site, so if anyone is really adamant about joining they can just grab it from the site. I have the file set up and ready to go if you guys want me to send it to you.

Except we rtv off rotation how often....?
[/quote]

While that might be true, we tend to RTV to a stock map or another map on rotation. Another idea is to put a map pack up in the downloads section so you can just grab all the maps at once.

[CaffeinePowered]

[quote author=Kirby, the AxeHammer Zealot link=topic=4712.msg159315#msg159315 date=1277048634]
[quote author=Ensign Epic link=topic=4712.msg159251#msg159251 date=1277007713]
Yeah, shit like this happened to my friend when he was trying to join the server one night.
A possible solution might be to have a downloadable rotation .7z file under the "downloads" section of the main site, so if anyone is really adamant about joining they can just grab it from the site. I have the file set up and ready to go if you guys want me to send it to you.

Except we rtv off rotation how often....?
[/quote]

While that might be true, we tend to RTV to a stock map or another map on rotation. Another idea is to put a map pack up in the downloads section so you can just grab all the maps at once.
[/quote]

wget the FTP, with how often we add new maps Ive just given up on making a map pack, its just not worth it

[Kirby]

Just because I'm bored and I was doing some boring tedious admin-style things, here's and excerpt from a mapchange to basalt wif an assload of pubbies in the server, so we have a good example here.

Code:

__USER IP_____________________TIMESTAMP____________________________________________________REQUESTED_URL__________________________________________HTTP VERSION__HTTPCODE_FILESIZE___REFERRER____________BROWSERTYPE
69.119.81.54 - -   [11/Jun/2010:20:45:08 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
74.140.210.176 - - [11/Jun/2010:20:45:09 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
69.142.166.191 - - [11/Jun/2010:20:45:09 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
24.38.182.34 -   - [11/Jun/2010:20:45:13 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
24.145.173.101 - - [11/Jun/2010:20:45:14 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
66.183.133.70 - -  [11/Jun/2010:20:45:14 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
184.56.124.108 - - [11/Jun/2010:20:45:15 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
142.162.122.17 - - [11/Jun/2010:20:45:16 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
76.254.25.175  - - [11/Jun/2010:20:45:18 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
114.77.254.237 - - [11/Jun/2010:20:45:23 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
72.135.111.140 - - [11/Jun/2010:20:45:24 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
72.241.141.107 - - [11/Jun/2010:20:45:27 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865985 "-" "-"
68.32.250.21   - - [11/Jun/2010:20:45:35 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp     HTTP/1.1" 404 2784    "hl2://8.12.20.112:27015" "Half-Life 2"
68.32.250.21   - - [11/Jun/2010:20:45:52 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
173.29.156.43  - - [11/Jun/2010:20:49:14 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"
216.104.15.130 - - [11/Jun/2010:20:50:57 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.0" 200 442418 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
68.32.250.21   - - [11/Jun/2010:21:00:04 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 262205 "hl2://8.12.20.112:27015" "Half-Life 2"
74.108.20.138  - - [11/Jun/2010:21:19:33 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865985 "-" "-"

So here's the HTTP logs for dummies on what this says, for those of you who are curious but don't know wtf that said. If you know what that said... well you already know what all the junk below here says. That's why I put the log 1st. I LOVE YOU GUIES.

Should always be <IP> <DATE> <URL> HTTP/1.1 200 <SIZE OF FILE IN Bytes> <RED/BLUE IP> "Half-Life 2"

Following the timestamps while reading down, we can see each user progressively loading to the point where it realizes it's a new map it doesn't have and it asks for it. The area where the log says "HTTP/1.1 200" - the 200 is the HTTP code for "File found, and served to client properly" - it's the opposite of the 404 Not Found error that p. much everyone on the internets knows.

Now, as you read progressively further down, you see some lines that obviously don't match the rest, these are the users who got an error and booted out with some random error, let's dissect these.

Code:

72.241.141.107 - - [11/Jun/2010:20:45:27 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865985 "-" "-"

Looking at most of the request, it's OK, the users TF2 client asked for the proper file, koth_basalt_b1.bsp.bz2 - and we can see by the HTTP 200 code and the filesize, it sent properly, even though the users client did not supply a referrer and it's user string like every other client did.

Code:

68.32.250.21   - - [11/Jun/2010:20:45:35 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp     HTTP/1.1" 404 2784    "hl2://8.12.20.112:27015" "Half-Life 2"
68.32.250.21   - - [11/Jun/2010:20:45:52 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 5865990 "hl2://8.12.20.112:27015" "Half-Life 2"

Now here's a prime example of what's going on. We can see by the IP it's the same user, and as indicated by the top to bottom order, the fileserver got a request for a .bsp file, said it's not here and then got a request for the .bsp.bz2 file 18 seconds later. The 18 second delay is far too long for this to have been the automated process, so that indicates to us that the user wanted to stay, and did the retry in console to connect, and his client asked for the proper map file after that.

Code:

216.104.15.130 - - [11/Jun/2010:20:50:57 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.0" 200 442418 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

I'm not even sure what to make of this really. Checking the IP across the whole log I can see this user has requested 9 maps so far this month, and has gotten 2 rejections from the server because of bad .bsp requests, and overcame both of them with a retry in console. Based on this log, they either quit TF2 or alt-tabbed out, hopped on firefox and downloaded the map manually, but if we look at the filesize #, it stops @ 442418B(0.4MB), which is far smaller than the 5.5MB size of the map, so looks like they alt-tabbed only to not download the whole map....

Code:

68.32.250.21   - - [11/Jun/2010:21:00:04 -0500] "GET http://rilwar.info/www.brbuninstalling.com/maps_ftp/sv_downloadurl/tf2/maps/koth_basalt_b1.bsp.bz2 HTTP/1.1" 200 262205 "hl2://8.12.20.112:27015" "Half-Life 2"

Here's the last weird one. I didn't get a referrer or host, but it also requested the proper file, but based on filesize, the user canceled the map download prematurely.