port blocking, throttling, and scrambling

[alphabeater]

http://yro.slashdot.org/yro/02/07/14....shtml?tid=153

roadrunner in texas is, apparently, blocking the fasttrack port.

http://homepage.ntlworld.com/j.bucha...x/blocked.html

this site is devoted to solving a similar problem with isps limiting traffic on winmx's ports. on top of this, people have been reporting similar things since the early days of opennap servers. some isps seem very eager to limit what their customers can and can't use their internet access for - and some are almost monopolies in their areas, too.

the way the tcp (as well as udp) protocol works is a key vulnerability of decentralised p2p. for a program to act as a server (ie. receive any incoming connections), it must listen on a port. fasttrack (kazaa/grokster) uses port 1214, winmx uses 6699 and 6257, opennap uses 8888, gnutella uses 6346, and even freenet has to use a standard port, 8481. although the port is usually configurable, about half the peers in the network need to use the standard port for the network to remain stable - otherwise, they can't find each other.

one solution would be to have users enter a port number (this is the solution widely adopted on opennap, server : port lists instead of just server lists). this can, however, become cumbersome.

thinking about this, i realised - what is the one thing that both the server and client will know before establishing a connection? the ip addresses of both. so, instead of ever using a default tcp port, ports for individual peers could be worked out using a algorithm known by all peers.

for example, if my ip address was 64.124.41.39...

each number in an ip is 1 to 255
4 x 250 = 1000
so i could take 5 from each number, making 59.119.36.34
add these numbers together, 59 + 119 + 36 + 34 = 248
multiply by 10, 248 x 10 = 2480

so my p2p program would listen for incoming connections on port 2480 - this could be worked out when the program was started, and then quickly worked out by any peer encountering mine just before it connected to me. i think that this technique would stop isps from quickly and easily throttling or blocking certain ports used by known p2p programs.

simple, but effective.

[butterfly_kisses]

Originally posted by alphabeater
http://yro.slashdot.org/yro/02/07/14....shtml?tid=153

roadrunner in texas is, apparently, blocking the fasttrack port.

http://homepage.ntlworld.com/j.bucha...x/blocked.html

this site is devoted to solving a similar problem with isps limiting traffic on winmx's ports. on top of this, people have been reporting similar things since the early days of opennap servers. some isps seem very eager to limit what their customers can and can't use their internet access for - and some are almost monopolies in their areas, too.

the way the tcp (as well as udp) protocol works is a key vulnerability of decentralised p2p. for a program to act as a server (ie. receive any incoming connections), it must listen on a port. fasttrack (kazaa/grokster) uses port 1214, winmx uses 6699 and 6257, opennap uses 8888, gnutella uses 6346, and even freenet has to use a standard port, 8481. although the port is usually configurable, about half the peers in the network need to use the standard port for the network to remain stable - otherwise, they can't find each other.

one solution would be to have users enter a port number (this is the solution widely adopted on opennap, server : port lists instead of just server lists). this can, however, become cumbersome.

thinking about this, i realised - what is the one thing that both the server and client will know before establishing a connection? the ip addresses of both. so, instead of ever using a default tcp port, ports for individual peers could be worked out using a algorithm known by all peers.

for example, if my ip address was 64.124.41.39...

each number in an ip is 1 to 255
4 x 250 = 1000
so i could take 5 from each number, making 59.119.36.34
add these numbers together, 59 + 119 + 36 + 34 = 248
multiply by 10, 248 x 10 = 2480

so my p2p program would listen for incoming connections on port 2480 - this could be worked out when the program was started, and then quickly worked out by any peer encountering mine just before it connected to me. i think that this technique would stop isps from quickly and easily throttling or blocking certain ports used by known p2p programs.

simple, but effective.

you forgot to add:

Brilliant. I don't know who you are..but I like your ideas. Thanks for sharing even more with us in the future.

Note to Developers: hope you are listening I know I am.

Thanks, alphabeater

[Mowzer]

Good post Alpha beater.

I hear what your saying about those pink jerky bastards, I am glad kazaa is being blocked.

Its nothing but a crap service. Should have went down the toilet years ago.

The creators and now its new owners have really spunked it up. As for ISP's blocking p2p in general.

Bad Idea. I rather beat off till my hands blister then have those kinda restrictions go into effect.

[Scyth]

Why not simply use random ports (like Freenet)? The port information is carried with search results anyway (which is why it's possible to change the port at all), so it shouldn't be necessary to be able to extract/generate the port information from other information.

Still, even then ISPs will simply switch to a more advanced solution that involves traffic monitoring. In order to make this difficult/impossible, some sort of encryption could be used (the keys could be passed along with the search results).

[alphabeater]

scyth:

generating port numbers from ip addresses would make the network easier to organise, in my opinion - while it's true that randomly generated ports can be carried within search results, that doesn't solve the problem of finding places to connect to in the network in the first place. freenet appears to do this using a large list of nodes' addresses and listening ports - seednodes.ref - that you download along with the client, and can update from sites within freenet (such as the freedom engine). this is a far from ideal approach to the problem.

i agree about encryption, however - it's the best way to stop traffic being monitored, although i can't help thinking that not many isps would be bothered with the time and money it'd take to try to monitor and police everything their users do, not to mention the legal implications of that course of action.

[butterfly_kisses]

Why not simply use random ports (like Freenet)? The port information is carried with search results anyway (which is why it's possible to change the port at all), so it shouldn't be necessary to be able to extract/generate the port information from other information.

I wish I was more familiar with the Freenet and how it works. I have tried this though with KaZaA by modifying manually the port numbers (default 1214) in the partially downloading .DAT files and for whatever reason the KaZaA program rejected the modifications to the .dat file (used Ultraedit version 9.x)...wait actually i tried modifying the file's path from the default path that it uses which is generally

/http://someipaddress:1214/012345/somefile.gif

I tried modifying it to grab /http://someipaddress:1214/c:/windows/temporary%20internet%20files/kmdb.html

just to see if the kazaa protocol/service was or would be vulnerable to this type of exploit...I may need to go back and test that theory again...only doing as you suggested with trying a different port.

I have been successful using KaZaA while running a socks server and KaZaA did use Non-standard ports (other than 1214) it's been awhile since i did this but I may still have my notes on it.

What alphabeater is suggesting letting the clients communication configure the port number on-the-fly based on a certain calculation is I think GENIUS....from using Packet capturing programs such as Ethereal you can see pretty much in real time how the dhcp servers communicate requests for ip addresses like 'you who had this ipaddress please tell suchandsuch ipaddress'. So it's quite possible this could be done with a p2p program.

[JackSpratts]

b-b-b-but you can't fool your isp. they know your ip# better than you do! if you change the port based on a anything public they'll use that info to reprogram their servers and reject the new one(s). piece o cake. it’s got to be something really disgiused.

- js.

[Stoepsel]

And it's also not a good idea to have the clients scan for open ports in search of supernodes. ISP rightly have something against this because it could be interpreted as hacking attempts.

Stoepsel

[AYB]

/http://someipaddress:1214/012345/somefile.gif

I tried modifying it to grab /http://someipaddress:1214/c:/windows/temporary%20internet%20files/kmdb.html

Suffice to say this doesnt work. The /012345/ directory doesnt actually exist. It's a hash based upon the file in the directory itself. So as far as I'm aware the HTTP server a FastTrak client runs will only allow access to such directories.

As JS said, if it becomes popular/controversial enough that ISPs want to block/throttle the ports, surely they can just calculate which port you're running on themselves?

[alphabeater]

Originally posted by JackSpratts

they know your ip# better than you do!

eh? it's 4 numbers and 3 dots that myself and my isp know equally well.

Originally posted by AYB

As JS said, if it becomes popular/controversial enough that ISPs want to block/throttle the ports, surely they can just calculate which port you're running on themselves?

they could calculate it themselves, but it would stop them just being able to block one port for all customers and instantly deal with the problem.

you think they'll bother to reprogram a port-blocking system to calculate ports using an ip-based algorithm? it'd be an expensive project when you think of the millions of customers (possibly) that such a system would have to deal with.

also, remember that the 'take 5 from each, add together, multiply by 10' was an example. anything actually used could be far, far more complex, and proprietary if need be.

[JackSpratts]

Originally posted by alphabeater

eh? it's 4 numbers and 3 dots that myself and my isp know equally well.


they could calculate it themselves, but it would stop them just being able to block one port for all customers and instantly deal with the problem.

you think they'll bother to reprogram a port-blocking system to calculate ports using an ip-based algorithm? it'd be an expensive project when you think of the millions of customers (possibly) that such a system would have to deal with.

also, remember that the 'take 5 from each, add together, multiply by 10' was an example. anything actually used could be far, far more complex, and proprietary if need be.

i could guarantee you 90% of pc users worldwide (including a few nu members) have no idea what their actual ip address is at any given moment and maybe 50% of those people have no idea what an internet protocol number is in the first place. their isps on the other hand absolutely have to know by definition, but this is a side issue nonetheless.

furthermore, all an isp has to do is use the algorithm to calculate this new port once, and within seconds have it for every subscriber on their network. it’s no work at all to block one additional port per customer, even if each succeeding one is different. i’m not saying it’s a bad idea alphabeater, on the contrary, it’s a good one. as a matter of fact it’s the best idea i’ve heard in a long while that deals with this problem. but i am saying that encryption may be required, and that the number used to choose a new port must in no way be accessible to your isp and probably not to you either by extension. it will make it more complicated for your firewall but hey, nobody says you have to use one. personally, i like the idea of running p2ps on stand alone pcs with no firewalls anyway. it’s so much easier and they’re so much faster all around. i call it ”skinny dipping”. just back-up your downloads regularly (including your new p2ps themselves and all their settings) and if you get hit a simple quick restore/reinstall puts you back in business.

- js.

another ip# look-up.

[db_]

Hi.

The RoadRunner block on the default WinMX listening port 6699 has been going on since at least January this year. The block only affects certain areas of the RR network, in particular the Texas regions Houston and Austin regions were reported by users frequently as having the problem. The 6699 filter usually allowed an outside user to connect into the RR users 6699 serving port, the transfer would usually get to around 8KB - 30KB before stalling and timing out completely.

The easily 'solution' to this was for these users to specify a port number other than the 6699 port. This then circumvents the 6699 block that RR are running and allows uploads to operate as normal. Another problem that's been occurring it seems is that these same RR users are finding they can't search for files on the network correctly, reporting searches for popular artists failing very early in, only a handful of results returned before stopping completely. It seems that the RR 6699 block is not only filtering incoming connections to RR users 6699 server ports, but is also filtering external 6699 connections. The RR user initiating a search for a file asks the/other Primary connections it's connected to, and it's invariably connected into these other Primary users via their 6699 port, and so if the RR block is restricting search results, it's restricting *external* connections to 6699, this IMO is unjustifiable, but I'm sure RR if faced with it could make up a reason to justify blocking the service WinMX, and not just incoming connections to their users.

Looks summit like this...
An off-net user connects into a RR user to download a file...

DL'er:1663 connects into RRuser:6699

The filter prevents this from operating, no problem, it seems the filter is also filtering outside 6699 connections though like...

RRuser:3527 is connected to PrimaryUser:6699 (superpeer)

When the RRuser initiates a search request, it asks the Primary, or Primaries it's connected to for results, and those Primaries ask every other Primary, all the results get returned to the initial Primary that the RRuser is connected to, and then received by the RRuser on their already established 3527 to 6699 connection. If the results are falling under the filter that RR have in place, they are not filtering their own users operating servers on 6699, they're in fact filtering users on external other networks running on the 6699 port. How an ISP can justify this I really don't know.

damn, did any of that make any sense or what? :-)

Anyway, the way I see it is the solution is.... the clients upon installation and configuration should try to assign a random port, instead of basing every user on the 'centralized' 6699 port by default. You can't assign dynamic ports really as far as I'm aware that change per session, etc, as users Routers need configuring with port forwarding very often to enable incoming connections to the client. This means the clients ports need to remain stationary so a Router forwarding rule can be created such as 'forward 16699 packets to machine 192.168.0.200', this requires the client have the port 16699 static. If the client port changes, the Router forwarding becomes useless, and no uploads will take place (note).

So, the clients just need a feature that upon installation of the client it chooses a fairly random port to use, say, anything between 1024 and 5000, this creates a network that is not centralized around a single common port, this means ISP's will find it harder to control specific services through the filtering of the common ports used for the service.

There are other ways an ISP can filter connections, but I think it's not a straight forward thing to do with apps such as WinMX, as the packet data appears as junk, and as far as I'm aware has no repetitive info within the packet to distinguish it as a specific service to be filtered. The Opennap and KaZaa protocols for example are far easier to filter as they contain clear information that can be filtered by using packet rules, you can even sniff a Opennap packet and read in English the user password, username on the network, file being transferred, files being served etc in plain English, this makes a filter rule easy to create. This is a good reason I no longer use Opennap, although I'm not scared of a note from my ISP stating I was sharing blah blah, I just think the protocol is insecure and old compared to what seems to be required nowadays to help protect one's privacy, and protect one's self from being busted. :-D

There is another client I noticed that has implemented the randomized port idea, I'm not sure what it was now, maybe Filetopia, which I was very happy to see added, hopefully it'll become a standard practice as new clients emerge to scatter the ports away from the default presently used.

Right I better shut up now, all the above is only *IMO* and *AFAIK* so there could well be inaccuracies here and there, and there are no doubt things I'm not aware of yet, other methods and complications maybe, but hopefully some of it is of interest to someone somewhere. =)


BTW the site mentioned up top is one I made up to help users with WinMX probs, nothing special or anything by any means. I used to have more information directly for the blocked RR users available on the site at one point, but thought it best to not mention the actual ISP anymore in case it stirred any trouble. I know RR where made aware of the info as the users were phoning RR tech and causing mayhem a bit, mentioning 'the site' etc. I did have some interesting RR tech support excuses knocking around at one point, only one person afaik was able to get RR to admit the blocks were in place, the other tech responses were a joke tbh. This all happened around February iirc, I'm surprised it's took so long to come out (with the news of KaZaA possibly now blocked).


All good fun.
cheers.

db

http://WinMXHelp.d2g.com/

[db_]

whoops didn't realise it was so damn long, sorry!

[JackSpratts]

Originally posted by db_

So, the clients just need a feature that upon installation of the client it chooses a fairly random port to use, say, anything between 1024 and 5000, this creates a network that is not centralized around a single common port, this means ISP's will find it harder to control specific services through the filtering of the common ports used for the service.

hi db, it wasn't too long. i can't remember who told me this, it might've been jaan - it was a while ago, but this is something fasttrack was looking at for an upcoming release. what's taking so long i don't know, but it's probably related to their defensive posture and resouce allocation vis a vis the riaa.

- js.

[butterfly_kisses]

Originally posted by db_
whoops didn't realise it was so damn long, sorry!

That was lots of GOOD information...please don't apologise for posting it. as a matter of fact...THANK-YOU for posting that and for taking the time/trouble to go into the depth of detail that you did.

We can learn a lot from each other this way.

so again..I thank-you.

[Snarkridden]

Being on NTL with a 512k connection 24 hour and going like the clappers most times, I really needed this info..

http://homepage.ntlworld.com/j.buch...mx/blocked.html


I tired out the suggested port changes and the test system seems to work Ok into my server, up & downloads Ok, also searches Ok with the rest of the network, so could be a winner in the "awkward" situations, but as was said, only if the primary nodes talk on those ports too...

Nice post Db ... Took me a long time to read it, which means it was usefull, and not skipped like so many other long posts..

(Snark's short span of attention!)



Snark..

[alphabeater]

db, thanks from me too for the huge amount of detail you posted - it's the best i've seen describing actual experiences of port blocking. everyone in this thread is suggesting a lot of things i hadn't thought of at all.

looking at the posts above, an interesting set of problems are posed. a way of calculating a port number is needed which:

- is almost random
- cannot be figured out by a peer's isp
- can be figured out by another peer on the network
- is static for use with routers/firewalls

to stop isps creating lists of all the ips they own and which port needs to be blocked for each, at first i considered involving the gmt date in the algorithm, and changing all listen ports every day at gmt midnight. this would solve the first three problems in the list, but make it impossible to solve the last.

however, an answer could be found if some other detail apart from the ip address of each user were known to every peer in the network about each other peer. this could work, to a degree, for kazaa usernames (supernodes return both usernames and ips with search results, although the network is closed so i can't really check that for sure). if kazaa were to implement a port worked out with usernames, not ip addresses, then that could work.

for example,

if i'm alphabeater@KaZaA

the first five characters of my name are 'alpha'
if, for the sake of programming, we say a = 0, then there can be 25 (a nice number) letters in the alphabet (any numbers in the name could be left as they were).
so,
a = 1, l = 12, p = 16, h = 8, a = 1

in this example, we now have 38 (it's min 5 and max 125). multiply this by 80 and we have a number from 400 to 10000. if the result turns out as 2000 or under here, then it can be subtracted from 10000 to get the final result.

in this example, we're now using port 3040 for listening, and this port could be obtained by anyone who knew that the first five characters of my name are 'alpha', but not automatically listed by an isp.

this calculation could, again, be a lot more complicated, and the string used could change depending upon the p2p network layout, as long as any peer on the network needing to make a connection could calculate the other peer's port reliably it wouldn't matter. this technique also has the advantage of generating a static port for as long as the user keeps their identity on the network the same, even if the ip involved is dynamic, making it easier to configure firewalls and routers.

overall, i think known string-based port scrambling could be more effective than ip-based port scrambling.

[Scyth]

I still don't under stand why any sort of generated port number is necessary. Since an IP address is required and must be communicated somehow, why not simply include the port, a mere 2 bytes of information, in this communication as well.

[alphabeater]

Originally posted by Scyth
I still don't under stand why any sort of generated port number is necessary. Since an IP address is required and must be communicated somehow, why not simply include the port, a mere 2 bytes of information, in this communication as well.

ips are generally communicated across a decentralised p2p using the standard ports of that p2p. there's no way to send ports around if you can't find out which port your gateway peer is listening on.

if these ports are blocked, then this communication cannot take place. if everyone is using random port numbers, then there is no way for communication to take place, as there's no port open for either peer that the other peer knows and can access (try connecting two firewalled hosts on gnutella, the effect is similar).

ports even become confusing and annoying in an opennap-like network, although the problem is not as great as in a fully decentralised one. i feel that generating port numbers from data available to both sides of the communication, before any communication actually takes place between them, is the easiest solution available.

[Scyth]

Originally posted by alphabeater
ips are generally communicated across a decentralised p2p using the standard ports of that p2p. there's no way to send ports around if you can't find out which port your gateway peer is listening on.

How do you know what IP your gateway peer is at? Who/Whatever gives you its IP can also give you the port number.