Add throttle per torrent

[rakshasa]

Add a throttle for each download, this should be a seperate ThrottleControl? from the root throttle. Consider whetever only uploading needs this throttling.

[anonymous]

yes, please add this feature. not all torrents are of the same importance.

and using several instances of rtorrent makes the multi-torrent-support pretty preposterous.

[rakshasa]

For the moment, prioritizing torrents will propably be implemented by having different priorities when allocating uploads.

[anonymous]

i 2nd this! :P

[anonymous]

I agree, please add this feature.

[mizoo@risp.pl]

great feature it will be :->

[Poidel]

That feature, properly implemented would be very usable for me. Many trackers I use are very hard to get a good ratio (i.e. a good long-term relationship) with, so I have to prioritize manually. This has the disadvantage of not being very efficient, and wasting "valuable" upload bandwidth.

Ideally, the upload/download ratios could be prioritized so that 100% of the allocated max_upload_rate is used, but with priorities/individual throttles.

[anonymous]

I agree.

Some torrents have ratios and some do not. I would like to have the ability to give the ratio based torrents more bandwith.

BTW, great program!

[anonymous]

I was looking for a way to add a request for upload priority when I saw this ticket. This is a great feature and save me the trouble for using multiple instances of rtorrent.

[Grey]

I too would welcome this feature, as a seeder on a number of private trackers, it would be made considerably easier if I were able to prioritise the seeding torrents to use the upload bandwidth over and above those few things I happen to be downloading.

[s0cke]

Yes, please add the throttle feature per torrent :) The priority thing is not always enough, it's better to set specific up/download speeds. Maybe the keys 7/8 for the upload and 9/0 for the download.. Anyway it would be really nice if you implement this feature soon! rTorrent just rocks hell :D

[anonymous]

Agreed. This would be a very good function to have when some torrents are ratio-based.

[anonymous]

Any updates regarding this feature? Thanks.

[anonymous]

This is acctually the only feature I miss after switching from Azureus. It would indeed be a very welcome feature.

[dan]

i miss this feature, too (without having used azureus before, heh). prioritizing a torrent is no exact possibility to determine the upload speed of each torrent. and with upload bandwith throttling you could estimate the effect on the download-speed of other torrents much better, which is a really important consequence for me...

[lvella]

I would appreciate this feature, but I would have better use for downloading throttle than for uploading throttle. I believe both should be implemented.

[anonymous]

That would be great. sometimes its a must because of different ratios. thx

[josef]

I've made a very experimental patch for this at  http://tk.ttdpatch.net/~jdrexler/throttles.diff - read the notice at the head of the file before applying it.

Define new throttles in .rtorrent.rc with

throttle=<name>,[<uprate>][,<downrate>]

e.g.
throttle=trackerA,50

Attach it to the selected download with the ctrl-X or xmlrpc command

d.set_throttle_name=<name>

Once attached, it will show up in the download info and after the ratio in the download list if your display is high/wide enough. When a custom-throttled download is selected, the throttle keys adjust the custom throttle instead of the global one. The custom throttle is inactive if the global throttle is not also throttled (i.e. set to 0), so you must make sure to set the global throttle though it may of course be arbitrarily high.

If the custom throttle bandwidth is significantly below the global throttle you should manually reduce the upload slots of the affected downloads (keys '1'/'2' in download view) to make sure rtorrent doesn't try to distribute the low bandwidth to too many peers. The patch still needs some work to make this happen automatically.

[aszwet1 at gmail dot com]

i completely agree.

josef, how exactly do you install a patch in rtorrent? do i have to set up rtorrent with the --with-xmlrpc-c command? and do i have to have apache running and all that for it to work?

[anonymous]

To install the patch, you need the right SVN revision checked out. The right one is the one the patch was built against, which you can tell by looking at the patch file. Where it says "--- rtorrent/doc/rtorrent.1.xml (revision 1023)" means you need revision 1023, or whichever one it mentions.

Then you go to the directory holding the "libtorrent" and "rtorrent" subdirectories and enter

$ patch -p0 < throttles.diff

If there are any errors or warnings while patching, you didn't check out the right revision, or the files were already modified. In that case do a clean checkout again, or fix the conflicts manually if you can. If you like, have svn update to the latest revision with "svn update" now.

Then you can simply compile it.

PS: The latest version of this patch adds support for IP-based throttling, and the special throttle name "NULL" to mean completely unthrottled even in the presence of a global throttle.

throttle_ip=throttle name,host
throttle_ip=throttle name,network/prefix
throttle_ip=throttle name,start,end

[anonymous]

I forgot to repeat that you also need to follow the steps in the notice at the head of the patch file before you can actually apply it.

[anonymous]

Can this patch be used to give as much upload as the connection allows to a specific peer ip using throttle_ip? I want that connection capped before splitting the rest of the upload to the other peers.

[anonymous]

"throttle_ip=NULL,<your home IP>"

Can <your home IP> in the above line be replaced with my domain (xxxxx.no-ip.net)?

[josef]

Yes, although it's only resolved at the time that statement is executed, so if it changes afterwards rtorrent won't notice unless you restart. Though you could put this statement in a schedule to have it executed every 10 minutes for example, I guess.

[josef]

I've updated the patch for the latest API changes, so it will apply cleanly to 0.12.0/0.8.0 and the latest SVN revision. You no longer need to do the "svn mv" commands before applying it.

I've also altered the user interface a bit, the download list no longer shows the custom throttle rates, only the name. The full custom throttle information is available in the download info window. In that window, you can use the usual throttle keys to adjust the custom throttle instead of the global throttle, outside of the download window the keys now apply to the global throttle always. And I've added the key Ctrl-T in the download list to toggle the custom throttle of a download, it cycles through all available downloads.

[Maxim]

Is it possible to do something similar to this patch but for following case: My ISP provides me internet connection with something about 512k bandwidth cap, gray ip (behind the NAT) and traffic in the between customers is about 10x to 40x times faster then internet connection. so - it's becoming popular to do "cooperative" downloading - group of users behind NAT are downloading the same torrent. There are 2 problems however. 1 - we are behind nat, so we are reporting to the tracker the external ip of the ISP router.

[Maxim]

2 - even if we have chunks locally (inside the nat) bittorrent client is often downloading this chunk from outside, even having the proper local peer in peer list. Especially often this happening when the number of peers reported by tracker is really large.

First part of the problem is partially addressed by some trackers - they dealing with some ISP and home lans modifying torrent file for the users from specific ip networks, adding the address of this ISP or lan tracker to the tracker list in the torrent, so the client will connect to local tracker as well and obtain gray ip addresses of the other peers from same lan.

Second part is more complicated - it's would be nice to have the option to not download from Internet chunks, which are available from local peers (ip in some list of networks).

And thank you for the previous patch:)

[anonymous]

Is there an updated version of this for 0.12.1? I'd like to give it a try. One additional feature that might be worthwhile is a per-tracker option for a throttle as well, so all torrents that use a particular tracker would fall under it by default.

[josef]

I've updated the patch for the last SVN revision. The new URL is  http://ovh.ttdpatch.net/~jdrexler/throttles.diff - the old one will redirect here until it stops working.

Replying to anonymous:

One additional feature that might be worthwhile is a per-tracker option for a throttle as well, so all torrents that use a particular tracker would fall under it by default.

You can probably already do that, by setting custom1 (or any of custom1..5) for downloads with that tracker (e.g. by using a separate watch directory), and then calling d.set_throttle_name in an on_insert trigger using a branch based on d.get_custom1.

[Mikko]

isn't this patch ready to be merged yet?? :(

[josef]

It works, but there are some aspects of the IP filter design that need to be made more general before it will be ready to be merged.

[anonymous]

Replying to josef:

It works, but there are some aspects of the IP filter design that need to be made more general before it will be ready to be merged.

Hi

Could you please be more specific with that?

And much thanks for your patch

[anonymous]

I'm having some problems to install this patch.

even though I have:

$ nm /usr/local/lib/libtorrent.so | grep slave

U _ZN7torrent16ThrottleInternal12create_slaveEv

00023694 T _ZN7torrent8Throttle12create_slaveEv

I get:

configure:21762: checking for XMLRPC-C configure:21806: g++ -o conftest -g -O2 -g -DDEBUG -I/usr/local/include -I/usr/include/sigc++-2.0 -I/usr/lib/sigc++-2.0/include -I/usr/local/include conftest.cpp -lncurses -L/usr/local/lib -lsigc-2.0 -lcurl -lgssapi_krb5 -lkrb5 -lk5crypto -lkrb5support -lcom_err -lresolv -lidn -ldl -lssl -lcrypto -lz -ltorrent -L/usr/local/lib -lxmlrpc_server -lxmlrpc -lxmlrpc_util -lxmlrpc_xmlparse -lxmlrpc_xmltok >&5 /usr/local/lib/libtorrent.so: undefined reference to `torrent::ThrottleInternal::create_slave()' /usr/local/lib/libtorrent.so: undefined reference to `torrent::ThrottleInternal::~ThrottleInternal?()' /usr/local/lib/libtorrent.so: undefined reference to `torrent::ThrottleInternal::disable()' /usr/local/lib/libtorrent.so: undefined reference to `torrent::ThrottleInternal::ThrottleInternal?(bool)' /usr/local/lib/libtorrent.so: undefined reference to `torrent::ThrottleInternal::enable()' collect2: ld returned 1 exit status configure:21812: $? = 1 configure: failed program was: | /* confdefs.h. */ | #define PACKAGE_NAME "rtorrent" | #define PACKAGE_TARNAME "rtorrent" | #define PACKAGE_VERSION "0.8.2" | #define PACKAGE_STRING "rtorrent 0.8.2" | #define PACKAGE_BUGREPORT " jaris@ifi.uio.no" | #define PACKAGE "rtorrent" | #define VERSION "0.8.2" | #define STDC_HEADERS 1 | #define HAVE_SYS_TYPES_H 1 | #define HAVE_SYS_STAT_H 1 | #define HAVE_STDLIB_H 1 | #define HAVE_STRING_H 1 | #define HAVE_MEMORY_H 1 | #define HAVE_STRINGS_H 1 | #define HAVE_INTTYPES_H 1 | #define HAVE_STDINT_H 1 | #define HAVE_UNISTD_H 1 | #define HAVE_DLFCN_H 1 | #define _FILE_OFFSET_BITS 64 | #define USE_VARIABLE_FDSET 1 | #define HAVE_SYS_VFS_H 1 | #define HAVE_SYS_STATVFS_H 1 | #define HAVE_SYS_STATFS_H 1 | #define FS_STAT_FD fstatvfs(fd, &m_stat) == 0 | #define FS_STAT_FN statvfs(fn, &m_stat) == 0 | #define FS_STAT_STRUCT struct statvfs | #define FS_STAT_SIZE_TYPE unsigned long | #define FS_STAT_COUNT_TYPE fsblkcnt_t | #define FS_STAT_BLOCK_SIZE (m_stat.f_frsize) | /* end confdefs.h. */ | #include <xmlrpc-c/server.h> | | int | main () | { | xmlrpc_registry_new(NULL); | ; | return 0; | } configure:21827: result: failed configure:21829: error: Could not compile XMLRPC-C test.

[anonymous]

You're using an old libtorrent which lacks this patch. Either you didn't "make install" it properly, or there's another version of it lying around, perhaps in /usr/lib.

[anonymous]

No, it's specific to the XMLRPC. Disable support for XMLRPC and it will compile just fine.

[anonymous]

Correction, it will configure, but still fails to compile. The current patch does not correctly update the makefiles to properly build:

(libtorrent) throttle_internal.cc (rtorrent) element_base.cc

That will probably fix all the issues...

[anonymous]

Did you forget to run autogen.sh after applying the patch?

[anonymous]

Never had to before, didn't run it this time either.

[anonymous]

Well, run it then and re-run configure.

[portwolf]

tryied the patch, but it seems to have some errors(while compiling, after patching) i would apreciate, if you'd make it official and import this!! this is the only thing i miss on rtorrent! thank you though!

[anonymous]

I've updated the patch for the latest revision. And I'm still waiting for rakshasa to comment on it or maybe even apply it.

[josef]

Sorry, forgot to log in. Anyway, the updated patch is at the same URL as before,  http://ovh.ttdpatch.net/~jdrexler/throttles.diff

[anonymous]

Josef, the latest patch seems to be missing some files - throttle_internal.h and throttle_internal.cc

[josef]

Whoops, fixed. It was also missing element_base.cc.

[anonymous]

Replying to anonymous:

To install the patch, you need the right SVN revision checked out. The right one is the one the patch was built against, which you can tell by looking at the patch file. Where it says "--- rtorrent/doc/rtorrent.1.xml (revision 1023)" means you need revision 1023, or whichever one it mentions. Then you go to the directory holding the "libtorrent" and "rtorrent" subdirectories and enter {{{ $ patch -p0 < throttles.diff }}}

When I attempt to apply the patch using the above instructions I get prompted for which file(s) I want to patch. The man page for the "patch" command seems to say that "patch" should be able to figure out which files to patch. Is this a problem with the current version of the patch? If not, what am I supposed to answer in response to the prompt(s)?

[anonymous]

Never mind. I just had to rename the subdirectories.

[Ammler]

Are there any chance this patch goes to trunk in near future? This feature is really important for Webseeding...

[morganw]

I'm noticing that the throttles seem to max out at about half of their assigned upload bandwidth:

Throttle: mvgroup2 [Max 20/--- KB] [Rate 10.5/--- KB] Throttle: mvgroup3 [Max 10/--- KB] [Rate 5.1/--- KB]

Increasing the throttle cap increases the throughput, although it still seems to be about half. I've tried using two different creation commands and see the same result with both:

throttle = mvgroup2, 10 throttle = mvgroup2, 10, 0

Anyone else notice this?

Running 0.8.2 for ages and now 0.8.4 since its release on FreeBSD with kqueue.

[anonymous]

What's your global throttle setting? The individual throttles are still bound by it, and in case of insufficient bandwidth, the throttles will all get about the same fraction of their allocated rate.

In other words, if the global throttle is set to half of what's needed to satisfy all throttles, each throttle will only get half of its allowed rate.

[anonymous]

Global is set to 50. Not all torrents are assigned a throttle, but the limits of the active individual throttles are 20, 20, 10, and 10. So when the global limit is hit, the individually throttled torrents will be allocated some fraction of the bandwidth. Interesting!

One feature that might be nice is what ALTQ calls "borrow", where the limit is only enforced if there is a shortage of bandwidth. That way I could assign throttles to every torrent but always make sure that the full 50 k/s is used.

[anonymous]

well... you could set the throttles to 100, 100, 50, 50. And the global throttle to 50. Then all throttles will use no more than 50 k/s, but any of them can use the full bandwidth if it's available. And if there's contention, the first two will get twice the share of the others.

In theory... I haven't actually tested this much. If you post the current and max rates of all throttles (including global), I can tell if it's working or not, at least.

[josef]

I've made another update to the patch. Throttle rates are now set separately for upload/download like this:

throttle_up=name,rate
throttle_down=name,rate

(Though for now it has the old command too but I imagine that would disappear when the patch gets integrated into rtorrent.)

Also changed and removed some of the xmlrpc commands, in case you were using them. To print a download's throttle settings, you'd now use, e.g.,

to_throttle=$get_throttle_up_max=$d.get_throttle_name=
to_kb=$get_throttle_up_rate=$d.get_throttle_name=

In other words, the get_throttle_{up,down}_{max,rate} commands now take the throttle name as argument, so you can get throttle rates even if no particular download is associated with them (e.g. for IP-based throttles). The new to_throttle conversion displays the throttle maximum properly, as "---" (no such throttle) or "off" (unthrottled) or number of kilobytes (no decimal point, unlike to_kb=).

The other changes were mainly internal cleanups and refactoring.

[Ammler]

Wouldn't it make sense to integrate it in the ratio group handling?

[anonymous]

Yes, I think rakshasa said he would look into that (presumably some time after he reviews and applies the patch)...

[profetasdelmetal@gmail.com]

Probably the most important feature rtorrent lacks. It would be nice to see this one.

[anonymous]

Up to which rtorrent/libtorrent version this is working? I am not so experienced with patching, i dont know how to find out which revision i have. Trying with libtorrent 0.12.4 and rtorrent 0.8.4. If i go to source of rtorrent it cant find the file to patch.

[anonymous]

It works with 0.12.4/0.8.4 or the latest svn revision. If it can't find the file to patch, you're running patch in the wrong directory, or try giving the -p1 option. Or make sure the directories are named "libtorrent" and "rtorrent" without the version suffix.

[simon <simonbcn@gmail.com>]

Thank you very much, josef.

Replying to anonymous:

...One additional feature that might be worthwhile is a per-tracker option for a throttle as well, so all torrents that use a particular tracker would fall under it by default.

+1

[simon]

Replying to profetasdelmetal@gmail.com:

Probably the most important feature rtorrent lacks. It would be nice to see this one.

+1

[anonymous]

The per-tracker config can probably be done using the new groups feature in 0.8.4.

[simon]

Replying to anonymous:

The per-tracker config can probably be done using the new groups feature in 0.8.4.

Yes, confirmed.
Example:

throttle_up = baja, 20
throttle_up = media, 50
throttle_up = alta, 100

schedule = watch_directory_1,5,5,"load_start=~/torrents/dir1/*.torrent,view.set_visible=vista1,d.set_throttle_name=media"
schedule = watch_directory_2,5,5,"load_start=~/rtorrent/dir2/*.torrent,view.set_visible=vista2"
schedule = watch_directory_3,5,5,"load_start=~/rtorrent/dir3/*.torrent,view.set_visible=vista3,d.set_throttle_name=alta"

Thanks

[belgarion]

I also think the individual torrent throttle is the most important missing feature. Despite TorrentFlux? being a ressource hog compared to rtorrent, I'm almost tempted to go back to TF just for this feature :(

Any ETA ? Any feedback on possibly merging and supporting the patch ?

[anonymous]

Replying to belgarion:

I also think the individual torrent throttle is the most important missing feature. Despite TorrentFlux? being a ressource hog compared to rtorrent, I'm almost tempted to go back to TF just for this feature :( Any ETA ? Any feedback on possibly merging and supporting the patch ?

thats honestly why I'm here. I'd also like to have per-torrent end behavior. global ratio management is very limiting.

[anonymous]

I'm having trouble convincing myself to use rtorrent until this feature gets implemented, it's too essential to go missing, especially with private trackers that have strict ratio requirements.

[rakshasa]

Commited Josef's patch.

[anonymous]

This is so confusing. I just tried setting a throttle on a torrent and it went ahead and used all of the bandwidth. I'm a cable subscriber, I never know my maximum upstream bandwidth. Having an individual throttle based on a global throttle is not of much used to me. I just want to limit some torrents to a maximum of 25KB/sec.

[anonymous]

As it says in the docs, the individual throttles only work if the global throttle is not unlimited.

If you want unlimited global throttle but limited torrent throttles, just set the global throttle to more than your maximum physically possible, i.e. 13 MB/s for 100mbit ethernet. Then it'll be effectively unthrottled but individual throttles still work.

Though you should set the global throttle to less than the upload bandwidth anyway to avoid long queues in the cable modem slowing down your downloads.

[anonymous]

In which version did this feature get implemented?

[anonymous]

latest svn, or 0.8.5

[pidori@xui.ru]

Не, пиздец, нахуевртили, а ты сди, думай что тут за иероглифы. Нет, бы бля, сделать нормальную доку. Да еще на русском. Хотя, какая нахуй дока на русском, если эти ароды и на англицком ни как не напишут.

[anonymous]

кому надо тот переведет. thx a lot for this feature to Josef and rakshasa

[anonymous]

I would like to thank rakshasa for such a great program and josef for a brilliant feature/patch.

Thanks guys.

[dqv79@free.fr]

This is a feature i waiting for a long time, but i don't understand how it works. Can someone explain the key short-cut tu use some customs throttle please ?

[deysmacro]

Quick per torrent throttling guide

Note1

This guide is tested working with libtorrent 0.12.5/ rtorrent 0.8.5

Note2

Set your global upload/download limit first in .rtorrent.rc; make sure it is 10x the amount of your speed so that the throttling capability would become effective

e.g:
# my download speed is just 1M
download_rate = 10M

# my upload speed is just 1M
upload_rate = 10M

Note3

Before applying the torrent specific throttling capability, the said torrent must be in CLOSED mode

throttle-limit-name

Usually any name would do; just stick to easy one

your-throttle-limit

The limit is in KiB; numeric only

---

Now, in .rtorrent.rc, add the following line:-

format:
throttle_up = throttle-limit-name,your-throttle-limit
throttle_down = throttle-limit-name,your-throttle-limit

e.g:
# use global download limit and apply upload throttling only with name uplimit; 200KiB limit
throttle_up = uplimit,200

# use global upload limit and apply download throttling only with name downlimit; 150KiB limit
throttle_down = downlimit,150

# both upload/download throttling with name updownlimit; 140KiB
# any upload/download speed combination would do
throttle_up = updownlimit,140
throttle_down = updownlimit,140

# to re-use global upload/download limit
throttle_up = global,0
throttle_down = global,0

# per tracker throttling applied automatically with upload throttling only
# just use the format as the above for more throttling control

throttle_up = baja,20
throttle_up = media,50
throttle_up = alta,100

schedule = watch_directory_1,5,5,"load_start=~/torrents/dir1/*.torrent,view.set_visible=vista1,d.set_throttle_name=media"
schedule = watch_directory_2,5,5,"load_start=~/rtorrent/dir2/*.torrent,view.set_visible=vista2,d.set_throttle_name=baja"
schedule = watch_directory_3,5,5,"load_start=~/rtorrent/dir3/*.torrent,view.set_visible=vista3,d.set_throttle_name=alta"

---

Ok, once the lines have been added, save and quit .rtorrent.rc and open rtorrent
At the main window, select which torrent that you want to apply the limit and when inside the torrent window, Ctrl + x and type the following:

d.set_throttle_name=throttle-limit-name

And press Enter. Again select the said torrent where at Info tab > Throttle: you would see the throttle-limit-name if it is applied correctly and you can control the limit using the same global throttle keys

And in case you want to bypass the throttle limit even in the global, just type:

d.set_throttle_name=NULL

Just re-apply throttle-limit-name if you want the torrent to be throttled again

Note

IP based throttling is not in here because I don't use it and I don't quite know how it works. Anyone wants to add it please do so.

Well, that's it from me. If anyone wants to add or correct this post, please don't hesitate to do so as everyone here is learning, including me.
With that being said, happy peering and throttling your torrent.

Thanks for reading this and thanks for your time.
Until then, have a good day everyone :)

[anonymous]

First a short layout what I want to do with rtorrent:

My computer has a couple of network interfaces. One is always on (but ToS and bandwith limited), the other is on most of the time, but gets a new IP approx. once a day. Now I want to exclusively use the later for torrenting, thus effectively stalling torrent activity until the later interface is available. I used to run a combination of a shell script (fetching the IP and updating the host file accordingly) and updating the binding in rtorrent on a regular basis. Worked, but not very elegant, as I needed to run scripts "outside" of rtorrent.

Now a couple of questions:

Does this custom throttle only effect peer traffic, i.e. is tracker and DHT communication not affected? Is there another special throttle like the NULL throttle, but reducing the speed to zero ("0" is afaik unlimited, right?)

Thanks for any hints or tips!

[anonymous]

It affects peer traffic, UDP tracker traffic and DHT traffic. It does not affect TCP (http) tracker communication because that's done via libcurl and IIRC not throttled at all.

There is no way in rtorrent to set the speed to zero.

The easiest way to achieve what you what would probably be to stop all downloads when the interface goes down, and restart them when it comes up. It will probably be useful to put all downloads you want to be activated in a new (invisible) view for easier selection via d.multicall.

[anonymous]

Replying to anonymous:

It affects peer traffic, UDP tracker traffic and DHT traffic. It does not affect TCP (http) tracker communication because that's done via libcurl and IIRC not throttled at all. There is no way in rtorrent to set the speed to zero.

Thanks for the details!

The easiest way to achieve what you what would probably be to stop all downloads when the interface goes down, and restart them when it comes up. It will probably be useful to put all downloads you want to be activated in a new (invisible) view for easier selection via d.multicall.

Short question: I guess with the ip_tick command I could somehow get the external IP adress accesible within rtorrent, now is there a way to build a evaluation like "if ip != member of range1 stop all torrents, else start all torrents"?

[brelie]

Hi, I am not that familiar with rtorrent yet, how will this affect attachment file affect the system? Brelie,  professional resume writer