Limit number of parallel uploads

Is it possible to set max number of files that are uploading simultaneously. Right now it is 12 on my laptop

I have a lot of large files (50GB+), and when 12 of those files are uploading it takes forever to upload them all, and if connection breaks for some reason, they all start uploading from zero, its really annoying.

If it is not currently possible, would you consider adding this feature in the future.

Thanks

10 Likes

This would be a welcome improvement.

1 Like

I’m having the exact same problem as the original poster, the uploads take forever and sometimes the connection breaks and they have to start again. For some files this has happened several times now. I’m reaching the point where I’m going to have to complete my uploads via the web browser interface, which defeats the point of having paid for InSync. I looked in vain for an option to configure the number of parallel uploads, it seems to me an obvious feature, but I note other threads here going back years with no action taken to make this an option. Surely it can’t be that difficult to add? I would much prefer to have a smaller number of files transferring more quickly.

1 Like

Further update, I’ve given up trying to get my large files to sync when copied in all at once, copying them into the synced folder one at a time is working and getting them onto OneDrive. Having spent literally an entire day trying persuade the last batch of files to sync when all were copied in at once, removing them from the folder and copying them back in one at a time is givng upload speeds that will probably allow me to complete the task in about an hour. So the ability to limit the number of simultaneous uploads is critical if you work with large files.

Also, looking around the forums I see that this was promised as a feature for v2 of Insync in 2016. Where did it go.

Furthermore, I have found that if I pause the large uploads for an hour or two because I need the bandwidth for work, they frequently re-start from 0. Again, if less were uploading at once, this would be far less of an issue.

Sorry for the rant, but feeling a bit short-changed at the moment having recently purchased InSync only to have it fall over this badly the first time I try to use it anger, when a seemingly simple feature that ought to have been there from the start would solve the problem.

1 Like

Hi @tim1mw,

I apologize for the trouble you’ve experienced with a large upload and I understand your sentiments about your experience. We would like to look into the cause of the slow sync and what’s causing the bottleneck in your sync progress.

Please send your logs with the link to this post and include the following files in the folder:

  • logs.db
  • out.txt
  • data folder
  • live folder

If these are too big for an attachment, feel free to send us a link so we can download these.

Hi @mia, I’m not @tim1mw but wanted to clarify - at least in my view - that the slow syncing isn’t due to a bug in Insync. It seems to actually be a feature, and related to the change made in v3 that uploads 12 files at once as @pedja1 mentioned in the OP.

The upload speed of most people’s internet connections is only a fraction of their download speeds, which usually results in 10Mbps/sec uploads or less. Then split that small amount of bandwidth over 12 large files at once, and it’s more likely that the file times out or the user has to shut down their computer or put it to sleep before the file uploads. And every time I’ve had to do that in the past, Insync v3 will start the upload over again.

The only way I was able to get my 100gb file to upload was to change my power settings so that my computer never went to sleep until the file was uploaded. Not an optimal scenario.

In my opinion, 2 things need to be fixed:

  1. Reduce the number of simultaneous downloads (whatever Insync 1.X was set at seemed to work well for me)

  2. Have Insync resume syncing a file if the computer gets shut down or put to sleep instead of starting over again. Insync 1.X seemed to do this well enough.

1 Like

Thank you for your feedback, @moeschl. I appreciate it! Let me forward this to our engineers so we can work to improve Insync’s sync speed and performance. We initially deployed the feature to sync 12 files at a time to reduce the CPU consumption but it does seem like it also affected the overall speed, which we will look into further. :slight_smile:

1 Like

This would be a very helpful feature. I spend the whole day with InSyncq turned off because of it.

The OS is MacOS Catalina

Hi,

I got the same issue. You are literally KILLING my i9-9900 rig with 12 parallel uploads?!? WHAT’S THE POINT? Why is it so difficult to add one simple swithch to limit number of downlads and uploads and limit the bandwidth? If you upload like 12 50GB files this way your software works is just pointless and harmful my the performance of my PC.

I wholeheartedly agree that the ability to throttle either uploads OR downloads would be a welcome feature.

This is for more than just bandwidth. On laptops, battery life is a concern, so it would be ideal if Insync could throttle itself for battery life.

Here’s what my ideal implementation of the feature would look like:

  • under App Settings, there is a section labeled “Performance.”
  • Under Performance, there are three options
    • Limit concurrent uploads: (which allows a numeric input from 1 to 12)
    • Limit concurrent downloads: (which allows a numeric input from 1 to 12)
    • and Enable battery saving mode.
  • Battery saving mode automatically limits concurrent uploads and downloads to perhaps 3 or 4, and additionally limit the rate at which Insync scans the drive.
  • And lastly, there is an option when right-clicking the Insync icon to enable/disable battery saving mode. (AND to pause/unpause syncing… but that’s unrelated to this thread.)
1 Like

@topjes I can imagine the trouble this has been causing you and your machine. I’ve raised this concern (and request) to our engineers to see how we can align this in our pipeline.

@Drakinite Thank you as always for sharing your insight. This has also been forwarded to our Product Team :slight_smile:

Hey,

any idea when this is going to be implemented? Need this very urgent. As I constantly have to sync big files it takes forever when it tries to upload 12 3Gb Files at a time. Because of consuming all of my bandwidth during upload (which I cannot setup neither) it is quite impossible to work meanwhile. Therefore I have to quit the sync after some time…which causes the “half-way” upload (of multiple gigabytes…) to be aborted and useless :frowning:

The use case was the one why I bought the software -.- and I am currently only medium happy with it without being able to scale the number of parallel upload on myself (which may be from my own developer background “only an if statement”) and to limit the used bandwidth.

1 Like

Hi @vankuchen :slight_smile: There is no ETA yet for this feature but I see how valuable this would be for you and the other users who have commented!

I will forward your feedback and use case to our Product team so we can determine how to incorporate this feature request in our near-future pipeline.

In my experience when a large file upload stops (e.g. due to shutting down the machine), it will continue where it left off. And indeed this matches Google’s documentation;

A resumable session URI expires after one week.
Source: https://developers.google.com/drive/api/v3/manage-uploads

For me parallelisation is great (when things upload one by one it takes forever when uploading lots of small files), but it would be nice to do some throttling on the upload speed perhaps (preferably as an option!). Most home internet routers don’t deal well with large uploads and they end up rendering the connection unuseable for anything else while the upload is running.

Both the NextCloud and official Dropbox client for example have an option to limit upload speed to around 3/4 of available upload bandwidth, and this seems to work quite well to keep this problem at bay.

1 Like

I’m trying to download large files from Dropbox and Insyc is completely unusable because it’s trying to download 12 files over a GB each at once. This makes no sense why it’s not a configurable option.

Thank you for the feedback and my apologies for the trouble, @John_Osborne. I’ll bring this up with our Product Team for review once more.

When I’ve created a support request I just noticed this post. I am also struggling with terrible performance for Dropbox account - to a degree I cannot replace Dropbox client with Insync.
For smaller files, it works semi-fine. But, if you have any bigger file (and by bigger I mean more than exemplary 50 mb) Insync is unusable.
It will try to download from the beginning, all the time, making all other folders unusable.
When you have more than 1 big file, it will start to resync 12 of them at the same time, which literally kills any decent PC/laptop

1 Like

Hi @Marcin_Bojko,

We’re received reports about big files hogging Insync resources and therefore lead to users being unable to sync the rest of the files in queue. I’ve forwarded the logs you sent via email to our engineers.

Thank you @mia
The issue takes all the living forces from a quite powerful client (pc). I had to revert to the native Dropbox client as NONE (literally none) of files bigger than a few hundred megabytes were downloaded properly.

1 Like

@Marcin_Bojko I completely understand your decision to revert to the native app! I am working closely with our engineers to pinpoint the root of this issue and deploy a fix as soon as possible.

1 Like