Thousands of items in queue

Bugs and issues
#1

I’ve seen this issue brought up a few times, but I thought I’d add another account of it happening. It seems like the db’s get messed up / out of sync when I log in from a non-InSync computer and make changes to drive. When I log back on my other computer (the one with InSync), sure enough there’s a random number (in the tens of thousands) of items “In Queue.” My assumption is depending on which file I’ve updated, every file up the chain is considered “out of sync” and needs to be re-queued. This is completely unmanageable, as I lose hours re-creating insync dbs every time I try to work from home.

I don’t have a log because I had to just nuke and reset my insync installation and folder. Can you please shed more light on this issue? It’s incredibly disruptive, and without a fix on the horizon my team might have to migrate off this platform on to something else more stable.

#2

Hi @Daniel_Balmert what is your OS? :smile:

#3

Windows 7 (both machines).

#4

I have a similar but altogether worse issue.

I’m a bulk user of Google drive with over 700,000 items stored there. And over 15,000 changes daily.

I upgraded to 1.3.2 and now there are over 760,000 items to sync. BUT it’s worse, the sync is happening so slowly that the number is actually getting bigger over time. Right now I am at 756,000 items and its syncing about 1 per second. Where can I goto for help on this matter?

#5

Hi @Daniel_Balmert and @Marcus_Bennett I apologize for the extremely late reply. Can you kindly send in your logs (How to find the log files) along with the link to this post to support@insynchq.com?

cc @dipesh

#6

I am already in conversation with your support Engineer

So far there is no resolution.

To my mind this issue will only affect power Insync Users

However, this is exactly why I investigated, tested, and currently recommend
Insync, because the Native Google drive client does not perform well
with large file collections.

Now it seems, for me, Insync is behaving similarly.

Current status: I followed Dipesh’s instructions (re --skip-local-scan)
and he also has my logs. However right now the upload/
resynchronisation is still moving at a CRAWL. There are over 800,000
items to sync. And nothing at my local computer end is very busy
including Insync.

The actual behaviour of Insync now is that

a) when actually syncing files the rate never seems to exceed about 1 file per second
b) There is activity of syncing, by which I mean in the feed screen I can see itema are, about 1 per second sychronising. However I did notice some periods lasting several hours where Nothing was sychronising and there appeared to be no activty on the insync process either (Windows Resource monitor, filtering by insync*.exe, checking open files, disk read writes, cpu usage)

I am at your disposal to provide whatever further information you need to help resolve this issue.

Still Thousands of Items in Queue
#7

It has been several days and no word from Insync. Because my synchronisation is broken and I have practically speaking unlimited and fast upload and download bandwidth I made some experiments.

INITIAL PROBLEM
Was that Insync had over 700,000 items to sync and it was doing so at 1 per second

Actually both the local and Google ends had ALL the files so all that Insync needed to do would be to compare each file (however it does that) i.e. a file upload from the Local system or Download from the Google cloud not required

OF COURSE
If it is was comparing more than file size and date/time then it WOULD have to download every cloud file, caculate a checksum and compare it with the local file. But It may not operate that way

EXPERIMENT 1

Without Insync reinstall, I quit Insync
Then on cloud and local ends I DELETED the entire contents. So nothing at either end
Then I restarted, and initially EVEN AFTER scanning changes messages had gone it decided there were still 798,000 items to sync!!
Eventually after some hours it changed and it then syncd SHARED ITEMS without asking (not much, say about 2 GB) and then it finished.

I went to the web google drive and removed i.e. disassociated all shares

EXPERIMENT 2

I tried a clean install

So I stopped the client
I uninstalled the client
Then I made sure bout local and cloud google drive areas are empty
Then I manually tried to clean up some files in ProgramFiles directory but they were locked
So reboot
On reboot clean out Insync directory
Install the latest 1.3.2 client again
Start up
Wait for scanning changes message to go
Now in a stead state nothing at either end
I copied into my gdrive local area my books directory which is 8.5GB size in 24,812 files
I left it synchronising i.e. copying upto host
Upload speed was always less then 4MB /sec (bytes not bits)
For small files it was almost zero
I screenshotted the progress to check speed
At 11.40 24988 items and at 13.30 20198 items, so that is 0.72 items per second
Basically useless

EXPERIMENT 3

I am installing the native Google Drive! Yes, I know, desperate times
Before installing Insync is stopped
Both local and cloud are cleaned out
Many items exist in Google Drive Bin
I’ve tried cleaning BIN, but the google bug of lots of items in BIN wont immediately delete still seems to exist.
This might be relevant to explain that
Google drive is now started
It is /preparing for sync/ claiming to have found 320,000 items. This is bogus because all the items at the cloud end are in the BIN.
Edit …
After going past the 320K items to sync mark it finally correctly finds about 25K items to sync
1.5 hours later, native google drive client, uploaded 25K items, that is 8.47GBytes.

So far then the native Google client works and Insync does NOT.
Not what I was expecting.

#8

EXPERIMENT 3

contined

So last night I loaded up 101GB of files , comprising over 113,000 files, 11,594 directories
The google client started at 17.49 and today at about 12.23 it finished
Let’s call it 19 hours, or 1140 minutes which is 1.65 files per second and that is an AVERAGE upload speed of 1.56 MBytes per second.

this is several orders of magnitude greater than Insync Pro.
So how about it Insync, can you provide me with a working Insync Pro please that can cope with these larger data volumes and file collections?

To repeat, I was a very happy customer. Now your product upgrade has rendered the product useless to me.

#9

Got the same problem now in 08/2016 after 3 months of non-syncing on one device is there some solution please help, I am basically in same situation as the user here, switch to insync due to large number of files which google drive sync wasnt able to handle in past but now the situation is as opposite.

Marek

#10

In the same situation here. Pro user and looking at going back to regular gdrive.

#11

its solve for me now … I noticed that Google Photos folder where I backup lots of pictures started to synchronize, I unchecked it and after some time it got into the normal, still I think Scanning for changes take a long time