Backing up and Syncing

I really like many of Google’s products. Of the ones I use heavily, they’re well-thought-out and designed. My favorite is probably GMail, second is Google Drive AKA GDrive.

I’ve used MacBookPros for years and backed them up with TimeMachine (which is also a really great product). The major problem with TimeMachine is vendor lock-in. If you’re not using OSX, you’re SooL.

Years ago, I started getting more into Android development and built a Linux Mint/Windows Gaming and “productivity” rig. I actually ended up using Mint nearly 100% of the time, switching to Windows only for games that required the OS.

However, the problem of file synchronization soon appeared. Occasionally, I would travel and work on my OSX laptop and if I didn’t remember to copy everything over, my files wouldn’t be there. I use Github and Google’s Source Respository for things, but if you forget to grab them or your key before boarding a flight, you can’t work on these projects.

Enter, Google Drive

OK, so I’m terrible at preparing my work machines for travel. How can I work around that?

Luckily, Google provides a Drive app as a way to transfer files continuously from your Google Drive to your workstations. For a product that started off as a hack way to sneak data into GMail to store regular files, it’s really come a long way. Seriously, before Google Drive, people were storing files as emails in the inbox.

However, the clients only exist for Windows and OSX. As of this writing, they do not have a Linux implementation. Womp-womp.

The field

As a workaround, I have been using Overgrive for quite some time. While it is somewhat passable and does the job (when it wants to), the site is sketchy and the software is non-responsive at times. For some reason, GNUCash causes Overgrive to freak out and refused to work until some combination of Trash empty and restarts are performed. I’ve occasionally had to re-authenticate the app with Google Drive.

Other problems

Additionally, it doesn’t have an integrated sign-in flow, it uses the Copy-Paste method, which is not user-friendly. This user-unfriendliness is actually how they work around the problem below.

While not that expensive, my annual license expired. I figured it would be a good project to build my own.

A project

In my travels through Google’s cavernous–and oftentimes poorly illuminated–documentation, I believe I’ve found the reason why most of these apps are terrible.

In order to use the proper OAuth2 mechanism with localhost redirect_uris for obtaining refresh tokens, you need to create a Google Cloud project, application client credentials, and use those to make requests on behalf of the user.

Because this type of application needs to access the entirety of the user’s Google Drive (what good is backup software that can’t read and write?), the required scope is “https://www.googleapis.com/auth/drive"

To make a drive sync app, I would need to use the inotify library on the local machine and Google Drive’s Drive Activity API, but they’re both listed as Restricted and not Recommended as some of the other scopes. So what does Restricted mean?

Restricted Applications

What does being restricted mean? It means you need Verification to distribute your app and let others use it.

That popout message reads

In this mode, your app is available to any user with a Google account. External apps that request sensitive or restricted user data must first be verified by Google. This process may take 4-6 weeks. In addition, if you selected restricted scopes, you’ll need to go through an independent security review, which can cost USD $15k-$75k. Learn more.

The user cap limits the number of users that can grant permission to your app when requesting unapproved sensitive or restricted scopes. The user cap applies over the entire lifetime of the project, and it cannot be reset or changed. Verified apps will still display the user cap on this page, but the user cap does not apply if you are requesting only approved sensitive or restricted scopes. If your users are seeing the “unverified app” screen , it is because your OAuth request includes additional scopes that haven’t been approved.

– Google Cloud OAuth consent screen configuration

In order to get that 100 user cap lifted, the app needs to be verified by an independent body and this usually costs–at minimum–$15,000.

Let’s say I get lucky and find someone at the bottom price of $15k, if I sold this application for $5 for an annual license, I would need 3,000 customers to make a purchase before I broke even in the first year. This doesn’t include refunds, credit card fees, taxes, etc.

For the casual side-project developer with a full time job, this is an insurmountable barrier to entry and is likely why few people would go through the effort of learning GTK and putting a Go application together using the poorly documented and unfinished(?) GoogleAPIs for Google Drive.

What can we change?

Look, I get it. Apps like this get full run at your data and that’s really dangerous. Backup apps could steal, delete, or hold data hostage. Danger abounds. However, there has to be a better way.

Short of automated tools to verify the app doesn’t exfiltrate files or delete them or encrypt them, I’m not sure what a good solution would be. Even if you had a test harness to verify these applications don’t do bad things, these tests can’t run forever and sneaky malware makers may set timers or file counts to sneak passed the testing process.

How do other providers get through this? Do they just write their own to work around these security issues?

This is probably the reason there are no backup utilities for Google Drive for Linux. Google has made it practically impossible to do so correctly.