Access Google at IU My Drive from Research Desktop (RED)

On this page:


Overview

Following are methods for accessing your Google at IU My Drive space from Research Desktop (RED).

Note:

In most cases, UITS recommends using IU Globus Web App to transfer files to and from IU's research files systems (see Use the IU Globus Web App to transfer data between your accounts on IU's research computing and storage systems), but currently Google at IU My Drive accounts cannot be accessed using this method.

Recommended method (Firefox)

To access your Google My Drive account from RED, UITS recommends using the Firefox web browser, particularly if you need to transfer files larger than 1 GB.

From RED, open Firefox, and then:

  1. Log into google.iu.edu.
  2. Click Drive, and then Go to Drive.
  3. If you are directed to an account other than your Google at IU account, click the circle at the top right, then select your Google at IU account. For more, see Manage multiple Google accounts at IU.

You should now be able to see your files under "My Drive."

Alternative methods

Alternative methods (and instructions) for accessing your Google My Drive files from RED are provided below; however, these methods are not supported by UITS, and each alternative has its own set of limitations.

Use rclone in a Terminal window

From a Terminal window in RED you can use rclone to access your Google My Drive space. You can copy files from the command line or mount your Google My Drive space as a directory:

  • Use rclone commands to copy files: In RED, open a Terminal window, and then:
    1. Add rclone to your user environment; on the command prompt, enter:
      module load rclone
      
    2. Authorize rclone to access your Google My Drive files:
      1. On the command prompt, enter (replace googlemydrive with label of your choosing, which will be added to the rclone config file):
        rclone config create googlemydrive drive
        
      2. Firefox will open. On the Google sign-in page, enter your Google at IU username (your_username@iu.edu). When you are redirected to IU Login, enter your IU username and passphrase, and then confirm your identity with Two-Step Login (Duo).
      3. When prompted to authorize access to your Google My Drive files, click Accept.
      Note:
      You need to authorize access only once. To check what (if any) remote destinations already are authorized, use the command rclone listremotes. For detailed instructions, see rclone).
    3. Use rclone commands in combination with the label you created for your Google My Drive space (for example, googlemydrive). For a list of commands and links to their help files, see Rclone Commands. UITS recommends reading about the rclone copy, rclone ls, and rclone tree commands.
  • Mount your Google My Drive to a directory: In RED, open a Terminal window, and then:
    1. If you haven't already done so, add rclone to your user environment and authorize rclone to access your Google My Drive files; see the first two steps in the instructions above.
    2. Use mkdir to create a mount directory for your Google My Drive; for example, the following command creates a directory named GDrive on your RED desktop.
      mkdir ~/Desktop/GDrive
      
    3. Use rclone mount and the label you created for your Google My Drive (for example, googlemydrive) to the mount your Google My Drive space to the mount directory you created in the previous step; for example, to mount a Google My Drive space labeled googlemydrive to the example mount directory created in the previous step (~/Desktop/GDrive), use the following command:
      nohup rclone mount googlemydrive: ~/Desktop/GDrive &
      

      The program will continue running in the background.

    Note:

    Unless you have a clear need to use this method, UITS recommends using one of the other options to access your Google My Drive files. Following are known issues with using rclone to mount your Google My Drive space:

    • In some cases, rclone may not unmount your Google My Drive directory cleanly. If you try to unmount your Google My Drive (by killing the rclone process that is maintaining the mount), you will not be able to remount it on the same directory until UITS reboots all RED nodes during the regularly scheduled maintenance window. Also, if you end your session, or the session is terminated for some other reason, the mounted directory most likely will remain locked until all RED nodes are rebooted. If this happens, you can mount Google My Drive to a different directory.
    • Google Docs will be listed with size -1 when you use the command rclone ls and with size 0 when you invoke a command that uses the VFS layer (for example, rclone mount or rclone serve). This occurs because rclone can't determine the actual file sizes without first downloading the files. Google Docs transfer correctly when you use rclone sync and rclone copy because rclone ignores file sizes when transferring files.
    • Although a Google My Drive mount is suitable for small data transfers (under 1 GB), larger data transfers may fail without any warning or errors.
    • Your Google My Drive mount will be visible only on the RED node on which your RED session is running. It will not be accessible from Carbonate login or compute nodes. Do not submit jobs that reference files from your Google My Drive mount; instead, copy the files your job needs to your Slate or Slate-Project space, and then reference them in that location.
    • On a RED node, any application that attempts to read from or write to your Google My Drive mount directory will likely throw an error. Even if it works, I/O will be at least 10 times slower than it would be when referencing files on Slate or in your home directory on Carbonate. Copying a full data set to your Slate or home directory space will be less time-consuming than computing with data from your Google My Drive mount.

      For example, if your application references data stored in your Slate space, its I/O activity may require 10 minutes out of a 60-minute run time; however, if your application references data from your Google My Drive mount, its I/O activity will require 100 minutes and extend your job's overall run time to 160 minutes.

Use the Caja file browser

Use the Gnome Control Center to set up access to your Google My Drive space from the Caja file browser on RED:

  1. Open a Terminal window, and then enter the following command:
    env XDG_CURRENT_DESKTOP=GNOME gnome-control-center online-accounts
    

    The Gnome Control Center window will open:

    Add online accounts in Gnome Control Center

  2. Under "Add an account", click Google.
  3. On the Google sign-in page, enter your Google at IU username (your_username@iu.edu). When you are redirected to IU Login, enter your IU username and passphrase, and then confirm your identity with Two-Step Login (Duo).
  4. When prompted to allow Gnome to access your Google My Drive account, scroll down and click Allow.
  5. If prompted to create a keyring password, enter a password you will remember. You'll need this password to reopen the connection to your Google My Drive space when you log into RED again.
  6. Open the Caja file browser and, in the left panel, look under "Devices". You should see an item labeled with your email address. Click that item to access your Google My Drive space. You may need to wait briefly for the contents to become visible the first time you access it.
Note:
  • Although using this method to access your Google My Drive space is suitable for small data transfers (1 GB or less), larger data transfers may fail without any warning or errors.
  • Using this method, your Google My Drive files will not be accessible from a Terminal window. You should copy files you need for computation by dragging and dropping them to your home directory space on Carbonate, or to your Slate or Slate-Project space, and then reference them in that location.

Get help

If you are using Research Desktop (RED) and need help or have questions, use this form to contact the Research Desktop development team directly. .

This is document bgsp in the Knowledge Base.
Last modified on 2021-07-08 16:07:12.