Jamie Todd Rubin

Writer, Coder, Avid Reader

Est. 2005 • 20th Year 🎉

My Google Docs Writing Tracker Can Now Be Used with Text-Based Files

31 Oct 2014 » 2 min read » Filed under: productivity, Writing & Publishing

I pushed an update this afternoon to my Google Docs Writing Tracker that allows text-based files to be used with the system.

For those who aren’t familiar: my Google Docs Writing Trackers is a system I created that automates the process of tracking my writing word counts and time spent each day, stores the data in a Google Spreadsheet, and produces neat daily summary emails. Until now, it required people to use Google Docs to do their writing. But not anymore.

Over the last few days, I tested an update that allows you to use any plain-text form of document. That is, any document stored as a plain-text file. In addition to plain text, this includes markdown files, (.md), and HTML files. This frees folks from having to use Google Docs for the writing. You can use whatever program or editor can produce plain text files. I’ve been using Sublime Text for the last several days with great success. But even Notepad would work for this purpose.

You must still store the files in your Sandbox folder on Google Drive. I use the Google Drive app on my MacBook and iMac which produces a Google Drive folder on my computer that synchronizes with Google Drive in much the same way that your Dropbox folder syncs with Dropbox.

I write my story in a text editor and make sure that it is saved in the Sandbox folder in my Google Drive folder. That’s it. The Google Drive folder syncs things up with the server, and the Google Docs Writing Tracker scripts run automatically each night, the same way they always have, and read both the Google Docs files and plain text files.

Here is a data flow diagram that I put together to illustrate how the overall system works. It looks complicated, but really, once you’ve installed and configured the scripts, all you do is write, and the scripts do all the rest.

Google Docs Writing Tracker DFD
Click to enlarge

I’ve pushed these changes to GitHub. All of the code and instructions for installing it and using it are available in the public repository.

And just a reminder of the usual caveat: while I am happy to make these scripts available to anyone else who wants to use them, I really designed them to make my life easier, and I don’t have time to support them for others. Use them at your own risk. They work great for me and have worked well for others. But bugs occasionally pop up. And it is highly tailored to my work-style, which may not work well for you. So if you are wondering why it was designe for Google Docs, or why it doesn’t work with [fill in your favorite editor], it’s because I use Google Docs, and it works for me.

I will say one thing: my success at getting the script to work for text files makes me hopeful that I (or someone else) can get it to work for Scrivener files sometime in the future.

Related posts

Get new posts by email

13 responses to “My Google Docs Writing Tracker Can Now Be Used with Text-Based Files”

  1. Nitin Khanna

    About your scrivener comment in the last line, doesn’t scrivener have an in-built writing tracker? I know it’s not comprehensive with graphs and such, but it shows your day goals and achieved status.

    What am I missing in the picture? What are you trying to achieve that I can see?

    Reply
    1. Jamie Todd Rubin

      Part of it just personal preference. Part of it is time-saving. In Scrivener, I have to take an action to see my stats, and they are ephemeral: they are for the particular day, and then it is lost. I want to keep the data, and I don’t want to have to take any action to collect it. My scripts automate the whole thing. I write, and the data is automatically captured for me. It might not seem like much, but it does free up some time, and it helps ensure a good data set.

      Reply
      1. Nitin Khanna

        hmmm… Perhaps a way to approach this problem is to sync Scrivener files with Google Docs as plain text. I usually sync with Dropbox, since it has support with a plethora of iOS and web apps, but I guess syncing with Google Drive will solve this purpose. Also, I hate the Windows Scrivener sync solution. This one would work seamlessly only with Mac.

        Or were you looking for a standalone solution?

        Reply
  2. Michael Cummings

    With regard to rescuetime -fwiw, if you group your different writing activities in rescuetime (I made a category called “Writing” that covered google docs, scrivener, focuswriter, and storyist – DON’T JUDGE MY FICKLE WAYS), you can pull by category instead of activity (and “thingy” would be the name for your category), in which case column 5 of the json is the time spent, well, writing 🙂

    Leave it to someone that can’t make up their mind to find another way to cheat.

    Reply
    1. Jamie Todd Rubin

      Michael, I’ve been meaning to do just that–put all of my writing activities into a single category. But for me it requires a little more discrimination, because I use Sublime Text for other things, like maintaining a to-do list and for coding. So I need to look at the document type as well as the application. (If I am using Sublime Text, and the document has a “.md” in the name, I know it is a writing activity.)

      Reply
  3. Craig

    Many moons ago when I saw your writing tracker/almanacs (before you even posted it to GitHub) I became very jealous so I set out to write my own. I didn’t have success using Google Drive/Docs for various reasons I won’t spiral into (including not wanting to use Google’s web editor)… but I did end up writing a bash script with Dropbox that accomplishes much of the same things (albeit with less elegant output).

    It works well with Scrivener’s sync-to-folder function (you could also try that with your sandbox folder; it would probably work). Same principle as yours, it runs every night around midnight on my mac mini, doing comparative word counts iteratively on the files in my sandbox folder on Dropbox, comparing them against the previous days. Then it outputs net word count to a CSV in Dropbox and emails me my daily report. In my Google spreadsheet, I have a live-updated link to the shared Dropbox CSV file, so I do get a nice graph of how I’m not writing as much as I should be.

    Anyway, it’s great to see you’ve adapted this, I might finally be able to give it a whirl.

    Reply
  4. C.C. Camlin

    Hi, ace program you’ve written here! I set up your Google Docs Writing Tracker a couple of weeks ago and it’s such a neat thing. I’m not a programmer so I’m very taken in with how this works and everything- super cool. I have a question though, I’ve noticed in the Daily Almanac emails that it keeps repeatedly saying that I’ve written for 1 consecutive day every day even when I don’t miss a day. And lately the “you’ve written 7 out of the last 15 days” will jump to “you’ve written 9 out of the last 17 days” after I had in fact missed a day. Would it be possible for you to help me fix this?

    Reply
  5. Blaine Moore

    While reading this in Feedly, my thought was to come and ask what the feasibility of integrating with Scrivener would be…then I saw your comment hoping someday you or somebody else could get it to work. Then I read the comments and see a potential method for making it work…

    Definitely hope that the sync to folder option does do the trick, that would be great. Would love to implement this into my daily workflow without having to leave Scrivener…

    Reply
    1. Craig

      Blaine,
      I just tested this with Scrivener and it works perfectly, with a simple caveat. Use the “Sync : with External Folder” function in Scrivener, and select your appropriate local Google Drive folder, but be sure the “Format for external Draft files” (and “other external files” if applicable) is set to Plain Text.

      For some reason Google Drive (desktop version anyway) doesn’t sync RTF as editable text files, even though it does convert them properly if you manually upload them through the web page. Weird. But if you don’t need RTF then you should be set with Scrivener and Jamie’s wonderful tracking scripts.

      Reply
      1. Craig

        I just discovered a 2nd caveat with this Scrivener/GDrive sync.. also uncheck the “Prefix file names with numbers” option in the Scrivener/Sync with External Folder dialogue. I have no idea why the script won’t recognize them as text files with that turned on… it’s very strange, but it absolutely ignored them when my test files had numbers in front of them. And totally works with it turned off.

        Reply
        1. Jamie Todd Rubin

          I’ll have to test it out with a setup like you describe, when I have some time. All the script is doing when it grabs the text of a file is getting the file content, depending on whether it is a Google Doc or text file. Here is the exact code being used:

          if (DocsList.getFileById(id).getType() == DocsList.FileType.OTHER) {
          var doc1 = DocsList.getFileById(id).getContentAsString();
          var name = DocsList.getFileById(id).getName();
          } else {
          var doc = DocumentApp.openById(id);
          var name = doc.getName();
          var doc1 = doc.getText();
          }

          I’m not sure why a number prefix would break that.

          Reply
          1. Craig

            Yeah it’s super weird. I tested it multiple times with the same results. There is something odd about the way Google translates text documents from the local drive .txt to .gdoc. I noticed in the log file the script is interpreting these straight-up .txt files as “.md” (FileType.OTHER in other words) so the Google API is basically ignoring/rejecting them as text files because they’re not in Google’s unique text file format. This is why I ended up writing my own scripts originally. I couldn’t get Google to recognize the text files as text unless I went in manually and opened the files on the web browser to “edit in Google docs” then it created a duplicate file with their gdoc extension in the local gdrive folder which screwed up external editing and defeats the whole purpose since it’s not automated anyway.

            The API/Drive is a weird system (at least compared to Dropbox et al) and I have no idea how the file prefix makes any difference… unless it’s making a fuzzy guess on what kind of file it is, and it decides that if it starts with a number, it’s probably not text…?

            Reply
          2. Craig

            Okay, to thoroughly beat a dead horse. It seems the real issue was not the number prefix, but something about the way Scrivener updates the synced text files or the way Google detects changed dates. Once I commented out the line: if (today == Utilities.formatDate(files[i].getLastUpdated(), TIME_ZONE, "yyyy-MM-dd")) {

            -which forces the script to diff every file in sandbox vs snapshot- it worked fine. Maybe consider changing the granularity of how it checks the changed files? Instead of a flat “today” vs last modified date, perhaps it would work better to just see if the dates are modified at all between the sandboxed and snapshotted files. I don’t know what would be best, but for my purposes now I’m going to leave the date modified check out and let it diff all the files…

            Reply

Leave a Reply to Jamie Todd RubinCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Discover more from Jamie Todd Rubin

Subscribe now to keep reading and get access to the full archive.

Continue reading