Tag: notes

My Shorthand for Notes and Other Writing

When I wrote recently about print vs. cursive, I said the following:

I tend to use a lot of shorthand in my journals. I rarely spell out names of my immediate family, resorting instead to first letters. I have dozens of shorthand codes for words and phrases I use commonly.

I was surprised by the number of people who reached out to ask me more about my shorthand. So I thought I’d take this opportunity to present peek into my shorthand and how it evolved. The latter is important because I can’t claim to originate all of it. I learn from the example of others in some cases.

First, a little background. When I was in 7th or 8th grade, I was fascinated by the concept of shorthand. I remember walking to the local library and pouring through books on Gregg’s Shorthand. 7th grade was right around the time I had to start taking notes and I was looking for ways to make it easier. Gregg’s shorthand never took, but the idea behind it stayed with me. Between my sophomore and junior years I found a book which in part, taught me how to take better notes. I wish I could remember the name of that book, because it has shaped the way I take notes ever since, but alas, it is lost to me.

Over the years since, I have refined the way I take notes, looking for shortcuts and adapting along the way. Eventually, it became second name to write longhand this way, and I turned it to all my longhand writing that I do for myself. (Although, admittedly, sometimes my shorthand creeps into longhand writing I do where someone else is the intended audience.) Here I’ll cover four that I use most frequently.

Examples of my shorthand in handwritten notes.

Shortcut, courtesy of the Ultima video games

When I was a teenager, I loved the Ultima games by Richard Garriott, a.k.a Lord British. I loved the detail in the games, to say nothing of the cloth maps. I also enjoyed how the game made use of Germanic runes for the language. There was a time that I could read those runes almost as well as I could read English.

One of the characters in the Ultima alphabet was a verticle line connected to a triangle (see the first item in the image above). This represented the letters “th” and I quickly adopted this as my shorthand for the word “the”. This is so ingrained in my today that I’ve lost track of how often I’ve used it in writing I’ve given to others. “The” isn’t used much in notes, but in other writing that I longhand, especially fiction, it comes up quite frequently and my shortcut saves time. I also use this to preface words that begin with “th”, like “there”, “then”, “theory,” etc.

The word “very” is very unnecessary

In a creative writing class in college, there was a discussion of the use of the word “very.” The general consensus was that the word is overused and should be avoided. Of course, it can’t always be avoided, espcially if I am taking notes that capture a quote or something someone is saying. But I did take it to heart in my notes and longhand writing. Instead of writing out the word “very” I draw a horizontal line over the word that “very” modifies. (See the second item in the image above.) So if I’m jotting down the phrase, “it was very hot today,” what I actually write is, “it was hot today” with a line over the word “hot.”

Shortcuts from the story of civilization

One of my favorite pieces of history writing is Will and Ariel Durant’s 11-volume Story of Civilization series. While some of the history is dated, the writing remains a real work of art to me. A few years back, I read their Dual Autobiography. One passage really struck me as practical:

We discovered an excellent typist, Mrs. Edith Digate, who soon learned to understand Will’s handwriting and abbreviations (d=ed, g=ing,…, etc.)

Ever since reading that, I’ve taken to using “g” at the end of a word instead of “ing” when I am writing longhand. (See the third item in the image above.) So in writing out words like,”writing” I write “writg”. I also write “runng”, “talkg”, “sleepg”, etc.

I never adopted the “d” for “ed”. I don’t know why. Maybe because it was not as efficient.

Taking a page from the British Secret Service

Anyone who has seen a James Bond film knows that many of the characters within the secret service are known only by a letter: Q and M being the most famous examples. For the most common names I use in piece of writing, I typically will resort to using just the first letter. (See the fourth item in the image above.) In my journals, I use the first letters of the names of my immediate family members instead of the full names. Fortunately there is no overlap. In fiction, I do this for main characters so long as the shortcut will cause no confusion.


Of course, I abbreviate many words for convenience and speed as well. I have grown used to my shortcuts, but I still often lust for being able to take notes using real shorthand. I remember attending a meeting early in my career in which an administrative assistant took notes in shorthand. She was able to reproduce verbatim, everything said in the meeting. The pages looked like gibberish, but I was impressed.

I’m always looking for ways to improve and refine my own shorthand, so if anyone has tips or suggests, drop them in the comments. I’d love to see how others do this.

My Current Obsession: The Tim Ferriss Show Podcast

For the last six days, I have done almost no reading, a thing virtually unheard of for me. Instead, I have been obsessively listening to back-episodes of the Tim Ferriss Show Podcast. Tim Ferriss is the author of the Four Hour Work Week, a book that I skimmed, but never finished. I have, however, read two of Tim’s books that I really enjoyed, Tribe of Mentors and Tools of Titans. I’ve never been much of a podcast fan, but the thing that attracted me to Tim’s podcast was that I knew he was a meticulous experimenter, and tried to learn from data. I’m this way as well–as I wrote about often in the days after I discovered the concept of the quantified self.

I have always been someone who tries to take actionable lessons from my reading and experiences. When I read biographies, I take notes on things that the subject found useful and see if I can apply them in my life. Tribe of Mentors and Tools of Titans were chock full of these kinds of actionable insights. (One example: I learned of the Calm app, and have been using it for over a year now for daily meditation.) So I figured I’d give the podcast a try. I started with the most recent episode this past Friday, which wasn’t an interview, but a kind of roundup. After that, I went through the back list of 517 other episodes and marked the ones I thought I’d be initially interested in listening to. The list below is the list that I have listened to in the six days since. It is listed in order beginning with my most favorite. I list the times of the episodes to give a sense of just how obsessed I’ve become with these.

I’ve done the math: that is 19 hours of podcasts in less than six days. Obsessed is probably not an exaggeration. I have filled pages of my current Field Notes notebook with notes, ideas, and scribbling from these podcasts.

Raw notes I've taken from podcasts
Raw notes I’ve taken from podcasts

I’ve then tried to turn these into curated notes in Obsidian, for example from the first Jim Collins interview on the podcast:

Some of my curated notes from the first Jim Collins interview
Some of my curated notes from the first Jim Collins interview

And this is what I’ve managed to get through so far. I’ve got at least another 20 or so in the list I pulled, including a second interview with Walter Isaacson, Ken Burns, Steven Pressfield, Michael Lewis, Edward Norton, Neil deGrasse Tyson, David Allen, Adam Savage, Nick Thompson, Drew Houston, Tim O’Reilly, and more.

I usually get a bit anxious if I am not reading a book, especially after a few days, but I’ve been so focused on these podcasts and what I can learn from them, that it hasn’t bothered that I haven’t done much reading in the last six days. It doesn’t bother me to think I may not do much over the coming week or so as I get through this initial wave of podcasts. It has actually been a pleasant change of pace.

The only problem is what to do with the mass of notes I’ve been generating. I think I’m going to need to set aside an entire day to compile and make sense of them, and figure out what actions I want to take first, and put together a plan. It’s really great fun, I find myself smiling often as I listen to the podcasts, and have been honestly surprised (despite how much I read) at how many of the books mentioned on the podcast I have already read. And of course, rather dismayed by how many I have yet to read.

My Obsidian Daily Notes Automation Script is Now Available on GitHub

Since I am on vacation and happened to find myself with an empty hour this afternoon, I managed to clean up my code enough to where I was willing to put my Obsidian daily notes automation script on GitHub. This is the script that I use to automate the creation of my daily notes in Obsidian.

You can find the repo here.

As I say in the README:

I’m posting this software as-is. It works for me, and a number of folks have requested it and I’m happy to put it here to share it. But I have no time to support it. If I make improvements, I’ll try to post them, but there’s no guarantee there either. I realize that this may not work perfectly on non-Mac systems, but the whole point of posting the code is to let folks see it, fork it, and roll your own from it. Hopefully it works for you the first time. If not, the code’s there for you to mess with.

For those who choose to use it, keep in mind that I run my script on a Mac, and icalBuddy, which I use for pulling in my agenda, is designed for Apple’s calendar app. You may need to look for alternatives for other platforms.

I’m always eager for feedback and suggestions, but as I said, my schedule is such that I don’t have time to provide any kind of support for getting the script working for you.

Obsidian and Vim Mode

For the last several days, I have been playing around with Obsidian in Vim mode. Vim, for those who don’t know, is a powerful text editor that can take some getting used to. It uses different “modes”: for editing, for navigating and issuing commands. It’s keyboard commands are designed for touch typists so that you can do anything you need to do without your fingers ever leaving the keyboard.

Obsidian offers a “Vim mode” which gives some of Vim’s capabilities. I like the idea of Vim but I’m not completely sold on its implementation in Obsidian yet.

Because of how navigation work (basic cursor movements use the h, j, k, and l keys) a fixed-width font is better for Vim. The theme that I use, Pisum, doesn’t make use of a fixed-width font. That meant I needed to edit the styles in the theme to get what I wanted.

This was a useful side-effect of experimenting because I found that it was pretty easy to edit the styles. I copied the styles for the Pisum theme into another .css file and edited there so I didn’t mess up the original theme. I made two basic changes:

  1. I switched to a fixed-width font in the editor (but not in preview mode).
  2. I modified the emphasis style to show an underline. I like seeing the underlying for emphasis because this is what I am used to from decades of writing manuscripts in standard format–in which italicized text is represented with underlines.

Here is an example of what this looks like:

An example of my Obsidian theme changes
An example of my Obsidian theme changes

But there are some serious limitations. For one thing, Obsidian is not Vim, and that is a good thing, since Obsidian is really focused on something different from what Vim attempts to do. It means, however, there is limited support for .vimrc files in which various setting, keyboard mappings, and other configurations that customize the editor are stored. There is a plug-in that provides some limited support for .vimrc files, but it is limited

It is nice that Obsidian includes the Vim mode option because it makes the transition to Obsidian easier for people used to Vim’s keyboard mappings. But after playing around with Vim mode for several days, I think I am going to turn it off. I asked myself how often, while writing do I need to do some of the fancy things that Vim’s commands let me do? The answer is rarely.

All is not lost, however. I learned that it is easy to edit themes in Obsidian–by far easier than any other editor I’ve played with. And some of my edits I’m keeping. I’m turning off the fixed-width font. I prefer the default theme font for notes. But I’m leaving my underlined emphasis in place. And there are probably other tweaks I’ll make. I have it in my mind to produce a theme that looks like Word for DOS 5.5 (which was my all-time favorite word processor). I’d do this more for learning than actual use. After all, these days, everyone want to use a dark theme because it’s easier on the eyes, and that bright blue background in Word for DOS is the antithesis of a dark theme.

Once I have my personal theme stable, I’ll write a separate post about it.

We Need More Practical Lessons

While reading Walter Isaacson’s new book, The Code Breaker, I was particularly struck by some seemingly minor details. The book is a fascinating look into the modern process of scientific discovery, and there was some discussion of how a discovery written in a lab book and then signed by witnesses in order to document the dates of the discovery. When do scientists learn to do this?

I took AP biology, and AP physics in high school, as well as physics, chemistry and organic chemistry in college and no one every taught me how to properly use a lab book. Indeed, what was implied, at least at that level, was that what the teaching assistants and grad students who led the labs really wanted was nice, neat copy in our lab books with clear results that were easy to grade. I remember many of my fellow students had two lab books: the one they worked stuff out in, and the one they turned in after everything was cleaned up. I couldn’t spend the money on two lab books, so mine were messy.

It seems to me that the mechanics of a lab book–its true purpose and how it is used the real world–is a practical lesson that any burgeoning scientist should learn. But who teaches this? Are there upper division chemistry classes that focus on this? Certainly o-chem didn’t.

This got me thinking about other practical lessons that I would have benefited from, but was never formally taught. How to read a newspaper is one example that I’ve written about before. What about keeping a diary or journal? I don’t ever remember this being taught in school. I don’t ever remember a class in which the pros and cons of journals were discussed. I would have found these things very useful. Instead, I learned how to keep a journal by following (initially) the example Isaac Asimov described for himself in his autobiography.

Lab books are useful tools outside of the laboratory. For the first half of my career, I didn’t keep any kind of notes about the code I was writing. If I had to recreate something, therefore, it was often hard work. At some point, it occurred to me to keep notes as I worked. When I do something particularly complicated, I often list it out in my notes in high level steps, and then fill in the details as I work. I keep one simple idea in mind: a person new to the organization should be able to take my notes and reproduce my work. Technical debt is a big problem in I.T. People come and go and leave behind lots of undocumented code in their wake. You’d think lessons in keeping good notes would be part of the training process, but I’ve never seen it.

For that matter, how about something as simple as keeping a to-do list? I was never taught this in any of my classes.

There was one class I had–a 7th grade science class–in which our teacher spent quite a bit of time teaching us how to organize our work. We learned how to keep our science folder, and how to keep our notes and assignments organized in the folder. It was practical information that served me well through the rest of my pre-college schooling. Beyond that, most of the practical things I learned from books.

I can’t remember a teacher teaching how to take notes: how to identify the important points, and highlight them; what to leave in and what to exclude from the notes; tricks of shorthand to capture information more succinctly. All of this I had to figure out on my own. I read a book between my sophomore and junior years in college, and one chapter was all about note-taking. It changed the way I take notes and I use that method to this day.

I try to pass on some of these practical lessons to my kids. The Little Miss keeps a journal and I encourage that, and allow her to look at my journals in order to take ideas, but mainly so that she understands she can make it whatever she wants it to be. The Little Man could benefit from a daily to-do list, and I’ve tried on a couple of occasions to suggest it, even offering to help him get started by reviewing it together. He resists it, but he is at the age where he doesn’t think he needs it. (He does.)

It seems to me that in addition to classes in science and math and reading and English and history and art and physical education, there should be some practical classes on topics like these. Better yet, practical lessons could be merged into the existing classes.

  • In science, you could learn how to keep a lab book while you do your experiments. The lessons would be about the purpose–not to show you got the right answer, but to be able to reproduce your results, whatever they were.
  • In English, there could be a section on the literature of diaries and journals. There are plenty to choose from: John Adams, Samuel Pepys, Henry David Thoreau, Anne Frank just to name a few. Discussions could ensue about why to keep a journal, the practical value, and the literature can provide examples of what other people have done.
  • In home room, you might learn how to better organize your day, keep track of your work, and manage stress.

We need more practical lessons. I certainly would have benefited from them earlier than I did.

Journal in Obsidian Notes?

Once I got the hang of how Obsidian worked for me, once I realized the power of its linking capabilities, and that it really did everything I wanted a note-taking app to do, it was natural to consider what could go into my vault. Daily notes were a given, of course. All of my reading notes, and even a version of my reading list could go in there. Borrowing some concepts from Zettelkasten, it could become a kind of digital commonplace book, something I’ve always wanted. What about my journal? With all of the other information in one place, linkable and searchable, it seemed to make sense that my journal should go there as well.

The thing is, my journal has always been handwritten, going back to 1996. There were times when I experimented with it in a digital form, but I always came back to the handwritten form. In the current incarnation (since late 2017), they fill eight Moleskine Art Collection sketchbooks.

My collection of journals.
My collection of Moleskine journals

As it turns out, how I keep my journal lends itself Obsidian linking. Rather than an entry-per-day, I number entries, beginning at 1. Each discrete entry gets its own number. I date the first one of each day, but there may be two or three entries in a day, each of which will have its own number. I did this thinking ahead: if I ever wanted to index the thing, I wouldn’t have to worry about what volume or page and entry was on. All I’d need was its entry number. (I took this lesson from Isaac Asimov’s description of how he numbered entries in his Biographical Encyclopedia of Science and Technology in order to avoid the tedious work of tying index entries to page numbers.) Thus, I have 1,782 unique “entries” each with its own number. This makes it ideal for linking in Obsidian.

Several weeks ago, I decided to give it a try, and I began writing my journal entries in Obsidian, giving each entry a unique number, continuing from where I left off. I liked being able to link these entries to other notes.

Journal entries listed in Obsidian
Journal entries in Obsidian

Something nagged at me, however. I missed writing in my journal. I missed how the pages contain more than just writing. I paste pictures and clippings in the pages. Sometimes I sketch things. It just didn’t feel the same typing the entries rather than writing them out in my journal.

A typical multimedia page from my journal.
A typical “multimedia” journal entry

It occurred to me that I might have the best of both worlds with a little effort. At the end of each week, for instance, I could type up the entries I’d written in the Moleskine notebook, copying the entries into Obsidian. Then they’d be there for searching and linking. After a little thought, that felt like a monumental waste of time.

Last night, I decided not to keep my journal in Obsidian and to continue with the notebooks. I did this for several reasons:

  1. I still think there is a compelling argument for how long paper lasts. Digital media has been around half a century or so. Paper has been around centuries. Witness John Adams’s diaries or Leonardo da Vinci’s notebooks.
  2. I don’t actually search through my journals that much to make it worthwhile to put every entry into digital form.
  3. When I do search them, I enjoy the feel of flipping through them, seeking out what I am looking for.

But there was one other thing that occurred to me that sealed the deal for me. My Daily Notes in Obsidian serve as an index to my life. If I needed to know when I wrote about something in my journal, I need go no further than my daily notes. I can search them for the appropriate reference and then use the date of those notes to look up any entries in my journal. Moreover, if I write something in the journal and want to make sure I can find it easily, I can just add a reference to the entry number in my daily notes.

That seemed to satisfy me, and with that, I began this morning, transcribing those entries I made in Obsidian back to my Moleskine notebook. Going forward, the journal will stay in a notebook, but I’ll rely more and more on the daily notes as a kind of compass for finding what I need.

How I Capture Reading Notes in Obsidian

In addition to automating my daily notes with Obsidian, it quickly became clear to me that Obsidian‘s note-linking capabilities would allow me to capture my reading notes in Obsidian in a really useful way. Moreover, because of Obsidian’s powerful linking capability, it occurred to me that my Obsidian vault could serve as a database for my reading. To describe how I managed to do this (so far) in a step-by-step manner will required a little history first.

A Brief History of My Reading List

I began keeping a list of every book I finished reading back on January 1, 1996. Although I am no longer certain of why I started keeping the list (was it part of a New Year’s resolution?) I am fairly certain that I was influenced by an early reading list I found on the Internet, Eric W. Leuliette’s “What I Have Read Since 1974“.

As a developer (even back then), I decided I would build an elaborate relational database to store my reading list. Over the years, it went through many iterations, and forms. When time became short, I moved the list out of the database and into Excel, or Google Sheets. Finally, several years ago, I settled on a plain text file using Markdown format, and that is how I’ve kept my list ever since.

But I’ve been bothered by shortcomings on this list. There are redundancies I don’t like about it. I have no easy way of referring to books or authors separate from the list. There are things I’d like to automate about it but that the format makes tricky.

A Brief History of My Reading Notes

With all of the reading I do, I have trouble remembering important details of what I read about. So I started keeping notes on my reading. This evolved out of how I kept notes on my reading back in college, and has continued to evolve over the decades since. It was in college that I first decided it was okay for me to write in my books. After all, if I was spending so much money on them, I might as well make them my own, right?

These days, I highlight books, writing margins, and with e-books, I highlight and make short notes on my Kindle devices and apps. But I still have no good way of aggregating these notes into useful groups, categories, and certainly no way of readily searching them.

As I started using Obsidian, and began to see how I could better organize my books and reading lists in its vault structure, I began to get a hint of ways that I might start to link my reading notes back to the books they are associated with, my reading, and other notes.

Enter Zettelkasten

I’d never heard of Zettelkasten before I started using Obsidian. Zettelkasten was originally invented as a way to link paper notes together to be able to easily create connections (links) between then. While it was workable on paper, such a process could be greatly improved with hypertext tools, and it so happens that Obsidian’s note-linking capability is idea for this.

One important idea from Zettelkasten is that a note should contain a single thought or piece of information (say, a passage highlighted in a book). That note is given a unique identifier. In addition to the passage, one would add their own thoughts to the note, and perhaps further link that note to other notes and ideas that are related to it. Zettelkasten has its own unique numbering system for “naming” the notes. Obsidian has a plug-in for creating a “Zettelkasten number” for this purpose that is based on the date/time the note is created. I wasn’t particularly fond of that identifier because it duplicates information already contained in the note itself. After all, the note is just a file in the file system, and has its own create and modified date/times as part of the file. A good identifier does’t embed real data. It’s just an identifier.

I also struggled a bit to figure out how this would work for my reading notes. I originally imagined that if I had a note for each book I read, I could simply add my highlights and annotations to that note. Zettelkasten, however, suggested that rather than adding that highlight to the book note, I’d create a separate note for just the highlight or annotation, and then link it to the book note–as well any other notes it might make sense to link it to. This took a while for me to process, and I thought about it a lot as I built out my reading library in Obsidian.

My Obsidian Library

So how did I decided to structure my reading notes in Obsidian? I’ll try to go through the step-by-step process I have for putting this all together, in case someone is interested in reproducing this.

Step 1: Establishing the structure

I decided that because of Obsidian’s great linking capability, I could use the file system itself as a relational database. In deciding this, I further decided that there were 3 main “objects” I wanted to be able to capture at a kind of atomic level. That is, three things that make up the structure of my reading library:

  1. Things I read, e.g., books, articles, stories, etc.
  2. Authors: the people who write the things in #1.
  3. My notes as they relate to #1 and #2.

From this, I established the following structure of folders in within my Obsidian vault:

My folder structure for Reading notes in Obsidian.
  • Commonplace Book contains all of my reading notes.
  • Library contains all of the “atomic” notes that make up my reading library:
    • Authors: a single note for each unique author in my library
    • Articles: a single note for each unique article (often not tied to a book) in my library.
    • Book: a single note for each unique book in my library
    • Essays: a single note for each unique essay in my library; these are often related to books.
    • Stories: a single note for each unique story in my library.

Step 2: Deciding what goes into a note

Once I had my structure, I had to decide what goes into a note of each type. What is it I want to know about authors, books, stories, etc.? This was fairly easy for me as I’ve been thinking about it for a long time (years, actually). I had in mind an idea that I could write an API that uses these files as a database to query them and produce results. With that in mind, I decided to start by keeping things simple, knowing that I could add detail as needed going forward.

For authors, I wanted just some basic information. Here is a typical author note, in this case, for Alan Lightman, whose new book I read earlier this week:

A sample author note for Alan Lightman.

The backlinks section is generated automatically by a script that I have that runs nightly. I know that I could just click on the “Linked mentions” in Obsidian to see all of the backlinks, but I wanted the related books on the note as a reference in case I access the file outside of Obsidian.

For books (or essays, stories, articles), I also kept things simple. A typical book (or essay, or article, or story) looks like this:

A sample title note for In Praise of Wasting Time

Note that in both authors and books there are links back and forth between the files. The book file refers to the author. The author file has link references back to the books. Moreover, you’ll note that in the book, there is an “Annotations” section with a list of links. These are auto-generated links to my notes and highlights for the book. I’ll have more to say on these shortly, but the important thing is that each note and highlight is a separate file (in the Zettelkasten vain) and is included with the book as a “transclusion” link, meaning that when I view the note in preview mode, it “includes” the links files as part of the note, like this:

Title note in preview mode with transcluded annotations visible.

Step 3. Populating the database

Once I had the structure I wanted, I needed to populate my database. I was fortunate in this regard on 2 counts: (1) I happened to recently create a SQLite database of my books, and (2) I can write code relatively easily. I wrote a script that crawled my book database, and from it, creating the notes for books and authors in Obsidian. This turned out to be a surprisingly simple exercise. (The Python script was 130 lines.)

My digital commonplace book

I first learned of commonplace books reading a biography of Thomas Jefferson (in this case, it was Williard Sterne Randall’s Thomas Jefferson: A Life.) Jefferson (and others in his time) would copy passages from their reading into a book. This helped with memorization, but it also provided a resource where they could add notes and observations. I’ve always liked this concept, and I decided that Obsidian would finally allow me to put it into action in a way I’d envisioned.

It is trivial to create a note and add it to the note containing the book to which it is related. But what if the note ultimately relates to more than one thing? Reading about Zettelkaten provided me with insights into how I might handle this. The naming convention in Zettelkasten (and the way it is implemented in Obsidian) bothered me. Neither made much sense. How do you search for things with essentially coded filenames?

I was in the shower when I finally had a breakthrough insight on this. I’m not searching for a filename, I’m searching for file content. If each annotation and highlight I can link it to as many notes as makes sense. Furthermore, I can add tags to each note. The name of the file doesn’t matter. What matter is how it links to other notes, and that all files are searchable.

I still didn’t like the file-naming scheme for Zettelkasten in Obsidian, which essentially uses a datetime stamp down to the current second. So a file might be named: 20210215084456. Given that one is not likely to create two of these notes within the same second, it guarantees uniqueness. But from a database perspective, identifiers like these are not supposed to embed any information. They should be strictly identifiers. Moreover, the with the date embedded in the note title, I would be duplicating information that already exists in the file properties.

I decided instead to use a Guid, or what is sometimes called a UUID. This is another form of a unique identifier that doesn’t embed information, just produces a unique code. (For those tech-savvy folks reading this, I used Python’s UUID4 which doesn’t use the MAC address as part of the identifier.)

When I have a new note or highlight for a book, it goes into my Commonplace Book folder in Obsidian. These notes also have a specific structure. A typical one looks like this:

A typical note, Zettelkasten style.

Each annotation begins with a Source that links back to the source for that annotation. It may or may not have tags associated with it. That is followed by the body of the annotation, which may be a highlighted passage. Finally, there are my own notes related to the specific passage. In the above example, my notes also link to another book, making this particular annotation related to more than one note. That is, a link has been created between Creativity, Inc by Ed Catmull and Amy Wallace, and On Writing by Stephen King.

Automating my annotations

Over the weekend, I got a start on automating these annotations. I wrote a Python script that reads a CSV files exported from Kindle, and creates a unique note for each annotation in the file, relating it back to the source book in my Library. My process is roughly this (I say roughly because this is still new):

  1. When I finish reading a book, I export the annotations from my Kindle, which sends me an email. That email has a CSV attachment which I save in a folder.
  2. A script runs, and processes and CSV files I have in the folder, creating the notes and links.
  3. The script, outputs a list resulting annotations for each file. I copy this and paste it into the “Annotations” section of the source book or article. That makes it easy to view the annotations inline when previewing the note. An example of the output from the script looks like this:
Output from my annotation import script.

Toward an API for my books and annotations

I am able to do the above automation because I have a standardized structure to my books and author notes. That standardization allowed me to write an API for my book library. From this API I can, for instance, check to see if a title exists in my library already. I can grab information about a book or author and then use it in some way. The API typically returns data in JSON format. For instance, if I call the function biblio.search_by_title("Beyond"), I get a JSON formatted return containing the following:

[
   {
      "title":"_Beyond Band of Brothers: The War Memoirs of Major Dick Winters_",
      "link":"[[Beyond Band of Brothers (334)]]",
      "type":"book",
      "authors":[
         {
            "author":"Winters, Richard",
            "authorFirstLast":"Richard Winters",
            "authorLink":"[[Winters, Richard]]",
            "gender":"None"
         }
      ],
      "source":"",
      "date":""
   },
   {
      "title":"_Beyond Apollo_",
      "link":"[[Beyond Apollo (58)]]",
      "type":"book",
      "authors":[
         {
            "author":"Malzberg, Barry N",
            "authorFirstLast":"Barry N Malzberg",
            "authorLink":"[[Malzberg, Barry N]]",
            "gender":"m",
            "alternateNames":[
               {
                  "name":"Barry, Mike",
                  "nameLink":"[[Barry, Mike]]"
               }
            ]
         }
      ],
      "source":"",
      "date":""
   },
   {
      "title":"_Beyond the Blue Event Horizon_",
      "link":"[[Beyond the Blue Event Horizon (259)]]",
      "type":"book",
      "authors":[
         {
            "author":"Pohl, Frederik",
            "authorFirstLast":"Frederik Pohl",
            "authorLink":"[[Pohl, Frederik]]",
            "gender":"None"
         }
      ],
      "source":"",
      "date":""
   }
]

The results so far

I’ve linked all of this together using my master reading list note. This note contains a list of everything I have read since 1996 and serves as a kind of index to my reading:

A sample from my reading list index note.

A big part of the way Obsidian works is that it can show you the relationships between your notes. While I am still working on importing all of the notes I have in my Kindle, I can already see a a network of relationships when I view the graph of my Obsidian vault:

A graph of the relationships between all of my notes.

Most of my notes are book and reading-related at this point. That big dot in the center is the master reading list illustrated above. If I highlight it, this is what I see:

Sample of a highlighted node on the graph.

From there, you can see other nodes and relationships that have started to form. For instance, if I hover over one of the Alan Lightman books I finished yesterday, In Praise of Wasting Time, you can see a little network of links coming off that book:

Some of those links point to annotation files. Another points back to the note for Alan Lightman. And a few of the annotation links point to seemingly unrelated notes.

Here is another example. One of the big nodes is for John W. Campbell, editor of Astounding Science Fiction in the late 1930s through his head in the early 1970s. I read many of those old issue when I was taking my Vacation in the Golden Age of Science Fiction. So Campbell shows up a lot on my master reading list:

Highlighting an author node on the graph.

You can see that Campbell is linked to all of the issues of Astounding that I have read. I have started to bring my notes in for those issues. If we look at the July 1939 issue, for instance, you can see this is related to all of the stories and articles and authors in that issue:

Currently, the notes for each story are part of the story note, but I plan on breaking those out into their own Zettelkasten-style notes as I’ve done for my other notes and annotations.

Conclusions

Keep in mind, that this is all being done with plain text files, something that I like because the format is compatible virtually anywhere. This could be done as easily on a Windows machine as a Mac. It could be done easily on a Linux machine. The openness and longevity of plain text (which has been around for fifty years now) is a big part of what I like about this system.

The linking that Obsidian provides from within its application makes all of this useful. But once established, those links are just as useful outside Obsidian with a little coding–as I’ve done with my API for books and authors. And this API is extensible. This week, I plan to add capability for the API to return any annotations when returning a “book” object. So in addition to what is returned by the JSON format illustrated above, that will soon contain a node for annotations related to that book.

Mostly, I am satisfied that I now have a simple way of keeping my reading notes in a useful form. These are easily searchable, they are easily linked. I can continue to capture highlights and brief notes as a I read. The import function allows a nice step to expand on my annotations as I review them after they’ve been pulled into Obsidian.

It did take me some time to get the infrastructure in place, but now that it is there, I am able to focus on reading, notes, and let the system organize them for me.

Notes with Obsidian: My Initial Impressions

Now that I have been using Obsidian fairly exhaustively for the last 5 days, I think I’ve got enough experience with it to share my initial impressions.

I mentioned earlier that Obsidian was the notetaking tool I’ve been looking for all my life. That begs the question: what is it that I’ve been looking for in notetaking software to begin with? Here are a few of my requirements. (Your mileage may vary.)

My requirements

Notes should be plain text

There are two kinds of notes I deal with:

  1. Ephermeral notes, or those notes that I am jotting something quickly for later use, but that can ultimately get thrown away because they will be transformed into something else. Examples of this might include a person’s name, or phone number, or a rough set of notes from a phone conversation.
  2. Permanent notes: the kind that I’ll continue to use, update, and search into the forseeable future. Examples of these include how-to notes, daily notes, notes and clippings from reading. Drafts of blog posts, lists I maintain.

When I think of notes, I think of the need for versatility. I keep a Field Notes notebook in my pocket at all times to capture many of the type 1 notes above. I’ve found over the years that is much quicker than pulling out my phone, unlocking it, finding a notes app, and tapping out what I want to capture. To that end, the most versatile file format for digital notes, for me, anyway, is plain text. Plain text has many advantages:

  • I don’t have to worry about the file format. Plain text files created in the 1980s can still be read today with any old text editor.
  • Plain text files are easy vehicles for automation. Automation is a big reason why I’d want to keep notes in digital form in the first place.
  • Plain text files are cross-platform compatible.
  • Plain text files are easy to search and are the perfect targets for advances searches with regular expressions.
  • With Markdown, even plain text files can be rendered with formatting.

Notes should be rendered as Markdown

When I have looked to convert all my notes (and even my writing) to plain text in the past, one thing that has help me back is the lack of a good tool for rendering the Markdown in the files. It never looked like how I wanted it. Most text editors are really for writing code, and they bias their features in that direction.

This is one place where Obsidian excels. It has a great UI, one that is designed for notetaking (instead of code) and one that renders Markdown well. It is also flexible enough to allow custom themes so that really, you can make your plain text notes look pretty much however you want.

Notes should be easily linkable

A few years back, I tried to put together a system that allowed me to create links between my notes. I understood intuitively how useful this would be. But it seemed like a difficult problem, and really, I didn’t want to spend my time writing code, I wanted to work with my notes.

Obsidian’s note-linking capability–one of its central functions–eliminates this problem, and removes all the roadblocks to linking notes. Not only have I jumped on this feature, I have even started down the Zettelkasten rabbit hole (something I will discuss in a future post). Being able to easily link notes, follow those links, and see the relationships they form is a huge plus in favor of Obsidian.

Notes should be kept locally

I like that underneath it all, these are just plain text files sitting on my local file system. Of course, I can store them in iCloud or Dropbox or some other cloud service if I want to, but the actual files are local. This just helps with automation.

My initial impressions of Obsidian

Where to begin?

  • I just love the UI. It is clean, and easy to use. It makes me want to live in the app when I am working on the computer.
  • The linking is fantastic.
  • I also love that is has a folding feature built in which allows you to easily show and hide sections of a document.
  • I love that moving notes around (which I do often for organizational purposes) keeps the links up-to-date. I don’t have to think about it, it just work.
  • I love not having to save the notes. They save as I type. Important since more and more tools work this way and I’ve gotten use to not having to manually save documents.
  • Great searching capabilities! I can even search regular expressions, which comes in hand.
  • The Daily Notes feature is awesome and I’ve started using it to create a kind of digital bullet journal.

I’ve liked what I’ve seen enough that I became an Obsidian Supporter. I find the forums and documentation both useful.

Some use cases to illustrate and illuminate

Let me illustrate some of what I like about Obsidian with some real use cases that I’ve got just in my first five days using the tool.

Reading notes

I take notes when I read. I mark up books (real and digital). Ultimately, I like to get those notes together in some useful fashion. (This is the one area in which I have really started to explore Zettelkasten, and I’ll have more to say about it in the future.) Here is one simple example of what I have started to do in Obsidian.

I try to read one feature article from the various magazines I get every day. When I finish an article, I create a note for the article with information about the author(s), source, date, a link to the online version, if available. I also tag the note.

In order to build a kind of searchable index, I have started to create a note for each author. On the article note, I’ll link to the author note, like this, for example:

An Obsidian note for an article in Smithsonian by Glenn Adamson

Note that I have linked to the the author on this note. So when I go and look at the note I have for that author, I see this:

Obsidian note for author with backlinks section listing all of the backlinks

The “Backlinks”section on the note is created automatically through some automation I’ve written. Again, I’ll talk about how I automated this in a future post. I know that I can just go to the backlinks section on the sidebar to see backlinks, but I like seeing the actual references in the note itself.

Presenting with notes

Block folding is incredibly useful. That is, being able to close some blocks and keep others open while looking at a note. Here is an example of this post (yes, written in Obsidian) so far with most of the blocks closed.

Obsidian note with folded headers

I find this allows me to use my notes as presentations in meeting. I’ll jot down the topics I want to discuss in the meeting. List out the points under those topics, and then, before the meeting starts, close all of the headings, so that I can walk through them one by one, sharing my notes on the screen. It allows us to focus only on the open block at hand, while still having a context for the rest of the discussion. No need for slides!

Searching for daily notes

One test I have for any good editor is how well it can search. In addition to the great capabilities that Obsidian has for searching already, it also passes my test for being able to do regular expression searches. Here is a regular express search I have ato surface all of my Daily Notes:

A regular expression search for my daily notes: /^\d{4}\.\d{2}/

I have actually gotten quite a bit of mileage already out of the daily notes features, especially after I added some automation to make it even better.

How I’ve got my Obsidian configured

I’ve already had people asking me thing like what themes I am using, so here is a summary of how I have Obsidian configured for my Macs:

![[Pasted image 20210131113120.png]]

Obsidian configured with my preferences.
My Obsidian configuration

Theme

I’m using Pisum (from the community themes). Incidentally, I did spend time loading every community theme to find one that fit well. I kept a note open and moved the ones I liked to the top of the list as I discovered ones I liked better.

Plug-ins

I’m currently using the following plug-ins:

  • Calendar
  • Daily Notes
  • Zettelkasten prefixer

Other settings I’m using

  • Fold Heading
  • Tab size: 8

Questions I’m still pondering

There are some things I am still trying to figure out, and I’ve been searching the forums to see how others approach these issues:

  1. What should be in my vault. When I started, I created my vault as a Notes folder within my Documents folder. From there, a sub-folder structure began to evolve. But I’ve found that I want to link to documents that are not in the vault, so I am beginning to wonder if really, the vault should be my entire documents folder. I’m looking for advice on the scope of a vault in Obsidian.
  2. Zettelkasten. This seems like overkill for a lot of the notes I have. Put it seems like the perfect solution for how I have envisioned organizing notes from my reading. I used to have a note for a book. All my comments and highlights would be in that note. Now, I have a note for a book, but highlights each get their own note with (a) a link back to the book in question, and (b) a reason for the link. I’ve even started linking some of these extracts to other notes, formiing the web of relationships that Zettelkasten is all about.
  3. Best format for my Daily Notes. I’ve actually done some automation here and getting happier with what I’ve got. But I’m still figuring it out. Here is an example of my Daily Note for today:
An example of a daily note
An example of a Daily Note

Automation preview

So far, I have automated 3 things using Obsidian notes and the plain-text framework:

  1. Collect “to-read” references into a single note. This collects tags marked “to-read” throughout my notes into a single “To Read” note so that I have a single list to look at. My script will ignore items tagged to read if they have already been checked off.
  2. Update People index. This is the script that scrapes notes for backlinks to specific people notes and adds the “Backlinks” section to that person’s notes.
  3. Daily Note generation. This genarates the daily note for the day (just after midnight). I pulls in the agenda from my iCloud calendars. It pulls the to-dos from incomplete to-dos found in other notes, and from any incomplete items in my Apple Reminders. It then formats the other sections and creates the note.

I’ll write about these automations in more detail in a future post.


So there you have it, my initial impressions of Obisian. I’m looking forward to exploring it more. And I’m eager for that mobile app that’s “coming soon.”

One More Note on (Obsidian) Notes

Okay, okay, you’ve convinced me. I’ve been playing around with Obsidian (h/t to Matt Banks for initially bringing it to my attention) for a couple of days now. I think it is the notes system that I have been looking for all my life!

I’m still getting used to some of the ins-and-outs, but I really like what I see so far, and I’ve started converting my Apple Notes (the ones that are not throw-aways) to Obsidian. I am also experimenting with using it as a digital bullet journal, similar to how I use OneNote in my day job for the same purpose. Indeed, I’m using my work to test out the bullet journal aspects of using Obsidian.

It is going to take me a few days to get my thoughts together, as I need time to play with it more. But I was convinced enough not only to try it, but to become a supporter as well.

Thanks again to all who mentioned it. I’ll have more to say about it soon. In the meantime, I’ll leave you with this preview of an Obsidian note for reading notes from a book I recently finished.

A sample Obsidian note

Some Notes on Notes

More and more I find myself trying to simplify things. Take notes as an example. I am a prolific note-taker. Wherever I go, I carry a Field Notes notebook in my pocket, along with a couple of pens. (I have ink stains on various pockets to prove this). Why carry a paper notebook when I have an iPhone in the other pocket? To keep things simple.

Over the years, I have not yet found an app that allows me to jot down notes as quickly and easily as a pen and paper. If something strikes me, I pull out the pen and paper and scribble it down. That’s all there is to it. A phone, at its simplest, involves pulling out the phone, getting through its security measures, opening the appropriate note-taking app, and typing in the note1. In the time it takes me to get through the security measures alone, I could have jotted a simple note with pen and paper.

Then, too, many notes are ephemeral. I’ll use them once and never again. What’s the point of filling up a phone with notes I’m only ever going to look at once? In a notebook, I could tear out the page, but what I typically do it just leave the note there, and when the notebook is filled, I added it to the collection of filled notebooks I have on a shelf in my office.

Of course, pocket notebooks get you only so far. If I am sitting in front of a computer, then I’ll use the computer for notes, especially notes that are not ephemeral. In this regard, Evernote would seem like a logical choice for notes. But I have resisted using Evernote for actual notetaking, preferring to partition it for use as a kind of digital filing cabinet. Instead, out of a sense of simplicity (or stubbornness, depending on your point of view), I’ve migrated toward the Apple Notes app, with one important exception2

There are a few reasons why I have settled on the Notes app:

  1. It is a simple app that is easy to use.
  2. It comes installed on all Apple devices and since I’ve bought into the Apple ecosystem, that makes it a convenient tool. I don’t have to install any additional software to access my notes on a new device.
  3. It syncs with iCloud, so notes I create on one device are available on all of my devices.
  4. It integrates with Spotlight so searching notes is pretty easy.

Item #2 above is particularly important because I keep all of my device bootstrapping-related notes in Apple Notes. These notes include, for instance, a checklist of things I do to new machines and devices (configuration settings, software I install, etc.) I have a file for every device we own which makes for easy reference.

I’ve taken to using Notes for personal development work I do. I’ve also started using notes to keep track of articles I read, copying highlighted passages, or my own annotations there. While it is lacking in a few features3, it has been able to do most of what I need. Here is an example of a HOW-TO note I have in my Tech folder:

A sample HOW-TO note from my Tech folder in Apple notes

The purist in me admonishes myself for not using plain text file for my notes, but you know what? I like being able to format my notes, into lists and tables. I like having hyperlinks, and images. True, each note is not a separate file in the file system. On the other hand, the backend is a SQLite database, which I am perfectly capable of accessing programmatically if needed.

The point is, I haven’t had a need to do so. That is the beauty of the simplicity of Notes so far. I don’t worry about tagging, or notebooks. I do have a folder structure for my notes, and it is evolving, but even there, I aim for simplicity. Being able to simply search for a term in Spotlight and see matching notes has been incredibly useful. I recently read an article in Smithsonian by Richard Grant, whose writing I enjoy. I’d created a note for that article, and so I just tried a Spotlight search for Richard Grant:

Spotlight search for Richard Grant

That’s good enough for my purposes.

I also light the lightweight feel of the Notes app. When I use Evernote today, the application feels big and bulky by comparison. Of course, it does a lot more than the Notes app, but for notetaking, I don’t need much more than what Notes can do.


  1. I stubbornly refuse to use Siri or dictation for notes, although I use Siri for other things.
  2. The exception, not worth getting into here in any detail, is my work-related notes, for which I use OneNote because it makes a lot of sense to do so.
  3. I do wish there was a way to add to the list of default styles provided.