Practically Paperless with Obsidian, Episode 20: Experimenting with the Dataview Plug-In

Welcome to my blog series, “Practically Paperless with Obsidian.” For an overview of this series, please see Episode 0: Series Overview.

One of the things I have always loved about plain text files is that they are easy to manipulate. More than fifty years of effort has gone into reliable methods for parsing text from text files. It is for that reason that for a long time, I maintained my list of books I’ve read since 1996 as a plain text file with markdown¹, and wrote scripts to pull all kinds of data from that file. Other people know this, and Obsidian has a rich community of plug-in developers. It should be no surprise, therefore, that someone came up with a plug-in that allows you to query data from plain text files. Lately, I have been experimenting with the Dataview plug-in, looking for practical uses for querying my notes.

Before jumping into my experiments, I wanted to point out that there is quite a bit of discussion about plug-ins in the Obsidian community and if and how they make the overall system less future-proof. There are good arguments on both sides. Not long ago, I made a post to the Obsidian Reddit group on this subject, outlining my thoughts. Briefly, I found a balance that I am comfortable with and I have no problem using plug-ins like the data tool. Your mileage (and requirements) may vary. There are lots of good opinions. You can find some of them in that Reddit thread.

One thing I really like about the Dataview plug-in is that it is familiar. As a software developer, I’ve worked with dozens of programming languages over the years, but alongside almost all of them was SQL, and the Dataview’s query language is very similar to SQL, which make it easy to learn. The harder part was figuring out practical uses for the dataview. After experimenting for a while to get the hang of things, I came up with a couple of criteria to help identify practical uses for the Dataview tool, over say, a embedded search query. Here are a few of those criteria:

1. Use to access or summarize frequently-accessed data.
2. Use in situations where the data is dynamic, and can change over time.
3. Use to help automating a process

With those criteria in mind, I’ve managed so far to find three places to make use of the dataview plug-in that provide me with practical, everyday use.

1. A Plain-Text Address Book

I have a top-level folder called People, and under that folder, I have a note for any person that I interact with on a regular basis. These notes serve two purposes:

1. YAML frontmatter in each note allows me to capture “address book” information about the person in question so that I have that information at my fingertips. Sure, it duplicates some of what I have in the Contacts app, but again, this is plain text and I consider it my “authoritative” source for this information.
2. The note itself serves as a place to link to when I mention the person elsewhere in my notes. The mention might be in a daily note, or journal, or some other note. By using a link to the person-note in my daily notes and other notes, I can see everywhere that person was mentioned by looking at the backlinks of the person note itself.

Now, I can open up the People folder and scan down the list of notes there, all titled in “Last, First Name” format, but all I can see are the names of the people. No other information. I have to click on a note to see a person’s phone number or email address, or birthday. What I needed was a dynamic index, and that seemed to me to be the perfect use case for the Dataview plug-in.

I created an “Address Book” note and in that note, I included the following Dataview query:

This generates a nice table of all of the notes under the People folder, displaying the name of the person (as a link to the note), along with their email, phone, and birthday. (Note: for the purposes of this post, I copied my People folder and then used a fictional name and contact info for each person to make things easy on me.)

When I change information in an individual note, it is automatically updated in the table when I look at the table. And from the table, I can easily locate a person and contact information. I’m sure this isn’t unique. I’d guess that this is one of the first practical ways that many people make use of the dataview plug-in.

2. A Dynamic Idea List for the Blog

Those who follow this blog for more than just my Obsidian posts know that I write about just about anything that comes to mind. I’ve got more than 7,000 posts going back more than 16 years now. Last year, I had a goal for publishing a post every day. This year, my goal was expanded to continue to publish a post every day, but to write two posts every day. That allows me to build up a backlog. But it also means I need a useful way to collect ideas to write about.

Typically what I do is jot ideas down in the Field Notes notebook that I carry with me everywhere I go. Later I transfer them into my daily notes. Or, if I happen to be sitting at the computer, the ideas go directly into my daily notes. The ideas go in the following format:

Since the format is consistent, this also lends itself to query that allows me to look for note ideas I haven’t yet written. Looking at how the Dataview queries work, I began to think of this query as “any task in my daily notes tagged with ‘post-idea’ that isn’t yet completed.” I experimented this with a straight dataview query, but I couldn’t quite get it to work the way I wanted, mainly because I couldn’t figure out how, from a TASK query, to filter out things by tag. Instead, I used a dataviewjs query. I created a note called “Post ideas” and then added the following dataviewjs query:

What the query does is it looks for all notes tagged with “#post-idea” where the text of the task also contains the tag “#post-idea” and the task is not yet completed. The results look like this:

There are several things I really like about this:

1. Because the ideas go in my daily notes, and the task query renders the name of the file that the task appears in, I not only get a list of un-used ideas, but I can also see how old they are beacuse they are listed under the daily note in which they were added.
2. I have one place to go to get a list of ideas to write about.
3. Once I finish writing a post, I click the checkbox on the task and it immediately drops out of the “not yet written” section of the Post ideas file and moves into the “Completed” section of the file.
4. If I don’t end up using an idea (I don’t use all of them, just the good ones), I’ll cross it out.

3. A Summary MOC for Services and Subscriptions

In Episode 18, I wrote about how I kept a folder with a note for each service or subscription we have. The YAML frontmatter contains information about the subscription/service like the monthly and annual cost, renewal date, etc. In addition, I use those notes to document any interactions I have with the service provider, so that all of the information related to the service is in one place and easy to find.

One use I found for the dataview plugin was to summarize all of these services in table form. This provides a nice summary index of all of the services and subscriptions we have that are currently active. If a new one is added, it automatically appears in the summary. If one is canceled, it drops off the summary list.

The note contains the following dataview query:

and the resulting table looks as follows:

This makes it easy to find everything we subscribe to. I can jump to a service by clicking on a note link to get more detail about the service.

---

I am still experimenting with the dataview plug-in, but these are a few of the practical uses that I have found, and that I use on a regular basis. Their dynamic nature provides simple automation that saves me a fair amount of time across several types of tasks that I perform several times each day.

In next week’s post, I’ll have something to say about tags and tag taxonomy, something I’ve struggled with since my early days using Evernote. See you back here next week.

Prev: Episode 19: Archiving Notes
Next: Episode 21: Tagging Notes: A Decade-Long Struggle

Written on February 27-28, 2022.