Practically Paperless with Obsidian, Episode 22: Daily Notes Revisited: The Best of Both Worlds?

Welcome to my blog series, “Practically Paperless with Obsidian.” For an overview of this series, please see Episode 0: Series Overview.

A couple of months ago, in Episode 15, I wrote about how I use daily notes as an index to my life. I was impressed by a post I’d read on someone who kept all of his daily notes in a single file, and I aimed to try to reproduce that in Obsidian. Since writing that post, I’ve tweaked things somewhat–enough that I felt it was worth reporting back on. Since this series is an ongoing experiment in going practically paperless, it isn’t unusual for me to change the way I do things when I find ways and methods that work better.

Two competing requirements

In the case of daily notes, I found myself pinned in by two competing requirements:

1. I want the ability to see more than just “today’s” note. I like to be able to easily scroll through past daily notes. This is a quick and easy way to review things without having to search through a bunch of files, or open a bushel of notes windows to reference something I am looking for. This is what attracted me to the single daily notes file in the first place.
2. I want to use the native daily notes functionality that comes with Obsidian. This functionality, however, is based on a single-note-per-day, and I’ve found it just doesn’t work nearly as well when all daily notes are in a single file. The main problem was that if I linked to another note from my daily note file, the backlink referred to the daily note file, but not the date in the file under which the link was listed.

So: I want to use a single file for all of my daily notes, but I want the daily notes functionality that is based on one note per day. What to do?

First solution

In order to address this conundrum, I decided to go back to the single-note-per-day. That immediately gained me the native daily note functionality in Obsidian. In a single-note-per-day model, if I link to another note, the backlink shows the date of because it is the name of the daily note file itself. For instance, I list out the blog posts I write on a given day within my daily notes file. In the case of this post, the backlink shows what day I wrote the post:

This is much more useful than when I used a single note for all my daily notes. In that situation, the backlink would show up as follows:

and “Daily” doesn’t tell me the date. That is a big drawback of the single-file model. As I pointed out in Episode 15, one thing that would help would be if the section path that the link was under was displayed as part of the backlink — that at least would pick up the date. I even submitted a feature request for this. But it doesn’t exist now and I have to work with what I’ve got.

One thing I did to attempt to get the best of both worlds was to setup my workspace so that the previous day’s note always appeared in a window above the current day’s note. That way I could easily scroll through the previous day’s note if I needed to reference something there. I pin the current day’s note, but leave the previous day’s note unpinned so I can quickly switch to another day using the calendar.

I played around with this model and it worked okay, but it just wasn’t as useful as being able to scroll through all of my daily notes in a single file. It did, however, get me the benefit of the native daily note functionality in Obsidian, so that was a partial win.

Second solution

After giving it some thought, I came up with another solution that works even better: a single daily notes file made up of transcluded links (embeds) to individual daily notes. This is achieved as follows:

1. Daily notes are created normally, one-file-per-day, using native daily note functionality in Obsidian.
2. At the end of each day, I add a transcluded link to the current day’s daily note to the top of a note file called “Daily”

The “Daily” note file looks like this in source mode:

When viewed in Live Preview mode, however (which is my default), this same file looks as follows:

This allows me to scroll through all of my individual daily notes as if they are in a single file, even though the daily notes themselves are each in a separate file. It allows me to work in a workspace where I keep the Daily file pinned at all times, even while having today’s daily note open in a separate window, like this:

In this setup, I can easily scroll through all of my daily notes going back to when I started intending to keep them in a single file (December 28, 2021). What’s more, as I add new items to my current daily note (the window on the lower-right), they appear instantly in the file on the left.

One change I made in the “Daily” file from my previous attempt at a single file is that I reverse sort the entries so that the most recent day appears at the top of the file. I found that I was constantly scrolling to the bottom of the single file to add new notes, and while I like the strict chronology, it was more practical to reverse sort it. Several readers pointed out to me that this is what they did, and it made a lot of sense to me.

While the embedded text within the file is not searchable in source mode or Live Preview mode, it is searchable in Read mode. And I can leave the file in read mode because I only have to made an update to it once a day, to add the current embedded daily note file. That means, if I hit Cmd-F to search in the file, results show for all of the embedded files. For instance, if I search for the term “- read” here is what the results look like:

Note that the search results span multiple days in this view, meaning they span multiple embedded files. This is what I was hoping for when I began looking into this solution.

The best of both worlds?

This solution — using individual daily note files and a separate Daily file with embedded notes via transcluded links — is better and gets me closer to the two competing requirements I’ve been aiming for. I can use native daily note functionality; I can see references to daily notes in backlinks, and I can scroll through the daily notes as if they are all in a single file. But it is not perfect. There is still at least one challenge to overcome, and a relatively easy one at that.

Challenge: Automating the Daily file

As I said, currently, at the end of each day, I add a new transcluded link to the current daily note at the top of my Daily note file. If I forget to do this, I won’t see the previous day’s note. This isn’t all that cumbersome, but it is something that I want to automate.

A simple shell or Python script can take care of that. The script will add the trancluded link to the new daily note at the beginning of each day so that I can see it there at the start of the day, instead of just at the end. This really is a simple script and when I get it written, I’ll post a link to it on GitHub¹.

Daily summary and Dataview

There is one other adjustment I’ve been playing around with in my daily notes. Now that each day is in a separate file, my daily notes template includes YAML frontmatter for capturing some information that I care about. Currently, that template looks as follows:

---
locations: ["Arlington, VA"]
sleep: 1
summary: Good writing day, busy work day. Upgraded Obsidian. Good response to latest PP post
---

Here is what this data is for:

• Locations: a list of places that I was at on a given day. Not every place I go, but rather where I was based on that day. On days I travel, this list contains multiple entries, like [“Arlington, VA”, “Wisp Resort, McHenry, MD”]
• Sleep: a rating of the quality of my previous nights sleep on a scale of -2 to 2, where 0 is an average night, -2 is a absolutely terrible sleep, and 2 is a dreamless, perfect sleep. For those curious, I learned of this method listening to Jim Collins describe this method on the Tim Ferris Show podcast.
• Summary: A short, one line summary of my day, written at the end of the day.

I use this in a Dataview table that provides a kind of index to my daily notes. At a glance I can see where I was on a given day. I can also use the summary to help remember what happened on that day. I can then drill into the specific note for more details. I have a note called “Daily Notes Index” and within that note, use the following dataview query:

```dataview
TABLE summary, locations FROM "Daily Notes"
SORT file.name DESC```

Here is what this Daily Notes Index looks like for March 2020 so far:

This hasn’t replaced my Daily file containing embedded daily notes. Instead, I find this file more useful as a high-level review of my days, something I can scan through at the end of the month or end of the year, or a place I can go to quickly see where I was or what I did on a certain day without flipping through the details of the individual daily notes file.

Putting it all together

To sum up the current state of my daily notes:

1. I have gone back to using a single-note-per-day to get the most benefit from Obsidian’s native daily note functionality. I always have “today’s” note open in a pinned window.
2. I maintain a “Daily” note that emulates the all-daily-notes-in-one-file that I started with back in Episode 15. This allows me to easily scroll through all of my daily notes and search them without having to open multiple files. Given the way that I work, this is more efficient for me.
3. I use the YAML frontmatter in my daily notes to populate a dataview table in a Daily Notes Index file that I use periodically for skimming my days

This is as close as I’ve been able to get to the best of both worlds: each day in its own note, and all notes in a single file. So far, it seems to be working for me, but as always, I am open to ideas an suggestions for improvment.

In next week’s post, I’m going to change pace a bit and talk about my personal views on information security with respect to my notes in Obsidian. I’ve been asked questions about security quite a bit. This was also try when I was writing about using Evernote a decade ago. The post will summarize how I think about security, some tools I use to protect my data, and what I do and won’t worry about. See you back here next week!

Prev: Episode 21: Tags in Theory and Tags in Practice (And Never the Twain Shall Meet?)
Next: Episode 23: Some Practical Thoughts on Protecting My Data

Written on March 9 and 11, 2022.