Tag: mathematica

Functional Programming

close up photo of person typing on laptop
Photo by Christina Morillo on Pexels.com

Late last year I set myself some goals to tackle in 2023. These goals fell into three areas in which I am seeking improvement: (1) consolidation, (2) simplification, and (3) automation. In setting those goals, I wrote the following:

I believe I can get away with two primary work product formats: plain text files used in conjunction with Obsidian; and Wolfram Language notebooks (.nb files) used in conjunction with Wolfram Language and Mathematica.

Halfway through 2023, I think I have been largely successful. My daily notes and all things related to events in my life (and that of my family) are recorded in Obsidian. Meanwhile, working notes, coding, writing, and automation has happened primarily in Wolfram notebooks, and using the Wolfram Language as my primary framework.

I’ve been a hobbyist user of Mathematica and Wolfram Language since 2009. I like its all-encompassing scope, its ability notional that nearly everything is computable, and its coding framework . A great thing about the Wolfram Language is that it can be used for complex mathematical experiments, or simply to copy files from folder A to folder B. It is a language that can be used for any task and that is how I have been using it. In the six months since setting my goals, for instance, I’ve done the following using Wolfram Language notebooks and scripts:

  • Created a script that adds work products for a given day to my Daily Notes file in Obsidian. Work products are defined as any file that I created or edited that day: notes, scripts, spreadsheets, etc. The script runs automatically each night and in the morning, I can see the files that I worked on the previous day.
An example of my Daily Notes file with the “Today’s work products” section generated automatically by my script
  • Created a script that selects a random article from the magazines I read and emails me the article the evening. I read a feature article each day and this script eliminated some decision fatigue while simultaneously adding a little spontaneity and surprise to the day — I never know what will get selected.
  • Create a script that exports my current reading list to an Obsidian vault, complete with charts and stats. This vault is published to the web using Obsidian Publish. For decades this was a manual task for me and now it is fully automated. All I have to do is keep my spreadsheet up-to-date.

I learned to write code in the early 1980s by trial and error. And from that first foray with BASIC through this year, I’ve been a procedural programming — a method by which a set of procedures are called in step-by-step fashion. Those procedures might be functions, events or anything that can be called at any point during the program’s execution. Wolfram Language can do procedural programming, but its real power comes from its ability to do functional programming. After a lifetime of procedural programming, I have begun to teach myself functional programming.

Over the weekend, I began exploring the use of stylesheets in Wolfram Notebooks. I created a simple stylesheet for writing, with styles and elements that I use frequently. There is a folder location with the Mathematica framework for these custom stylesheets to be stored. I had several choices:

  1. I could simply save my custom stylesheet to the folder in question.
  2. I could store the stylesheet in one place and copy it to the folder in question whenever I made a change.
  3. I could automate the process.

One of the things I’ve learned as a professional software developer is that it is worth spending the time to do things the right way the first time. I opted, therefore, for option 3.

In my Documents folder, I have a folder called Settings, where I keep my preferences for all of the software I use. This includes things like settings files, templates, stylesheets, preferred fonts, etc. This is the source for my personal preferences and when I get a new machine, it makes it easy to apply all my settings to that machine.

My settings folder

I placed my newly created Writing.nb notebook in the Mathematica folder in my Settings folder. Whenever I updated my stylesheet, this is the file I’d update. But for the file to appear in Mathematica, it needs to live in a special folder in my library. For that, I write a Wolfram Language script. The script does the following:

  • makes a list of all files in my StyleSheets folder (a sub-folder of my Mathematica folder)
  • for each of the files in the list:
    • checks to see if the file exists in the Mathematical library
    • if it exists, it checks to see if my master copy has been update more recently than the version in the library folder; if it has, it copies the updated version to the library folder

I wrote the code as I have written most of my Wolfram Language code, procedurally. It looks as follows:

Do[
 If[FileExistsQ[
   FileNameJoin[{customStyleSheetDestPath, customStyleSheets[[i]]}]],
  If[FileDate[customStyleSheets[[i]], "Modification"] > 
    FileDate[
     FileNameJoin[{customStyleSheetDestPath, customStyleSheets[[i]]}, 
      "Modification"]],
   CopyFile[customStyleSheets[[i]], 
    FileNameJoin[{customStyleSheetDestPath, 
      FileBaseName[customStyleSheets[[i]]] <> "." <> 
       FileExtension[customStyleSheets[[i]]]}], OverwriteTarget -> True
    ]
   ],
  CopyFile[customStyleSheets[[i]], 
   FileNameJoin[{customStyleSheetDestPath, 
     FileBaseName[customStyleSheets[[i]]] <> "." <> 
      FileExtension[customStyleSheets[[i]]]}],
   OverwriteTarget -> True
   ]
  ],
 {i, Length[customStyleSheets]}
 ]

Having written and tested the code, I decided that this was a simple enough task for me to try to rewrite the code as a functional program instead of a procedural one. To my surprise, I’d was able to do it fairly quickly. I was further surprised that it worked on the first try. Here is the same program, written as a functional program:

If[FileExistsQ[FileNameJoin[{customStyleSheetDestPath, #}]],
    If[FileDate[#, "Modification"] > 
      FileDate[FileNameJoin[{customStyleSheetDestPath, #}], 
       "Modification"],
     CopyFile[#, 
      FileNameJoin[{customStyleSheetDestPath, 
        FileBaseName[#] <> "." <> FileExtension[#]}], 
      OverwriteTarget -> True]],
    CopyFile[#, 
     FileNameJoin[{customStyleSheetDestPath, 
       FileBaseName[#] <> "." <> FileExtension[#]}], 
     OverwriteTarget -> True]
    ] & /@ customStyleSheets;

It looks much shorter, and is probably a bit more difficult to read if you are not used to working with functional programs. Essentially, this version of the program defines the entire procedure as an anonymous (i.e. Lambda) function and then maps that function over the list of style sheet files.

While this likely seems obscure and insignificant to most people, it was an important milestone for me. I did this early Saturday morning, and I was excited about the results all weekend. Indeed, I was excited enough by the results to want to tell people about it in a post. This was a milestone for me. Something about functional programmed had clicked in my head and, it seemed, I suddenly got it.

This isn’t the first time something like this has happened to me. I remember in late 1994 or early 1995 when I was teaching myself Perl, I looked at regular expressions as utter gibberish, something I would never understand. A year later, I recall, to my amazement, that I could in almost-realtime craft regular expressions for a pattern in my head. Like functional programming, it just clicked one day.

There is a lesson here. I set about my Saturday morning with a practical task in mind: creating a stylesheet for writing in Wolfram Notebooks. I came away with that stylesheet, but much more important, I had new confidence in my ability, and now I am eager to try flex my functional programming muscles even further.

Written on June 26, 2023.

Did you enjoy this post?
If so, consider subscribing to the blog using the form below or clicking on the button below to follow the blog. And consider telling a friend about it. Already a reader or subscriber to the blog? Thanks for reading!

Follow Jamie Todd Rubin on WordPress.com

Goals for 2023

pexels-photo-13088176.jpeg
Photo by Engin Akyurt on Pexels.com

Over the years I have spent quite a bit of times on various experiments. In the 2010s, I considered the idea of the paperless office, embraced Evernote as a tool for paperless productivity, and wrote a popular series of posts on the subject. So far, in the early 2020s, my experimenting has shifted somewhat. I’ve re-embraced the idea of plain text files as my fundamental unit of work. A plain text file is both versatile and long-lasting. Text files created in the 1970s can still be read today. I found Obsidian, and took lessons from my Going Paperless experiment, to see what I “practically” paperless[1. As in “practical” uses, as opposed to absolute migration to a paper-free lifestyle.] lifestyle looked like using primarily plain text files.

In 2022, I’ve been pleased with how things are working out with Obsidian and plain text files, but there is room for improvement. I’ve identify three areas to experiment with throughout 2023. My hypothesis for this experiment is that improvements in these three areas will lead to more creative time in the years beyond. The three areas are: (1) Consolidation; (2) Simplification; (3) Automation.

Consolidation

When I look at my computer, I am often overwhelmed by all of the apps and tools I find on it, the vast majority of which I rarely use. It would be nice, I tell myself now and then, to get rid of those tools that I never use, but I am sometimes reluctant to do so, thinking that I might need the tool some day. To some extent, this is no different than trying to declutter at home.

While thinking about this recently, I realized that I made huge strides in consolidation over the last two years in terms of my work products, the vast majority of which are now plain text files. With a work product as simple and versatile as a plain text file, it should be much easier to give up apps I no longer need to manipulate those files.

The more I think about this, the more I believe I can get away with two primary work product formats: plain text files used in conjunction with Obsidian; and Wolfram Language notebooks (.nb files) used in conjunction with Wolfram Language and Mathematica.

Using text files as work products of Obsidian is pretty obvious and I’ve written extensively about my use of Obsidian. But Mathematica1?

I’ve been a very casual, hobbyist user of Mathematica for years now. But the more I play with it, and the more I learn about its symbolic and functional structure, the more impressed I am with the language. Moreover, the language now reaches into all aspects of computing so that it seems useful as a general purpose language. Finally, as a developer, I’ve already mastered more common languages like Python, JavaScript, C#, Perl, PHP, Ruby, etc., to my personal satisfaction. Becoming equally proficient in Wolfram Language would something new.

I am mentally dividing this consolidation into two parts:

  1. Plain text files as my primary work product: in the form of notes and writing.
  2. Wolfram Language scripts and notebooks as my primary tool for development and automation.

As I write this, there are about 100 applications in my Application folder on my laptop. That, of course, does not include the command line tools inherent in any Unix-based computer system. I’m using this number of 100 applications as a baseline to see how much I can consolidate over the course of 2023. Obviously, I need more than just Obsidian and Mathematica. For instance, I use Photoshop for photo editing. However, Mathematica has powerful photo- (and video- and sound-) editing tools that may be able to automate and supplant most of what I do in Photoshop. This is an example of the consolidation I am looking to do.

It will be interesting to see how well this consolidation works over the course of 2023.

Simplification

As I worked through my Practically Paperless series, I noted that I was building up more and more complex structures for my notes, as well as relying increasingly on plug-ins, tags, etc.

In the months since finishing that series, I’ve been attempting to simply how I use Obsidian with plain text files. In the process of simplifying, I have the following things in mind to guide me:

  1. My primary work product is a text file. Whether a note, a blog post, a list, a trip summary, a reading note, the primary form in which the content appears is a plain text file.
  2. Using tools like Obsidian is a good for me, but I also keep in mind that these text files need to be useful in the absence of a tool like Obsidian. Keeping the work products themselves simple help in this regard. For example, the structure of markdown makes for simple way of parsing and interpreting these files outside of Obsidian.
  3. I’m trying to simplify my file structure, both within Obsidian and as a whole, including cloud platforms.

Over 2023, I’m looking to continue this simplification of my work products. With the following ideas in mind:

  1. My notes need to be usable outside Obsidian.
  2. My notes and files need to be easy to understand, not just for myself, but for others, especially Kelly and our kids.
  3. My notes and files need to be easy to find for me and for the family.

Automation

I spend a lot of time trying to automate processes. The idea here is to automate the stuff that is repeatable, so that I can spend more time on creative stuff. But often, the time it takes to implement such automation offsets the time it saves. Put another way, if it takes me 2 hours to write a script to automate a task that currently takes me 5 minutes per day, it will take 24 days once the automation is in place to pay back the time it it took to build the automation before it starts paying off.

In the past I’ve used all kinds of tools to implement automations: Python scripts, tools like Alfred or Keyboard Maestro, Apple Shortcuts, etc. In 2023, I am looking to use Wolfram Language as much as possible to automate things, in part because I’m fascinated to see the possibilities in the language, and in part as a way of becoming proficient in the language.

One simple example of this, which I will talk about in more detail in a subsequent post, is adding a list of files I created or updated on a given day to my Daily Notes. it is convenient for me to be able to see at a glance all of the files I worked on in a given day. The way I structure my daily notes makes the daily note the perfect place to store this information. But I don’t want to spend time searching my file system each day for all of the files I created or modified. So I’ve created a Wolfram Language script to do this automatically.

An example of my current Daily Note format, using today's note.
An example of my current Daily Note format, using today’s note.

Throughout 2023, I’ll be looking at how I work2 to identify inefficiencies, things that can be eliminated, and tasks and processes that can be automated, particular:

  1. Things I do frequently and are repeatable. In addition to the above, examples, might include: taking a blog post in note form and automatically publishing it to WordPress; or cropping and editing a photo in a repeatable way
  2. More complex tasks that I perform less frequently but at regular intervals, like archiving information.

Follow along on the journey

Because I know that other people are interesting in this sort of thing (a quick search of “productivity” gives an idea of just how much), I plan to write about this journey of mine throughout the year. As always, I am experimenting to see what works best for me. I know, for instance, that using Wolfram Language for a primary automation and scripting tool is probably outside the norm for many people. But I’m fascinated by the idea and that makes it fun for me. These will tend to be more technical posts, but they might be of interest to others, so, feel free to follow along, ask questions, and offer suggestions.

And Happy New Year, everyone.

Written on 29 December 2022.

Did you enjoy this post?
If so, consider subscribing to the blog using the form below or clicking on the button below to follow the blog. And consider telling a friend about it. Already a reader or subscriber to the blog? Thanks for reading!

Follow Jamie Todd Rubin on WordPress.com

  1. There are other reasons for me to use this, but they are too technical to get into here.
  2. To be honest, I’m doing this constantly, but this year, I’ll be putting a special emphasis on it.