Tag: coding

More database progress

Over the weekend, and this evening, I’ve been working away at getting my book collection and reading databases converted from their old formats into a normalized MySQL database. This has been done and the database was successfully imported into my instance of MySQL with my ISP.

The next step is to write the code to allow people to access and query these databases. I started off using perl as a quick and dirty way to make sure I could programmatically access the databases over the web. Once I established this, I dumped perl and started the actual writing of code using PHP, which is much easier for this type of work. This phase will take longer as I am trying to modularize the code in a way that makes sense, but based on my progress this evening, I may have a new version of my reading list up and running in a few days.

The new versions will initially be mostly style-free: this means that no CSS or other fancy styles will be to applied to the pages; just some basic HTML formatting. Making it look pretty is secondary to functionality as far as I am concerned. But eventually, it will look pretty too.

Free day

Today is my free day–no workouts and I can eat whatever I want. I slept in until 9 AM, which is late for me and then I had a quick breakfast and headed out to the grocery store to do some shopping. Since then, I have been working on the book collection and reading database applications and have made good progress in converting the old formats to the new data model. I should be able to start writing actual code soon.

I’ve been listening to Pink Floyd all day. So far I’ve listened to Piper at the Gates of Dawn, A Saucerful of Secrets, Meddle, Obscured by Clouds, and Dark Side of the Moon. I’m now in the middle of listening to a live version of the Dark Side of the Moon album.

I talked to Doug earlier today. Their power has been restored and they should be able to go home today. (They live in Seattle and were affected by some of the storms there.) I also talked to jen_ashlock briefly. She and jkashlock are in San Diego this weekend and take the red-eye home tonight.

I’m off to treat myself to something good to eat.

Database development

Up just after 8 AM and breakfast. Then I started on the conversion and migration of my book collection and reading lists databases from Booxter/MySQL on my local machine, to the instance of MySQL I now have through my ISP and I have been working on that most of the day thus far, making some pretty good progress. This has to be done before I can write any of the perl or PHP (more PHP than perl, actually) code for the web interfaces. There are two parts to the web interfaces: public-facing, which is what people who go to my website will see; and private-facing, which is what I will use to keep the data up-to-date.

But I’m taking a break at this point to head over to the gym and do my strength training workout. This afternoon it’s arms and shoulders.

Improvements to online databases coming soon!

At home, I maintain my book collection, reading list, and story submissions databases within MySQL on my iMac. I have scripts that take this data and either build static pages, or create flat files that can be used by perl or PHP scripts to present the data on my website. Currently, my reading list is of the former category, static pages updated nightly. My book collection and story submission log are of the latter category, where flat files of the data are uploaded nightly and the data is presented via perl scripts on the website.

As of this morning, I have signed up for the MySQL database service offered by my ISP, Panix. With this service, I can maintain all of this data in MySQL databases hosted by my ISP and access these databases via Perl, PHP, Python, and C. What all of this means is that in the coming weeks, there should be some dramatic improvements to the capabilities you will see in these three databases, better searching, richer information, some pretty cool stuff.

Developing the public-facing code is not the problem (after all, I make a living doing application development, most of which involves databases). Finding the time to develop the public-facing code is another issue. But keep your eyes peeled for these changes. I’ll be sure to announce them as they are completed.

Book collection database online

Yesterday, I worked at getting my story submission log online. Today, I began the process of getting my book collection database online. I’ve stated with a few simple, common queries and over the next few weeks, I will add in some additional features, including the ability to search the database. For now, you can see things like most frequent authors and signed books by going to:

http://www.jamierubin.net/pcgi-bin/books/collection.cgi

In the coming weeks and month, I am will be working on changes to my web site. I’m not worrying about look and feel for the moment, just trying to improve the content. The submission and book collection databases are the first steps in that direction. You can also look forward to improvements to my reading list as well as some useful scripts to help track my workouts.

Mine will be a web site in the true geek sense of the words.

Story Submission Log

I’ve been meaning to do this for years, and I finally got around to it this afternoon. For the last 14 years, my haphazard system for keeping track of my story submissions has been a set of index cards. Yes, that’s right! I make my living as a software developer and I still use index cards in place of a database. The system is far from perfect, but it has worked for me, for the most part, except between 1998-2000 when I was hardly writing at all.

Today, I have not only captured this information in a simple database, but it is available for viewing online at:

http://www.jamierubin.net/pcgi-bin/submission.cgi

Clicking that link will show you my active submissions: those stories that I have sent out and that I have not yet heard back on. There are a few links at the top of the page that will allow you to view all submissions in the database. Also, you can view submissions by story are market by clicking on the link for the corresponding story or market. It’s a very simple system but I hope to make some improvements in the not-too-distant future.

The system currently contains my submissions back through 2000. When I have a little more time, I will get the rest of the submission into the database. My list goes back to 1993 so there is still a lot more to get in. I will also be adding some submission stats when I have more time. But I’ve done enough for this evening.

Odds and ends

It has already been a busy and productive morning, and it’s only 10:30. Much of my morning has been spent writing some perl scripts to transform and import a large amount of legacy data into a newly designed database for the data. It is a multi-step process but I’m pleased with the progress I made this morning.

When I got into the office, my iBook AC adapter was sitting on my meeting table, right where I left it.

Smallville is a repeat tonight, but Gray’s Anatomy is all new. I think I’m going to take the opportunity to skip TV tonight and make progress in other arenas. I’d like to finish up Humans today, and maybe even get a bit of writing done this evening.

This was a meeting-heavy week for me, but some meetings have fallen off my calendar, which has helped a little bit. I blocked off my calendar for next week, indicating to one and all that I am not available for any meetings. We’ll see how well that works, but I’m hopeful I’ll be able to get some meaningful work done.

I found out today that my boss will be in town next week (he works in Santa Monica, while I work in Arlington, Virginia). He’s taking me to lunch on Friday, December 8 because I won’t be able to make my team lunch in Santa Monica the following week.

The DirecTV service people are coming tomorrow afternoon, and so I’ll be working from home in afternoon tomorrow. I was scheduled to interview someone in the afternoon because I had forgotten about the service appointment so I don’t think I’ll be able to participate in the interview; fortunately, there are others to fill in for me.

It’s going to hit 70 degrees in the Metro D.C. area today–for the last day of November that has got to be close to a record.

Late lunch

I had a team meeting at noon today (my entire team is on the West Coast, so it was 9 AM out there), so I pushed my lunch hour back to 1 PM and so here I am. I had a very productive morning, hitting all my milestones for today and then some. I’ve got a little more coding to do after lunch and then I need to start on some basic user documentation for this application.

I didn’t do much reading over the weekend, but I picked up Lunar Prospector on the train into work day and I’ve reached the half-way mark. I originally predicted I’d be finished with the book on November 11, but it looks as though it will be a bit longer than that.

I had another interesting dream last night, but more on that later. For now: eat, read, nap. Then back to work!

The Model

I don’t use Quicken or Microsoft Money for keeping track of my finances. Since 1997, I use some software that I developed myself and which I call “The Model” for doing this and it has worked very well for me in all of that time.

I am about to start down the road of doing the first major revision to this software in several years, adding in some new features and making some efficiency enhancments.

Details on the history of this software

The cost of not being skeptical

One of the projects on which I am working is a conference room reservation system of inordinate (and unncessary) complexity. This project involves the integration of three different software packages: (1) An off-the-shelf conference room scheduling package; (2) Microsoft Exchange; and (3) an off-the-shelf tasking package. None of these packages have native interfaces to any of the others and integrating them has been a challenge. (My specific role, aside from being an advisor) has been to integrate package (1) with package (3), something that I have done successfully and one of the few successes ofthe project thus far.

The off-the-shelf conference room scheduling package is developed by a small company, but they have been fairly responsive in acceeding to customizations we have requested. We ask for a customization, they give us a time estimate and a cost, and we then pay them to make the customization. But the product itself, in my opinion, is not very well written. I have read through much fo the code because I needed to integrate it with the tasking product so I have a good sense of this. Early on, I discovered, for instance, that when you book a conference room, the software sets a default meeting time of 2 hours. This is just too long. Most meetings don’t last 2 hours (they just seem like they do). If people aren’t paying attention, they will end up booking a conference room for twice as long as it’s needed and that prevents other people from using an already scarce resource.

I pointed this out months ago, and also pointed out that in a better written application, the default length of a meeting would be a system setting. But in this application it is not. It is hardwired into the product. I pointed out that if I had some time, I could find the problem and fix it, but I didn’t have the time to do it. Instead, we contacted the manufacturer and asked them to make this change. They were happy to do it for us. They told us it would take 1 month, and it would cost $1,700 to evaluate the change (figure out what they needed to do) and another $1,700 to “fix” it the way we wanted it. This was a few months ago. We didn’t expect them to get started right away. The “1 month” would begin once there was an opening on their calendar.

Yesterday, frustrated that this hadn’t been fixed, and by the fact that it seemed to me we were being over-charged, I decided to tackle this problem on my own. Their code is somewhat of a mess, if you ask me. So instead of working on what I was supposed to be working on, I set aside an hour to solve this problem. It ended up taking me 57 minutes. It was like untying a knot, but with some trial and error, I eventually found the code that set the default meeting time. Sure enough, it was hardcoded in at 120 minutes (2 hours). I changed this to 30 minutes, which is what The Powers That Be wanted it set to. I tested it and it worked great!

Here is where my skepticism paid off. For budgeting purposes, my team “bills” our time at $500/day. There are 8 hours in a day or 480 minutes. It took me 57 minutes to solve this problem, and doing the math, you could say that to solve this problem it cost $59.38 of my time. Compare that to the 1 month and $3,400 estimate that the guys who actually made the software quoted us. Essentially what this means is that their cost quote was 57 times what it actually cost me to fix the problem. And their time quote, in terms of business hours, was 168 times the time it took me to fix the problem.

The irony is that no one really questioned this when they gave us the quote. They merely said “okay” and issued the check. Once the check was issued, there is nothing we can do to get the money back. We emailed the company yesterday telling them that we fixed the problem, what we did and how long it took. We argued that in the interested of maintaining a “trusted partnership” they should return the money. But of course, they are under no legal obligation to do so. So who is really at fault here?

I’d argue that the vendor genuinely thought it would take a month to solve the problem. They simply don’t know their own code well enough to give a better estimate. I’d argue further that we should have been more skeptical about the quote to begin with. We should have asked for more detail about why it cost so much or why it would take so long. But we didn’t. So perhaps we got what we deserved in this case. I’d love to think that we would learn from this experience and not repeat a mistake like this in the future, but you know me: I’m skeptical.

Speed-coding

I am working on an application prototype, that will eventually evolve into a full blown application that allows the tracking of projects through their life-cycle (beginning with the funding and through the closing of the project). As part of this, I have outlined an API of about 50 stored procedures that provide the database functionality for this application (which is otherwise an ASP.NET application). Now, since this is a prototype, I’ve just been coding away, but in the next two days, having the stored procedure API would speed up my ability to do my work. So… I am going to spend the rest of the morning doing some speed-coding. Just how many of the 50 stored procedures can I get written in the next 2-1/2 hours?

I’ll let you know after lunch…

The Zone

This morning, I was in “the zone”. I had a very specific set of tasks I wanted to complete, and with some focus, I completed them! These included finishing off design documents for a .NET project I am managing, as well as fixing a problem with the installation of some .NET libraries on my work machine.

I’ve still got a long way to go and some pretty short deadlines, but I was very pleased with what I got accomplished in the first 6 hours of the day.