mlissner's blog

Research Idea - The Age of the Internet

I blogged a while back about a Firefox command that would tell you the last modified date of the page you were looking at, and it got me thinking....what is the age of the Internet on the whole?

I've been thinking about it a bit, and it seems like knowing this kind of information could prove pretty useful for certain circles. If there was a way to summarize the last modified date of every page on the Internet, we could pretty easily figure out how useful the information is.

Firefox add-on?

Meyer Lissner Was Famous (Go Figure)

There has been some talk in my family over the years about my great grandfather, Meyer Lissner. Over the holidays, I scanned some old photos, and discovered that he was quite the famous fellow. Below is a liberty ship that was named after him in 1943.

I also learned that my grandfather donated a LOT of his papers to the Stanford Library as the Meyer Lissner Papers. I haven't read up much on him yet, but he seems pretty influential to California history as part of a reform movement.

Interesting stuff. I'm going to have to do some researching on this fellow one of these days.

Food Choices

After having visited with so many people over the last couple weeks, I have been thinking that writing down some of my food restrictions would be useful. A lot of people ask me about these, and when they do, they really want to get it right. Hopefully this will help.

I've listed the generalities first, followed by exceptions to the rules.

Foods I Don't Eat

  • Meat - This includes cow, pig, chicken, fish - essentially anything with an eye except for potatoes. You can also find meat in broth, lard, beans, soup....

  • Gelatin - This is in a lot of things, mostly as a thickener, though it's used for some other things too. Look for it in marshmallows, jelly beans, Jell-O, any rubbery "candy" (like Starbursts), See's dark chocolate, and other stuff.
  • Hydrogenated oil & partially hydrogenated oil - This one is in a lot of food in the U.S. In many other countries, it has been banned. Look for it in margarine, peanut butter, candy, bread, junk food, and pretty much everything else.
  • Almost all bottled drinks and sugar or corn syrup drinks - This includes Coke, Pepsi, Sprite, etc.
  • Red bell peppers - I'm allergic, but only to the red ones
  • Rennet - This is sheep gut that is used to make a lot of cheese. I try to avoid it, when I can, but it's hard to know if it's been used.
  • Jägermeister - There are rules.
  • Fast food

Foods I Do Eat

  • Anything not listed above (I think)

  • Eggs - I'm pro choice ← That's a joke, in case you missed it
  • Milk and milk products - I'm lactose intolerant though, so I minimize it
  • Beer - it's not a soda.
  • Gatorade
  • Juice
  • In-N-Out fast food - It's generally a lot better than other places

Foods that Confuse Me

  • Escargot

Dear Mr. Gore

Honorable Al Gore
2100 West End Avenue, Suite 260
Nashville, TN 37203

29 December 2008

Mr. Gore:

I applaud your efforts towards educating the world on the subject of anthropogenic climate change, and I congratulate you on your Nobel Prize.

I write today to urge you to adopt a more open copyright on your materials. From what I can deduce, you are relying on the standard American copyright for your movie, presentations and books. As a lawyer, I am sure you are aware that this copyright system gives you incredible strength to protect the fruit of your efforts, however for many people, this system hampers their access to reading your books, watching your movie, or enjoying your presentation.

You and the IPCC have been recognized as the greatest laborers towards educating the world on this subject, and I understand if you have taken the traditional route with your work. Recently however, as I began purchasing the third copy of your movie to give away to a friend, I couldn’t help but think about how great it would be if it were simply streamed from your site.

There are many options for creating a more open copyright system, and I hope you will take the time to consider the implications of your current method.

Sincerely,

Michael Lissner

Yelp Scraper to Get Business Info in a Geographic Area

I spent the past couple days on one of my first Python projects - using the Yelp API to compile a list of restaurants in a defined geographic area.

It's been a good project. Because of some limitations of the API, I had to do some interesting tricks to make it work. One problem with the API is that it only allows 20 hits per query, so if you want to do a big query, you have to divide it up into tiny queries that have fewer than 20 hits each.

To accomplish that, if a query gets 20 hits within those two points, it will divide the longer dimension of the rectangle created by the points in half, and perform a query on each of those two new rectangles. For each of those, if there are 20 hits, it will again divide it in two and perform two new queries, and so forth until less than 20 hits are found for the rectangle. Once less than 20 hits are found, the data is entered into a database. Once all the points have been added to the database, a comma separated file is created, and the program ends.

It was pretty incredible switching to Python for this project from my usual Java, and also using an official API for the first time. This project ended up being about 200 lines (half of which are comments). I can't imagine how long it would be with Java, since I used some rather powerful Python modules to accomplish this (namely, csv, urllib & json).

If anybody is interested in seeing/using the code, let me know. It should be useful if you need a list of restaurants or other businesses in a certain area. Worthy causes only please!

Cheney is so f***ing shady, it pisses me off.

Cheney is at work to keep his records from becoming public as part of the National Archives when he goes out of office.

"The vice president alone may determine what constitutes vice presidential records or personal records, how his records will be created, maintained, managed and disposed, and are all actions that are committed to his discretion by law."

I can think of no reason why this would be necessary unless he's the shadiest person ever in office. Oh, right.

As an ex-government worker, I can testify to how ridiculous this is: It's really f***ing ridiculous.

Dear God, This is a Terrible Interface

The UI for a KDE Python IDE is about the worst I have ever seen:

That's about 90 buttons.

Rsnapshot Backup Solution OR Why Backing Up Is Hard To Do

I've been working on getting this post figured out for about five months. In this post, I am going to try to explain exactly how my backup works, and why. It's ridiculously complicated at times, but the detail is necessary on paper in some form or other.

For my backup system, I rely heavily on rsnapshot, which is a tool that uses rsync and some perl scripting to create snapshots of directories. Rsync is a pretty awesome tool. It functions like a simple copy/paste, except that it will check the destination directory of the paste and will only copy the necessary files. As such, it can be interrupted in the middle of a copy, and will be able to continue later where it left off. Perl is a scripting language that has been used with rsync to give it some extra power.

This power is the ability to perform incremental backups, which is to say that if I have 5GB of data that I backup 10 days in a row, it will only take up about 5GB of data, total. However, if I have 5GB of data today, an additional 5GB tomorrow, and another 5GB the day after, which I backup each day for ten days, it will only require a total of 5GB of space the first day, 10GB the second day, 15GB the third day, and no more space after that for the remaining 7 days.

This is important if you want to backup your data on a regular basis. Since I run a server, I have several things that I must backup. I back these up on a daily, weekly, and monthly basis. The list includes:

  • My laptop is backed up wirelessly to the server's hard drive
  • The email server gets copied to an external USB drive (this includes all the Zimbra configuration files as well as thousands of emails)
  • The web server gets backed up to the USB drive (this includes the Drupal installation and the MySQL database)
  • Lots of configuration files for the servers go to the USB drive (i.e. the /etc/ directory)
  • And finally, the backup configuration itself goes to the USB drive

Each of these backups presents some challenging difficulties. For the web server, it is challenging because it is backing up MySQL, Zimbra and Drupal. In order to do this, I have to coordinate the MySQL database dump so that when the Drupal backup is triggered, it will copy the MySQL information over to the USB drive along with the normal Drupal information. For Zimbra, the email server has to be stopped, backed up, and then started again, which means control of the email server has to be carefully scripted.

The laptop presents a challenge because it is the only thing that is backed up wirelessly, and in order to do so, the server must authenticate itself to the laptop before it is allowed to log in and make the copies. If that wasn't complicated enough, in addition, the laptop needs to be set up with a static IP address so that the server can find it to perform the backup. Finally, the laptop needs to be ON, and connected to the network when the server attempts the backup.

Once all of that is figured out logically, you have to authenticate the laptop to the server, create the scripts, backup configurations and cron files. I have attached some of these configuration files to this post, provided they don't reveal too much of my network topology.

One final challenge that had to be overcome was connecting the USB drive to the server in such a way that it would always be mounted in the same location. In addition, I learned that FAT32 doesn't support file system links, and so I had to format the USB drive as ext3.

As of today, it's about five months since I began this project, and I believe I can say that the backup happens flawlessly on a daily, weekly and monthly basis. There are a few things I'd like to change however:

  • I'd like to get an email notification when a backup fails
  • I'd like to begin backing up /etc/ on my laptop
  • At one point, I was backing up a list of all the installed software on my system - it'd be nice to have that again
  • The backup is unencrypted, so anybody can take the USB drive and have a heck of a lot of emails.

Files of Interest

  1. rsnapshot configuration for my laptop
  2. rsnapshot configuration for the Drupal server
  3. rsnapshot configuration for the backup configuration files and the /etc directory
  4. rsnapshot configuration for the Zimbra server
  5. scripts to stop/start Zimbra
  6. My cron file

All in all, this just goes to show that backing up is a very difficult thing to do properly and automatically. It's one thing if you have a desktop that backs up to a USB drive. It's another if you have a server and a laptop. Had I known how long this would take going into it, I'm not sure I would have figured it all out...

We Knew It Existed - The Official Definition of The Internet

On October 24, 1995, the Federal Networking Council met and decided that the "Internet" refers to the global information system that:

  1. is logically linked together by a globally unique address space based on the Internet Protocol (IP) or its subsequent extensions/follow-ons;
  2. is able to support communications using the Transmission Control Protocol/Internet Protocol (TCP/IP) suite or its subsequent extensions/follow-ons, and/or other IP-compatible protocols; and
  3. provides, uses or makes accessible, either publicly or privately, high level services layered on the communications and related infrastructure described herein.

Oh. Thanks for that.

To Be Silent is To Lie

C. William Steele
Director
National Eagle Scout Association
1325 West Walnut Hill Lane
P.O. Box 152079
Irving, TX 75015-2079

5 December 2008

Sometimes to be Silent is to Lie
– Miguel de Unamuno

Mr. Steele:

When I was four years old, I was a Tiger Scout. Later, I became a Cub Scout, and then a Webelos. When I was old enough, I became a Boy Scout. Over the next few years, I advanced in rank, became an Eagle Scout and achieved many Eagle Palms. I had over 100 merit badges, was the Junior Assistant Scoutmaster, went on the National Jamboree in Arlington, Virginia, and the International Jamboree in Chile.

I was what could be considered one hell of a Boy Scout. Today, with much bitterness and sadness, I give up the only remaining symbol of those parts of my life. Today, I return to you my Eagle Scout Badge, and in so doing, renounce and reject the prejudices and discriminative practices of the Boy Scouts of America.

When I was about 16, I learned that agnostics, atheists and gays would not be tolerated by the policies of the Boy Scouts. While it's true that as of 2000, the supreme court backs your (well-funded) position as a private organization to discriminate institutionally, that does not make it right. Each of these positions flies in the face of scouting elsewhere in the world, and each of them reinforces an environment in which people with different opinions are awarded different value. While it is true that Baden Powell said that “[no] man is much good unless he believes in God and obeys His laws,” we sometimes must reflect on the beliefs of our founders, and put them in the context of their day. When Baden Powell made this remark, women had not yet gained suffrage, Nazi Europe had yet to come and go, and apartheid had yet to be institutionalized, let alone dismantled. In short, it was a different time.

We must ask ourselves what are we doing today that our children will disdain. Today we live in a world where religious differences can be enlightening and can lead to greater understanding of ourselves and our peers. Yet in this world, religion can be used publicly as an accusation (Obama as Muslim), and defenses to such accusations need not defend the religion, only the man. We live in a world today where religious differences lead to war, terrorism, and deaths of thousands of people. We live in a world where an organization designed to create leaders out of boys discriminates based on religion and sexual preference. If we are not careful, our children and their children will not look kindly upon our time as one of freedom of opinion and religion, but rather as one of oppression, war, and discrimination.

Your organization has at its heart some of the greatest lessons I have learned during my lifetime, and it saddens me greatly to be forced to abandon it. I plan to have children one day, and it saddens me greatly that they will not have the Boy or Girl Scouts of America in their lives. Most of all though, it saddens me that a national organization would choose not to include a diversity of people and opinions.

Enclosed is my Eagle Scout badge, one of my more treasured possessions. I hope some day to get it back from you, and I hope someday that you will realize the error of your ways.

Sincerely,

Michael Lissner
Eagle Scout, '98