Personally, I don’t have any kind of battle-hardened wisdom. Most of what I know comes from the wisdom of others. In particular I would like to acknowledge the real geniuses up front:
My First Petabyte: Now What? (Jacob Farmer from Cambridge Computer)
Google Platforms Rant (Steve Yegge of Google)
http://perspectives.mvdirona.com/ (James Hamilton of Amazon)
Ted Dzuiba
Tons of blogs posts, LISA talks, that I can’t site specifically.
Seriously, we are all a product of the RSS feeds we read?
I believe that a graphing tool should be flexible enough for the user to decide what metrics are important to their environment, and it should be able to accept metric data from any source.
There are pretty good graphing tools out there for the Linux engineer: Munin, Ganglia, Cacti, Zabbix, Collectd, etc.
I call these “graphing” tools, because I don’t consider them “monitoring” tools, like Nagios, WhatsUp, or Zabbix. Here is are my thoughts on graphing/monitoring:
Every job has metawork. Sometimes it becomes so commonplace that we just forget about it. We should never forget about it. I hope this blog post reminds you of what your metawork is. For the purposes of this blog post it will focus on code-centric stuff, but could apply to anything.
Different jobs have different types of metawork. Sometimes it is just paperwork. Other times it is called “overhead”. Commonly it is referred to as “Process”.
Update: I wrote this article when I was young and stupid. It is obsolete. Puppet wins, just use Augeas. Both Puppet and CFEngine are formidable configuration management systems. Let’s compare the two to see how the match up on the common task of editing files. Feel free to jump straight to the conclusion if you are impatient.
Why would you need this? What is the use-case here? These are some common tasks I do that involve editing files:
Ethernet Link Aggregation (aka PortChannel, Etherchannel, ethernet bonding, NIC teaming, trunking, link bundling, Smartgroup, Ethertrunk, etc) is a way to combine multiple Ethernet links to a single logical link. This improves redundancy and increases aggregate performance. It is good stuff, especially if you are into High Availability.
Here is a typical setup: A good start. If you can spare the ports and cables to make this work, you should do it.
Below are a few philosophical guidelines that I use when thinking about systems. Everyone has systems that they are not proud of, even me. But I hope to someday say that I am proud of all of my systems, and I believe that these philosophies will get me there.
1. Try to endeavor to build systems with as few dependencies as possible. Sometimes systems can sprawl and get out of hand.
DRBD is the bomb. Think Raid 1 over the network between two servers. It enables sysadmins (in combination with heartbeat) to increase service uptime, while enabling server downtime. This means you can do kernel updates, do hardware maintenance, upgrade ram, move the server to another rack, etc, all while keeping the service up.
Because DRBD is often used in high availability scenario, there isn’t much room for experimentation with in-production systems.
I apologize for the verbose post. This is a complicated problem and it merits full explanation. If you are experiencing this problem I advise you to avoid skimming and read it very carefully :)
Sometimes you come across a problem that is a little more complex than average. Sometimes it is problem that is rare enough that there isn’t an obvious answer in the first google hint. Sometimes a problem may seem unavoidable, which leads to work-arounds, and before you know it you have a mess.
So you have some new harddrives. Maybe you are thinking about building a DIY storage server? Maybe some Raid? Maybe you just want to know if your drives are performing as well as when you bought them? How can you know? Measurement is Knowing.
The thing about benchmarks is that you always must be skeptical. Each system’s particular disks, controller, raid level and settings, cpu, ram, filesystem, operating system, etc can GREATLY affect the performance of a system.
Starting with the new year, I decided to move my xkyle.com domain and related stuff away from my account at Dreamhost, a shared hosting server, to my virtual machine at Tummy.com. I have nothing but good things to say about Dreamhost though, they are excellent at what they do. However, I wanted to do more than what a shared hosting provider could do for me, I needed Root!
Moving my websites and dns to a dedicated server grants me the ability to implement a few technologies to usher in the new year.