Memcache is a dead simple cache and it works very well when you remember that it is a simple secondary level to your data.  Developers tend to forget this, which leads to some very interesting consequences.

What memcache is.

I can not stress enough that memcache is a very simple cache and works best when you treat it just like that and nothing more.  Its ultra fast because it does that 1 simple thing very well.  Adding even minor things like delete in x number of seconds adds overhead that defeats the purpose of it.

Don’t get me wrong, without some more complex features memcache can be pretty difficult at times.  Things like pagination and key organization are some of the biggest problems.

What memcache is NOT!

• Its NOT a database – Don’t treat it as your primary database.  MySQL is designed to protect and organize your data.

• Its NOT a queue system – Think about what you are trying to do.  You are putting important flags into a system that is not guaranteed to have your data.  Can you deal with your queue missing performing work a few times ?

• Its NOT intended for session handling – Similar to the reason above would you want users to randomly get booted out ?  Don’t get me wrong, you want something fast .. I understand.  But, there are alternatives such as MCache.  Keep in mind, I have not actually used it.  It has an ultra annoying build system.  Before you think of using memcache for sessions realize what the impact on your users is.  Wouldn’t you be annoyed if you were checking your E-Mail on Gmail and got booted randomly ?

• Its NOT a proxy – If you are serving files or cached dynamic HTML you should strongly consider using varnish or mod_proxy.  If you are experiencing issues with io load on your server and you want to serve files very fast you could also consider using TempFS.  Remember to populate your data on boot time

• If you need replication you might be using memcache wrong. If you are looking for replication for memcache step back and think why do I really want this ? There is a pretty good chance you don’t have a fall back plan.  The same holds true for wanting to create a backup of your memcahe to restore in the event of a node failure or reboot.  There is a never a good reason to do this.

• Its NOT a locking daemon – Don’t get me wrong, there aren’t many alternatives but this is just not an option.  Never forget, memcache is never guaranteed. But with that in mind, you could use it as a locking daemon but only if you can deal with duplicate runs.

Best practices

• Use connection pools, don’t get crafty with your connections unless you FULLY understand what you are doing.  Running multiple instances and separate connection handlers is not the proper way to handle things.  Don’t use 127.0.0.1 when you are running multiple instances.  Use the external IP and specify your entire pool on EACH webserver.

$memcache = new memcache;
$memcache->addServer(’192.168.1.1′,11211);
$memcache->addServer(’192.168.1.2′,11211);
$memcache->addServer(’192.168.1.3′,11211);

• Always have a fall back. If the data isn’t found in memcache, go to your datastore.  That doesn’t mean that you need to exit execution of your site if the datastore isn’t reachable.  One of my sites can run with 85% of the features when MySQL is unreachable.  Basically, anything that requires a write will not function and is disabled.  I use warm up scripts that pre-populate memcache during each build.  Its a simple script that uses my existing code to populate, severally lowering the maintenance requirements.  I set some randomness to the TTLs so they don’t all expire at the same time.  This would cause sudden spikes to your database.

• Use low cache times until stable. Until you are properly cache busting in your application keep your cache times very low.  This will allow you to pinpoint problems, get a little bit of performance and not piss off your visitors too much.

• Create a wrapper or extend your memcache library, especially in PHP. This will allow you to create stastics during development and troubleshoot key management.  Not to mention, create an “internal cache” that prevents duplicate over the wire requests.

• Cache negative search results, lets say you have user profile pages and you ban a user.  There might be a high amount of traffic looking for that record that doesn’t exist in your database (well, if you actually delete the data ).  Prevent those look ups from hitting your database and you will be happier for it.

• Don’t md5() your keys! This really drives me nuts. Why on earth would you want to do this?!?  There are rare times when your key name could contain things that violate the memcache protocol.  However, to my knowledge pretty much every library will strip out violation characters.  If they don’t strip these out, it could lead to command injection.  Doing things like this make memcached verbose (-vv) pretty worthless.  Not to mention, a lot of calls to md5() just chew up CPU cycles.  Sure, with modern processors its very quick but this is a huge problem these days.  No one wants to create tight and solid code, they just want it done yesterday.

• Go easy on calling memcache for stats. If you are running Cacti on your memcache farm go easy on how much you poll.  Don’t hit it every minute as after you have your memcache setup working well you won’t watch those graphs all the time.  Your graphs will reach a point where they don’t really move too much…  Unless for some reason you are doing frequent memcached restarts.

• Use increment / decrement for things like number of views on a page.  After you run an update query, why bother running another select query right after that?  Depending on your storage engine, this could cause lock contention.  Instead, run the update and use memcache::increment(). This will help performance greatly since you are just publishing your data, not trying to process it.

• Don’t use a shotgun to take down a spider. Lets say you have a box on your home page that shows the last 10 posts.  Don’t set the cache key for 10 minutes, set it for 2+ hours and create a cron job that performs the select and updates memcache.  This will also reduce lock contention, you don’t want all those vistors fighting to update the same key. Remember that during development, you will want to keep that expiration low and increase it to hours when you consider it stable.

• Use memcache during development. This is another one that baffles me.  It drives me crazy when developers won’t develop with memcache enabled.  You need to really test this. Test your CRUD (Create Read Update Delete) and few times over and for multiple users. If you are not using your production strategies during development, you are essentially writing code that is untested (and in some ways could even be considered a fork of your app for lack of functionality)

• Running memcache verbose (-vv) during development helps a lot. You can quickly see when variables aren’t being set properly.  Imagine seeing a key like user_ scroll by, you can spot problems quick.

I have a few follow up articles that I have planned that will touch on more advanced usage of memcache.  One of these updates will include the memcache wrapper that I am using for my projects.

What really drives me nuts is finding a site that accepts OpenID, I then authenticate with my OpenID provider, only to be bounced to a registration form.  What the hell ?  My OpenID profile isn’t empty, it contains some basic information about me.  Zip code, timezone, Firstname, Lastname and email address.  Why would you go and ask me for the same information again ?  This information has already been passed back to you with the OpenID protocol!  At very least, can you please pre-populate this information ?

When you return to an OpenID site and change your timezone in your OpenID profile, why doesn’t the site you are logging into update your information with the new data sent back from your OpenID provider ?!  I know this can be a tricky thing to do, I am working on it for Linkped.  I had this support at one point, but had to remove it when I refactored the OpenID login code. My first run with OpenID was a very lame way to support it. Don’t worry, support for updates will be returning very soon.

Many websites hide their OpenID login screens and for the life of me I can’t figure out why!  Hell, I use it as a selling point on Linkped.  With 83% of the user base using OpenID who can blame me   The OpenID logo is a slick design that fits into almost any website.  Don’t make me hunt for your OpenID login screen, put the logo right on your registration form to let your vistors know they can sign up even faster by using OpenID.

OpenID is a great thing, and I am happy to see it getting adopted everywhere.  I know that one day spammers will become a problem and site operators will revert to using a captcha after login, but I am ok with that.  However, unless more developers stop to think about how difficult and ass half backwards their OpenID implementation is, OpenID isn’t going anywhere.

If you are developing a site that uses OpenID, take some time out of your day and go use some sites that have OpenID.  See how bad and annoying some of the implementations are, learn from their mistakes.

Needless to say, I need to sleep in dirty cloths tonight and need to wear them tomorrow as well.  With some lucky maybe my bag will appear tomorrow morning.  They have a limited number of flights here per day, and I was on the last one of the day.  Great.

What pisses me off more then anything is that I have no clean socks.  After wondering around my hotel for a while, I was able to purchase some dress socks.  Guess what, that is good enough for me. (For now)

I asked what the airline expected me to do about brushing my teeth and what not.  They responded with this small bag of stuff to get me through the night.  I am happy they provided it for me, but not happy I needed to beg for it.

So far Circus Circus has been an annoying hotel.  It took far too long for me to find my room.  I love how they have maps all over the place, and none of them indicating “You are here”.

I was told they have internet access in all the rooms, however I can’t find a jack and no wifi signal.  Good thing I brought my EVDO card.  The connection speed is far faster then it was last year, so I am just going to stick using it.

I am off to find something to drink and maybe a light snack.  I have a bit of a headache that I need to shake.

Well, the CC they had on file was wrong. Rather then just re-checkout, I had to go through the entire process again.

Needless to say, I went back to iTunes and purchased the tracked I wanted in a fraction of the time.  Screw Amazon and their tax evading ways.

Now, in order for Time Machine to work your only options either direct drive connected or AFP to another Leopard client or server.

Although connection to another Leopard box via AFP isn’t bad, its doesn’t work for me. I own 2 Macbook Pros and a Linux server. I had originally planned on using the Linux box as my server for Time Machine. Since I already had AFP running, it would be easy to drop in a new volume for it.

The reasons for Apple doing this is quite unknown and really drives me nuts. Now, I am forced to do one of the following work arounds. None of them are very appealing, and none I have gotten to work yet.

1. Find work around for backing up over SMB or AFP. Although I would love this one to happen, I have yet to see any progress. AFP isn’t being actively developed, so there is an even less chance then SMB. The strange thing is, you can’t even mount a Tiger AFP mount which doesn’t make any sense. I wonder what they changed in the protocol.
2. Find a work around for Airport Extreme Base stations or a fix from Apple. There is a small work around for getting Timemachine to backup to it. However, the Time Machine interface doesn’t work with it and its unstable. I could put together some simple scripts to make sure my mount point is valid at all times I am connected to my LAN.
3. Install Leopard in VMWare. (I am not interested in talking about the EULA. I purchased a Family Pack and I disagree that Apple has the right to limit what machine you can install it on. Its like aftermarket parts) I have been working on trying to figure this one out. I can’t get the image to boot, been trying for a week now. I have posted on a few message boards, this is feeling like a dead end. To top things off, my Linux server is SSE2.
4. Similar to #3, I was thinking that I could purchasing an ultra cheap bare bones system to run it. It doesn’t have to be powerful, just needs to handle Time Machine and serve iTunes to my AppleTV.
5. The AppleTV is another cheap system I might be able to install it to. I don’t think it has SSE3, so it might be a serious problem. I have heard of getting Tiger to run on it.
6. Find very cheap G4 on craigslist, I haven’t been able to find much under $300. I am trying to do it for no more then $250. I haven’t had much luck with eBay either. Not to mention, I don’t know much about older Macs.

There is also the big issue of I can restore from Time Machine after these work arounds are put in place. Once I find a working idea, I am going to test it. My plan is to install Leopard to an external drive and backup to the network mount. From that point, I am going to delete my test partition. Will all my test data be there ?

I will try and post my progress.

With HD, comes along many new terms. To be honest with you, up until a few weeks ago. I had no idea what the difference between 1080p and 720p. DLP vs Plasma vs LCD. After weeks of reading, I now understand many of the ins and outs of HD. I have looked at sets ranging from CRT, to Plasma, to projection. I have also been doing research down to the component level of sets using DLP technology.

With all that being said, what is the difference between a $30 HDMI cable and a $120 HDMI cable. Go around an ask some people on the street. Compare that response to what the stores will tell you, what the packaging will tell you. Then, do some research on the truth.

The truth is, there is no difference. I can’t explain it better then some of the following resources that will really help people wake up to whats going on.

Here is a repost from someone named RUSirius who really explains the truth of the matter, you can read the orginal post here.

1) Wires send electrical signals… Plain and simple. Anything sent over a wire is ultimately just a voltage/current applied to that cable. Let’s say we’re talking about an analog video signal that’s 1 volt peak to peak… In other words, measuring from the LOWEST voltage to the HIGHEST voltage will give a result of 1 volt… With an analog signal you have “slices” of time that are “lines” of signal… It’s too complex to go into here, but basically you have a “front porch” which is known as the “setup”… This is what helps your tv “lock onto” and sets the “black level” for the signal. After that you’ve got each line of the image (455 half cycles per line). Again I won’t go into how chromanance (color information) and luminance (picture or brightness information) is combined, seperated, etc.. It’s too complex for this discussion, but irregardless, just know that following that porch you’ve got all the lines of the picture (and some that don’t show up on the picture… these carry closed captioning, test signals, etc…). All of these “lines” of information when you look at them on a scope look like this…

That waveform is all of that information in analog form… In other words, if you look at one VERY SMALL timeslice of that waveform, the EXACT position of the form (i.e. what voltage is present) represents what information is at that position…

Because of this, it’s VERY EASY for other radiated signals to get “mixed in” with that information. When this happens, the more “noise” you get mixed into the signal, the more degraded the picture will be… You’ll start to get snow, lines, weird colors, etc… Because “information” is getting into the waveform that doesn’t belong there…

With digital however, (i.e. the signal sent over an HDMI cable), the information is encoded differently… At it’s lowest level, it’s nothing but a string of bits… In other words, each signal is either ON or OFF… It doesn’t care if a particular timeslice is 4.323 volts or 4.927 volts… It’s just ON… See on the right side here, the “square wave” pattern?

That’s what a digital signal looks like… For each “slice” of the signal, the “bit” is either on (if the signal is high) or off (if it’s low)…

Because of that, even if you mix some noise, or even a LOT of noise into the signal, the bit will STILL be on or off… It doesn’t matter…

Now, for a slightly easier to understand analogy…

B) Think of it this way… Let’s say you have a ladder with 200 steps on it… An “analog” signal represent information by WHICH step the person is on at a certain time. As you move further and further away (get “noise or interference in the signal), it’s very easy to start making mistakes… For example, if the person is on the 101st step, you might say he’s on 102nd, or as you get further away, you might start making more and more mistakes… At some point you won’t know if the person is on the 13th step or the 50th step….

NOW… In a digital signal, we don’t care if he’s on the 13th or 14th or 15th step… All we care about is rather he’s at the TOP or the BOTTOM… So now, as we back you up further and further (introduce more noise), you might have no idea what STEP he’s on, but you’ll STILL be able to tell if he’s a “1″ or a “0″…

THIS is why digital signals aren’t affected by cheaper cables, etc… Now eventually if you keep moving further and further back, there may come a point where you can no longer tell if he’s up or down… But the good news is, digital signals don’t “guess”… If they SEE the signal, they work… If they DON’T, they DON’T.. LOL

So if anyone ever tells you they can “see the difference” between HDMI cables, etc… You can knowingly laugh to yourself and think about how much money the poor sole wasted on something that was pointless.

Now, I’ve seen others say that they make a difference in audio… ALL audio carried over HDMI is STILL in digital format… So again, since it’s a digital signal, it will not make ANY difference at all….

I’ve also seen various posts in regards to things like “Make sure you get a v1.3 cable”… The various HDMI versions determine the capabilities of the DEVICES on either end of that cable (most of the HDMI versions (other then 1.0 to 1.1) have to do with AUDIO and how many channels / type of audio are carried…) Because of this, the cable itself is NO DIFFERENT… It’s just marketing that some companies charge more for a “v1.3″ cable then a “v1.1″ cable, etc… The cables themselves will work now and WELL into the future for any other HDMI versions that come along the way….

Sure, the above can be confusing to most people. Do some googling on the issue and you will be suprised how much people are getting ripped off.

Before you buy something, make sure you do some research. Going to the store and trusting the sales persons pitch doesn’t count.