Joped.com

For the longest time I have been trying to figure out how to force GCC to use multiple cores when doing compiles. Today, I finally came across the -j flag for make. This basically tells make to run x number of jobs at a time. Warning, don’t specify -j without a parameter or it will spawn an infinite number of jobs which is very bad.

Make command	Compile time
make	8min 1sec
make -j 2	4min 42sec
make -j 4	3min 9sec
make -j 6	2min 46sec
make -j 8	2min 37sec
make -j 10	2min 43sec
make -j 12	2min 38sec

Release notes for Linkped 2.16

• Introducing new shorter URL  and new algorithm for generated links. All links generated will now use lkp.cc instead of linkped.com.  Old links will direct to the new location, so don’t worry they won’t break.
• SSL for login, registering and change password has been restored.
• Fixed issues with the view count on links not always updating properly.

Alpha testing of the new send link feature has been going well, but its still not quite ready for beta.  Although the interface is pretty much the same, all of the backend code was completely rewritten.  Presently, there is no ETA on when this will be available.

I have been noticing an interesting pattern for a while now, and figured it was about time to help set the story straight on memcache.  If you don’t know what memcache is, I recommend you reading up on it first.

Memcache is a dead simple cache and it works very well when you remember that it is a simple secondary level to your data.  Developers tend to forget this, which leads to some very interesting consequences.

What memcache is.

I can not stress enough that memcache is a very simple cache and works best when you treat it just like that and nothing more.  Its ultra fast because it does that 1 simple thing very well.  Adding even minor things like delete in x number of seconds adds overhead that defeats the purpose of it.

Don’t get me wrong, without some more complex features memcache can be pretty difficult at times.  Things like pagination and key organization are some of the biggest problems.

What memcache is NOT!

• Its NOT a database - Don’t treat it as your primary database.  MySQL is designed to protect and organize your data.

• Its NOT a queue system - Think about what you are trying to do.  You are putting important flags into a system that is not guaranteed to have your data.  Can you deal with your queue missing performing work a few times ?

• Its NOT intended for session handling - Similar to the reason above would you want users to randomly get booted out ?  Don’t get me wrong, you want something fast .. I understand.  But, there are alternatives such as MCache.  Keep in mind, I have not actually used it.  It has an ultra annoying build system.  Before you think of using memcache for sessions realize what the impact on your users is.  Wouldn’t you be annoyed if you were checking your E-Mail on Gmail and got booted randomly ?

• Its NOT a proxy - If you are serving files or cached dynamic HTML you should strongly consider using varnish or mod_proxy.  If you are experiencing issues with io load on your server and you want to serve files very fast you could also consider using TempFS.  Remember to populate your data on boot time

• If you need replication you might be using memcache wrong. If you are looking for replication for memcache step back and think why do I really want this ? There is a pretty good chance you don’t have a fall back plan.  The same holds true for wanting to create a backup of your memcahe to restore in the event of a node failure or reboot.  There is a never a good reason to do this.

• Its NOT a locking daemon - Don’t get me wrong, there aren’t many alternatives but this is just not an option.  Never forget, memcache is never guaranteed. But with that in mind, you could use it as a locking daemon but only if you can deal with duplicate runs.

Best practices

• Use connection pools, don’t get crafty with your connections unless you FULLY understand what you are doing.  Running multiple instances and separate connection handlers is not the proper way to handle things.  Don’t use 127.0.0.1 when you are running multiple instances.  Use the external IP and specify your entire pool on EACH webserver.

$memcache = new memcache;
$memcache->addServer(’192.168.1.1′,11211);
$memcache->addServer(’192.168.1.2′,11211);
$memcache->addServer(’192.168.1.3′,11211);

• Always have a fall back. If the data isn’t found in memcache, go to your datastore.  That doesn’t mean that you need to exit execution of your site if the datastore isn’t reachable.  One of my sites can run with 85% of the features when MySQL is unreachable.  Basically, anything that requires a write will not function and is disabled.  I use warm up scripts that pre-populate memcache during each build.  Its a simple script that uses my existing code to populate, severally lowering the maintenance requirements.  I set some randomness to the TTLs so they don’t all expire at the same time.  This would cause sudden spikes to your database.

• Use low cache times until stable. Until you are properly cache busting in your application keep your cache times very low.  This will allow you to pinpoint problems, get a little bit of performance and not piss off your visitors too much.

• Create a wrapper or extend your memcache library, especially in PHP. This will allow you to create stastics during development and troubleshoot key management.  Not to mention, create an “internal cache” that prevents duplicate over the wire requests.

• Cache negative search results, lets say you have user profile pages and you ban a user.  There might be a high amount of traffic looking for that record that doesn’t exist in your database (well, if you actually delete the data :D).  Prevent those look ups from hitting your database and you will be happier for it.

• Don’t md5() your keys! This really drives me nuts. Why on earth would you want to do this?!?  There are rare times when your key name could contain things that violate the memcache protocol.  However, to my knowledge pretty much every library will strip out violation characters.  If they don’t strip these out, it could lead to command injection.  Doing things like this make memcached verbose (-vv) pretty worthless.  Not to mention, a lot of calls to md5() just chew up CPU cycles.  Sure, with modern processors its very quick but this is a huge problem these days.  No one wants to create tight and solid code, they just want it done yesterday.

• Go easy on calling memcache for stats. If you are running Cacti on your memcache farm go easy on how much you poll.  Don’t hit it every minute as after you have your memcache setup working well you won’t watch those graphs all the time.  Your graphs will reach a point where they don’t really move too much…  Unless for some reason you are doing frequent memcached restarts.

• Use increment / decrement for things like number of views on a page.  After you run an update query, why bother running another select query right after that?  Depending on your storage engine, this could cause lock contention.  Instead, run the update and use memcache::increment(). This will help performance greatly since you are just publishing your data, not trying to process it.

• Don’t use a shotgun to take down a spider. Lets say you have a box on your home page that shows the last 10 posts.  Don’t set the cache key for 10 minutes, set it for 2+ hours and create a cron job that performs the select and updates memcache.  This will also reduce lock contention, you don’t want all those vistors fighting to update the same key. Remember that during development, you will want to keep that expiration low and increase it to hours when you consider it stable.

• Use memcache during development. This is another one that baffles me.  It drives me crazy when developers won’t develop with memcache enabled.  You need to really test this. Test your CRUD (Create Read Update Delete) and few times over and for multiple users. If you are not using your production strategies during development, you are essentially writing code that is untested (and in some ways could even be considered a fork of your app for lack of functionality)

• Running memcache verbose (-vv) during development helps a lot. You can quickly see when variables aren’t being set properly.  Imagine seeing a key like user_ scroll by, you can spot problems quick.

I have a few follow up articles that I have planned that will touch on more advanced usage of memcache.  One of these updates will include the memcache wrapper that I am using for my projects.

These are frame by frame screen shots of the 1080p HD trailer for Terminator: Salvation. I know they aren’t in order of the trailer but organized them so you can see the connections to them. I took them the same night the trailer was released but I have not had time to post them.  The trailer shows off some cool aspects of the movie.  Warning though, these could be spoilers if you don’t know the Terminator series that well

Click the arrow keys to move around through the gallery and you can click the picture to resize it to full.  Once in full screen, you can click it again to have it resize to a thumbnail.  I am trying out a new picture gallery plugin because I don’t have time to do anything complex.

I forgot to publish this before, the YouTube video is really shitty quality … the version I uploaded was *FAR* nicer then this.  This landing was done in X-Plane 8 right before I upgraded to X-Plane 9.

Linkped 2.15.5 has been released:

• New buddy list and email address list out of beta and released to general public, thanks beta testers!  I am going to be working on colors for the graphs, I know they are a bit hard to read right now.
• New redirect frame, similar to the old one but cleaned up the CSS and images.  You might notice that the up and down arrows match the new ones in “My sent links”.
• The top frame shows the number of views a link has, this is just the first phase of a big feature in development. (secret for now)
• When you view a link, it will that remove your alerts. (Thanks PJ for the suggestion)
• The “Show per page” location has been moved above the pagination, and only appears when you hover over.  Helps keep the interface clean.
• Moved the RSS icon

Known issue

• Favorites are only showing up for page 1

I have not fully tweaked out my video settings so this isn’t the best. I missed the landing spot a tiny bit and forgot the hook.

X-Plane 9 - Carrier landing fail, no hook from Joseph Engo on Vimeo.

I was playing around with multiple monitor support in X-Plane 8 on my Mac Pro and I have a few suggestions if you are having trouble getting this to work.

• Inside System preferences -> Displays -> Arrangement make sure your menu bar is on your left most monitor.
• All monitors need to have the same height resolution set.  If you are using monitors that support different max resolutions, you should change all of them to match the lowest monitor resolution.  When setting things up initially, you should set the width all the same as well, then tweak from there.
• Add up all monitor widths and set your X-Plane resolution to that width.  For example, during my testing I was using a 20″ Samsung, very very old Samsung 15″ and a Samsung 61″ HD DLP set.  The max the 15″ could accept was 1024, so I set X-Plane to 3072×768.  At that low of a height, the TV had large black bars around it … but it works for my needs.

When money gets better, I want to get another Samsung 20″ monitor … they aren’t too expensive and work fairly well.  Then I will use my laptop for the other screen.

Totally unrelated to the above, but here was me flying a jet around tonight chasing this airliner

I played a few games of Urban Terror tonight, I needed to get some practice in because Friday I am having a match with some co-workers.

In celebration of 2009, I will be releasing 2.15 of Linkped which includes the new my sent links and new home page.  This is a feature that has been in beta for a while now, and new years day seems like an excellent time to release the new feature to the masses.

This post was created the day before, so the new version won’t actually be released until later in the day.