intel's hyperthreading: super smart marketing of a super weak feature

Here's my beef:  Intel's asks consumers to pay 50% more for their hyperthreaded chips, but those hyperthreaded chips will only ever, in the best case scenarios, be 10-15% faster than their non-hyperthreaded cousins.  Few people understand exactly what this means - so let's discuss.

Consider Intel’s latest Ivy Bridge series.  The i5-3570k is a quad-core clocked at 3.4 GHz and does not support hyperthreading.  The i7-3770 is a quad-core clocked at 3.4 GHz and supports hyperthreading.  When you bring up the Windows Task Manager with the i5-3570k, you will see four cores.  When you bring up the Windows Task Manager with the i7-3770, you will see eight cores (four logical cores plus four hyperthreaded cores).  As a result, you, the consumer, might be led to believe the i7 is twice as fast as the i5 – it has twice as many cores, right?!  Sorry, not even close.  Here’s the real story.

If you understand parallel computing, you know that a perfectly parallelized algorithm will continue to improve its performance based the number of available cores - its called scaling.  The computation is simple.  If an perfectly parallelizable algorithm takes 840 s to compute using a single core, it will take 420 s using two cores (twice as fast, or 2x), and 210 s using four cores (four times as fast, or 4x).  But what happens if we perform this scalability test on an i7-3770?  The numbers (assuming the best case that hyperthreading performs at 15% the speed of a logical core):

i7-3770 Cores employed	Time (s)
1	840
2	420 (2x)
3	280 (3x)
4	210 (4x)
5	202 (4.15x)
6	195 (4.3x)
7	189 (4.45x)
8	182 (4.6x)

Did you see what happened after four cores?  The scaling tanked.  Even using all of the i7-3770’s eight cores is only 10% faster than using four cores – and that’s assuming the best case scenario that each hyperthreaded core is 15% as fast as a logical core.  In a system with eight real, logical cores, the performance should have scaled to 105 s (eight times as fast as one core, or 8x).

That's the ideal world – but as it turns out, few real world algorithms will ever get close to seeing the 10-15% hyperthreaded improvement hypothesized.  While purely scientific applications will sometimes approach this ideal, the real world is riddled with constraints:  in particular, threads/processes share memory, disk, and oftentimes data-structure/resources within the parallel application itself.  Shared resources prevent ideal scaling, as concurrently executing threads/processes are forced to wait for one another to acquire/use these shared resources.  This means there is just a fraction of room left for hyperthreading to make its little voice heard above the loud, real world din of shared resources.

Here’s a real world algorithm that I deal with every day:  compiling my C/C++ source code.  Visual Studio (and other third party build systems) support multithreaded/multiprocess builds.  This means that the compilation phase is done in parallel.  When compiling, each thread/process is reading a source file from disk and writing an object file to disk – concurrently.  What’s the shared resource?  The disk.  What can happen to disks?  Fragmentation.  What exacerbates fragmentation?  Lots of threads/processes writing to a disk concurrently.  Which is exactly what is happening!  As much as I love the fact that multithreaded/multiprocess builds speed up my compilation phase, fragmentation causes the “win” to diminish over time – and eventually, the subsequent link phase, which is particularly disk intensive and single-threaded, becomes a disappointing drag.  What happens with hyperthreading?  Well, even if hyperthreading improves the compilation performance by 10% (a best case given the disk is shared), disk fragmentation is increasing at twice its normal rate - and whatever win was witnessed during compilation is quickly lost over time – and can sometimes lead to reverse scaling - when using more cores actually slows things down.  (And don't even get me started on how Windows manages its NTFS partitions and how performance can degrade over time, despite the fact they've led consumers to believe that "defragging" is all they have to do... this cake is a lie.)

I have to give credit to the Intel marketing department.  They spun the hyperthreading message and have made tons of money as a result.  Even human intuition is on their side – when I see eight cores in Task Manager I’m wayyyyy faster than four cores, right?!  Here’s what’s crazy – even though there are tons of independent benchmarks to back-up my statements (that i7's are only fractionally faster than i5's) – people still dole out the big bucks for the i7-3770.  Please stop.  Please use your common sense.  Invest your money sensibly.  Buy an i5-3570k and use the savings to buy a bad-ass solid-state-disk (ssd).  I wager you’ll never notice the difference between an i5-3570k and an i7-3770 – but I guarantee your life will be shockingly improved when you upgrade from a mechanical disk to a killer ssd.

using size_t pragmatically

size_t is awesome - but I rarely see it employed correctly. You may think I'm here to lay down some dogmatic "type correct" philosophy, but rather, I'd like to instead give a pragmatic treatment for using size_t. First, though, some background.

Background

Predominantly, size_t is used to represent the size of objects. For example, malloc is prototyped as:

void * malloc( size_t );

As such, size_t needs to be scalable. On 32-bit platforms, size_t is an unsigned int. On 64-bit platforms, size_t is an unsigned 64-bit int.

The next most predominant usage of size_t is the "length" of strings and containers. For example, strlen is prototyped as:

size_t strlen( const char * );

Also, many STL containers (e.g., vector, queue, etc.) have methods for returning their "length". These "length" and "size" methods (when boiled down primitives) also return size_t.

The pragmatic view of size_t

It's pretty simple:  know your application's needs.

For example, if your app has a ton of classes/structs for storing fixed length strings (I'm talking char[]) that are programmatically limited to 256 characters or less, storing their length with size_t is overkill - especially on 64-bit systems. In this case, you can get away with using an unsigned char for storing your string's lengths. (You just saved seven bytes:  sizeof(size_t) - sizeof(unsigned char). Multiplied, this savings adds up quick.)

In contrast, if your application places no restriction on the length of strings stored - you'll definitely want to use size_t for storing string lengths.

If you're rolling your own container objects (e.g., vector, dynamic array, hash table), my preference is to use size_t for storing your element size and your element count. As an author of a "generic container", it is difficult to predict its usage - and size_t provides the maximum headroom. That said, if you're watching your memory usage closely - and you can reliably predict your application's usage of your container - you can choose a smaller built-in for storing your element size and/or element count. You would do this to save memory, of course - and depending on the container's usage, you could see a big savings.

My pet peeve for size_t is when I see it casted away needlessly. For example:

int length = (int)strlen( psz );

for ( int i = 0; i < length; i++ )

   ...

The above is just lazy coding. Why downcast the length of psz? (Probably: "to quiet the compiler 'downcast' warning".) On 64-bit systems, you're adding down-cast instructions  In all cases, stack is cheap. In these cases, I always use size_t:

size_t length = strlen( psz );

for ( size_t i = 0; i < length; i++ )

   ...

The above is cleaner, faster, and in my opinion, smarter.

A couple final items of note:

• size_t is completely portable. Use it with confidence between Windows and Unix. As mentioned, on 32-bit systems, it is an unsigned int (4 bytes); on 64-bit systems, it is an unsigned 64-bit int (8 bytes).
• size_t is not a built-in type. When name mangled, it will be reduced to the built-in types mentioned.  Therefore, when you dumpbin/nm your library you will see your usages of size_t replaced by unsigned int (32-bit systems) and unsigned 64-bit int (64-bit systems).

In a future post, I'd like to get into portable usage of integral built-in's like int, long, long long, 64-bit ints, and pointers. But if you have questions on size_t - let's have 'em.

why software projects are like progress meters

Why is it so hard to make a decent progress meter?  How many times have you seen a progress meter do something ridiculous like:

• Get stuck at 71% for a long time.
• Get stuck at 100% for a long time.
• Get to 100%, then start over at 0% and go to 100% again.
• Repeat the prior bullet five times.
• See a time estimate of 324518 hours, then two seconds later, see an estimate of two minutes.

I'm being rhetorical when I ask "why can't we make better progress meters?"  I expect answers like, "how can I account for a network blip?" or "what should I do if the user suddenly loads the machine?" or "the work units being measured aren't equal complexity".

Funnily enough, executing a software project is quite similar to watching a progress meter.  Low effort planning, inexperience, failing to account for interruptions, and other combinations of factors can cause your software project to go totally awry of the estimate, get stuck at 71%, and in some cases, spawn off entirely new projects.

It all starts with the estimation process. A good estimate is the prescription for a smoothly run project.  Sadly, though, I've heard more than once, "the software process can't be accurately estimated".  This matter-of-fact resignation is bogus.  Authoring quality estimates isn't easy - but it allows your development and project managers to more accurately plan upcoming work.  So let's get to it:  my suggestions.

My first suggestion:  for those of you who write software estimates - try harder.  Study the existing code intensively, you may find logic that (nearly) meets the need.  Also, consider writing experimental logic "outside the app" to test your approach.  You won't be able to author a decent estimate until you've "cut a mental path" through the challenging parts of the project.  Once your estimate is done and the work begins, there should only be small mysteries remaining.

Second, know your "load" - your meeting schedule, how much time you spend on email, whether you'll be taking vacation, etc.  This is an Agile concept - knowing how many hours per day/week you can work on your projects.

The third suggestion may be obvious: let experience be your guide. However, you may be asked to work in unfamiliar code.  In these cases, spend a couple hours studying the code - then review your thoughts with one or more of your team members.  When no one is available to consult, you may need to more than a few days studying the code in order to provide a reasonable estimate.

Finally, make every effort to break your estimate into sub-one-week tasks. Admittedly, I struggle with this.  I argue with myself that I can call on my experience - that "I've done something similar before".  Invariably, though, my three week estimate either takes a week or five weeks.  In other words, I goofed my estimate because I didn't break it down.

If you put some effort into it, you should be able to author estimates that allow you to execute your software project with reasonable predictability - which is a heck of a lot more than I can say about most progress meters.  :)

interviewing ftw

The interview process - what a mess.

Sometimes, superstar candidates get looked over.  Sometimes, less than stellar candidates get hired.  I've pondered this at length... it's frustrating.

As an interviewee, you're at the will of an unpredictable, inconsistent, filtering process.  As such, I suggest the following conventional wisdoms:

• Your resume and cover letter should present your best efforts and qualities.  Sell your "wins".
• Enumerate your skills, e.g., C++, SQL Server, Perl, etc.  It might seem trivial, especially if you think of yourself of more than just a set of skills, but if you don't, you'll be at a competitive disadvantage.
• During the entire interview process - from the first contact, the first phone call, and the first interview - speak calmly and purposefully.  Answer questions directly, don't speak wastefully, and communicate as eloquently as possible.
• Whether asked or not, when you're posed with a "thought" question, speak through it.  Reveal yourself, your manner of thinking, and above all, be honest - in particular, when you don't know an answer, say so.
• Know something about the position and company for which you're interviewing.  Think of poignant questions to ask your interviewer(s).

The frustrating part is this:  even if you're at 100%, your interviewers might not be.  They might be facing deadlines of their own, interviewing other candidates that day, or just having an "off" day.  You simply cannot control their perspective on that particular day, regardless of how prepared you are.

For interviewees, that's the reality of it:  despite your best efforts and best qualities, your interviewers may fail to expose how awesome you are.  If your interviewers decide to pass on hiring you, that's their loss, unless in fact, you're not awesome.  But that's another issue altogether.

So let's assume you are awesome.  Now we need to examine the process from the interviewer's standpoint.  Because the failure is the interviewer's if they choose not to hire you.

As an interviewer, you're faced with a huge responsibility.  Being one of three to five interviewers (typically), you share the responsibility of whether to spend tens to hundreds of thousands of dollars on a new hire.  Will they be worth more than their salary?  Will they add to the social glue of the team?  Will they motivate their fellow team members, junior and senior, to rise to new levels?  Holy crap this is a big deal.

Here's the weird thing about hiring:  typically, senior staff will conduct interviews.  Being senior, they'll have biases.  Their opinions on hiring will be based on 1) whether the candidate shares similar personal/educational growth, and 2) the statistical pattern bias of their hiring history.  For example, if the interviewer has spent ten+ years becoming a master at recursion, they will take a negative bias of the candidate if they don't firmly grasp recursion - regardless of how much recursion is used in their products.

If you're an interviewer, here's my advice:  erase your biases.  Think of every candidate of being great until you've proven otherwise.  Your job is to discover the candidate's qualities - not eliminate them because they don't have the same technical "chops" as you.  This becomes the interviewer's challenge...  so how do you do it?

I have an approach to interviewing.  Of course, it is biased to my personal viewpoints and past interview experience.  But I strive desperately to avoiding succumbing to my biases.  I honestly believe that, as an interviewer, I am burdened with the challenge of discovering the candidate's greatness in 45 minutes or less.  So let's get to the meat of it - my approach to interviewing:

• At some point during the interview, I like to take a negative stance to something the candidate expresses a belief in.  I'm trying to evoke a response - because work isn't all positive.
• All good workers care deeply about the quality of their work.  So I might ask them to explain, comprehensively, how they ensured the quality of a particular project they implemented.
• As I interview, I listen to every word choice - and every sentence.  Communication skills are paramount.  This is a potential teammate - they need to be articulate.
• Also, as I listen, I am measuring the candidate's passion.  I am seeking natural, unforced passion.  The best workers are naturally motivated.
• Finally, I want to measure the candidate's talents for the position.  Being technically savvy is crucial - and I usually pose a handful of technical questions to measure their technical aptitude.

That's just a flattened perspective on conducting a decent interview.  The fact is, I cannot write a recipe on how to conduct the perfect interview.  I will say that, as an interviewer, I am always prepared:  I've read and researched the candidate and their past few employers, I've written my questions on paper, and I am mentally prepared to make the most of the few minutes I'll have with the candidate.

Regardless of what side you're on, I suggest putting forth a strong effort.  Distinguish yourself, whether interviewee or interviewer - your (potential) employer will recognize the effort.