We had some conversations recently with customers about JackRabbit solutions that suggested what we thought as our small configuration was in fact too large. This was an eye-opener. This group has particular needs well suited to the design, but they have to keep the system costs down, and are willing to trade some aspects of design for cost. So we worked on it, and came out with a unit that should be able to provide about 8 TB for about 9k$ US.

You have a 4 node cluster. You want to share data among the nodes. Pretend it is a “desktop” machine. Fine. Setup NFS. Or Samba/CIFS. Share the data. End of story. But this doesn’t work as well when you get to 40 nodes, and starts failing badly at 400. At 4000 nodes, well, you need a special filesystem design. What happened? Why when we scale up do problems arise? Well, you have several factors.

Won’t grab historical data, just stuff going forward …

My day job is at a company in the Metro Detroit area. Metro Detroit, and actually Michigan in general is a very nice state. This is a good place. There are good people here. Prices are reasonable, cost of living isn’t terrible, and for the moment, taxes are under control. The problem is that the area is completely dependent upon the fortunes of the US auto manufacturers. Since manufacturing continues its effort to seek out and use the lowest cost systems, Metro Detroit and Michigan will continue to hemorrhage jobs in this area.

Intel announced some details on Penryn and others today. It looks like a sweet chip. The problem I am having is, if the Clovertown is memory bus bound with 4 cores (2 x 2-core chips) for a number of memory intensive workloads, won’t 8+ cores be worse? Think of this in terms of public expenditure and return on investment. If something you are investing more money in isn’t giving you the return you want/need, doesn’t it make sense to stop throwing more money at it?

REDWOOD CITY, Calif.-(Business Wire)-March 27, 2007 - PeakStream, Inc., a leading software application platform provider for the high performance computing (HPC) market, today announced its innovative PeakStream Platform(TM) is now available in beta version for Microsoft Windows.PeakStream Workstation(TM) for Microsoft Windows(R) Edition allows software developers to easily program new high performance processors such as multi-core CPUs and graphics processor units (GPUs) directly on their desktops. Now, programmers working with Windows can enjoy the same advantages that their Linux-based counterparts have been benefiting from since the PeakStream Platform’s initial launch last September: the ability to develop technical and scientific applications faster, and run them at higher performance, using their existing tools and programming languages.

Now that I (think) understand most of the major issues here, and I can be reasonably sure that I have a good grasp of the tuning, I think I want to take it out on the test track and give it one final once over. Lets open the throttle. Wide. I can tune the IO scheduler, number of outstanding IO requests (for sorting), various buffer cache, and the works. I now have the clock left alone (need to set it that way by default), so it is running full speed.

I had an interesting epiphany over the last few days. We normally turn on powernow to let idle machines … idle … during low load times. This way they consume less power. Of course, the road to penultimate benchmark results are paved with such good intentions. I noticed that when run this way, several CPU/memory/IO benchmarks didn’t always hit the throttle on the CPU. It remained clocked lower. Which meant I was getting very odd buffer cache timing, that I could not quite grok.

So I have a “new” phone. Long story, not worth going into. It runs Windows CE. My Palm Treo 650 was frustrating (PalmOS is inconsistent, and largely broken, missing important things … and it crashed … occasionally wiping out the email program and all settings). If anyone from Palm is reading this, please understand that I have every intention of avoiding your future products. For a very good reason. This phone looked better than mine from a feature perspective.

Amusing… Of course there is more than a little grain of truth to it, even though it is marketing. And there is a second “ad” here, and a third. Update: Ok, this is something I don’t quite get. I have done some searching for market share data for Linux. Why not, it is of some interest to know what customers want and are interested in using. Almost all the “data” I have seen puts Linux penetration into the noise.