Quite early in the process, but we see probes have dropped off. Some of this may be due to the IP level (draconian) restrictions. The outline of the new security measures are as follows: The user has to have a valid VPN certificate to ssh to the system. Users cannot share a certificate. This isn’t simply policy, it is enforced on a technological level. outgoing traffic is pretty much restricted to VPN, and set specific ports.

This is over a single 10 GbE link, using a pair of Intel cards with the ixgbe driver, and a CX-4 cable. The ΔV is configured in one of our two basic modes, in this case, a RAID10 unit. It is exporting an about 3.5TB partition over iSCSI to the Windows 2008 x64 Server box. This is an Intel dual Woodcrest 2.66 GHz box with 4 GB RAM. I wanted to see what the bandwidth limits are.

I thought people might like to see this: root@dV3:~# dd if=/dev/zero of=/big/big.file ... 2048+0 records in 2048+0 records out 34359738368 bytes (34 GB) copied, 97.0368 s, 354 MB/s root@dV3:~# dd if=/big/big.file of=/dev/null ... 2048+0 records in 2048+0 records out 34359738368 bytes (34 GB) copied, 57.4585 s, 598 MB/s and some bonnie++ Version 1.03b ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP dV3 16G 360487 53 162147 32 542351 45 610.

A petaflop here, a petaflop there, and soon we are talking about real performance

Should have some exciting news around the day job and a partnership we are working on. Will be in time for the SC08 show. Also, we should have some product announced soon. Real soon. Very exciting stuff for us … We have already sold one (as of yesterday, we have the PO), and I am hoping to have one or more sold by early next week. Not “game changing” from a technological perspective as Henry Newman posited on a blog post recently, but from other perspectives, it very well could be.

We have customers interested in using these drives. So we are looking into getting some. I am seeing some awful reports on the fora about buggy firmware. The response from Seagate doesn’t appear to be good. First they say they don’t support Linux, then they say they don’t support RAID with these (despite small local RAIDs being on their list of best fit applications). Now they claim it is a Linux problem, and the folks on the kernel list … um … disagree.

NFS over RDMA between a pair of JackRabbit systems with a Mellanox Connect-X card and a direct cable. We are getting about 500 MB/s +/- some on reads, and about 300 MB/s +/- a bit on writes. Not great, but actually in line with what is observed here. See the NFS over RDMA plots near the middle. This is about 1/3 of the available disk bandwidth on reads, and a little less than a 1/3 on writes.

On this blog, we have done some testing of Connect-X using OFED 1.3.1. This is on Ubuntu 8.04, and not on RedHat. Yes, we have a mostly functional OFED 1.3.1 we distribute on JackRabbit (and soon ΔV) I wanted to see what sort of performance we could get out of a single port DDR connection between two units. On the server, I ran iperf -s and on the client I ran iperf -c 10.

The Connect-X cards arrived this morning, so we are late in shipping the units to the customer. Ugh, but manageable. Most everything is ready, about to launch octobonnie on it for the burn. Octobonnie is an example of an extreme stress test. We stress [rent a car bulgariaJackRabbit](http://jackrabbit.scalableinformatics.com) hard during basic testing. As a result, we learned quite a bit about how this system comes up and performs. Startup is fine, though PCI-e busses and bioses what they are, every now and then PCI-e x8 cards negotiate at x1.

For some of the lower level tools we are developing for ΔV (and for JackRabbit), we need to parse device strings. That look like this “/dev/sda1”. We have been doing this for years with the Perl Getopt::Long module. Well, something interesting happened today. I need to dig into it, but it looks like a (bad) change to some of the standard Perl libraries installed on this machine (Ubuntu based with our kernel).