There’s a new Humble Bundle available from http://www.humblebundle.com/.
You get Crayon Physics Deluxe, Cogs, VVVVVV, And Yet It Moves and Hammerfight all for a price you decide. You also get to donate to charity too 
Go and get it
I’ve just had a hard drive that I tried to join to a ZFS pool. The drive had been used before when testing ZFS on Linux. However, when trying to add the drive it would complain about not finding the first partition:
cannot resolve path '/dev/disk/by-id/ata-ST3160215ACE_9RX5F4K9-part1'
fdisk showed the drive as being empty but having a GPT table.
fdisk -l /dev/disk/by-id/ata-ST3160812A_5LSBGY5S
WARNING: GPT (GUID Partition Table) detected on ‘/dev/disk/by-id/ata-ST3160812A_5LSBGY5S’! The util fdisk doesn’t support GPT. Use GNU Parted.
Fdisk can’t do GPT so I fired up parted.
root@vhost:/home/administrator# parted /dev/disk/by-id/ata-ST3160812A_5LSBGY5S
GNU Parted 2.3
Using /dev/sdg
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Warning: /dev/sdg contains GPT signatures, indicating that it has a GPT table. However, it does not have a valid fake msdos partition table, as it should. Perhaps
it was corrupted -- possibly by a program that doesn't understand GPT partition tables. Or perhaps you deleted the GPT table, and are now using an msdos partition
table. Is this a GPT partition table?
Yes/No? Yes
Model: ATA ST3160812A (scsi)
Disk /dev/sdg: 160GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 1049kB 160GB 160GB zfs
9 160GB 160GB 8389kB
It reported that the GPT signature was damaged and appeared to fix it for me as I could now see two partitions on the drive. I removed them and set the partition table to a simple msdos one:
(parted) rm 1
(parted) rm 9
(parted) mktable msdos
Warning: The existing disk label on /dev/sdg will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? yes
Warning: /dev/sdg contains GPT signatures, indicating that it has a GPT table. However, it does not have a valid fake msdos partition table, as it should. Perhaps
it was corrupted -- possibly by a program that doesn't understand GPT partition tables. Or perhaps you deleted the GPT table, and are now using an msdos partition
table. Is this a GPT partition table?
Yes/No? yes
Now let’s see if it’s all gone….
(parted) p
Model: ATA ST3160812A (scsi)
Disk /dev/sdg: 160GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
(parted) q
Information: You may need to update /etc/fstab.
After reboot I tried to create the pool again but was met with the same error I had at the start!
I then tried the following:
After each of these the GPT partitions kept coming back!
Plugging the drive into Windows didn’t help as the Disk Management utility wouldn’t let me change the GPT table either. I then remembered the Windows command line tool diskpart but had forgotten which commands I needed:
C:\Documents and Settings\Me>diskpart
Microsoft DiskPart version 5.1.3565
Copyright (C) 1999-2003 Microsoft Corporation.
On computer: PC1
DISKPART> help
Microsoft DiskPart version 5.1.3565
ADD – Add a mirror to a simple volume.
ACTIVE – Marks the current basic partition as an active boot partition.
ASSIGN – Assign a drive letter or mount point to the selected volume.
BREAK – Break a mirror set.
CLEAN – Clear the configuration information, or all information, off th
disk.
CONVERT – Converts between different disk formats.
CREATE – Create a volume or partition.
DELETE – Delete an object.
DETAIL – Provide details about an object.
EXIT – Exit DiskPart
EXTEND – Extend a volume.
HELP – Prints a list of commands.
IMPORT – Imports a disk group.
LIST – Prints out a list of objects.
INACTIVE – Marks the current basic partition as an inactive partition.
ONLINE – Online a disk that is currently marked as offline.
REM – Does nothing. Used to comment scripts.
REMOVE – Remove a drive letter or mount point assignment.
REPAIR – Repair a RAID-5 volume.
RESCAN – Rescan the computer looking for disks and volumes.
RETAIN – Place a retainer partition under a simple volume.
SELECT – Move the focus to an object.
DISKPART> list disk
Disk ### Status Size Free Dyn Gpt
——– ———- ——- ——- — —
Disk 0 Online 233 GB 0 B
Disk 1 Online 149 GB 0 B
Disk 6 Online 149 GB 0 B
DISKPART> select disk 6
Disk 6 is now the selected disk.
DISKPART> clean
DiskPart succeeded in cleaning the disk.
DISKPART> exit
Leaving DiskPart…
I then removed the drive, put it back into my Linux computer and successfully added the drive to the ZFS pool. YAY!
I’m surprised that it took my Windows machine to do something that seems so simple. It’s odd that Seagate’s own utilities couldn’t wipe the disk either and the fairly comprehensive Linux utilities dd, parted and fdisk, which normally do so well, couldn’t wipe this disk.
As with everything to do with computers it shows that there are multiple tools for the same job and some work better in some situations that others.
I’ve just re-installed and re-configured my home fileserver as the old install was in need of a refresh. Several ‘quick hacks fixes’ meant I wasn’t happy with the setup and knew I could do it better. I’ve also learnt a lot since setting it up two years ago and wanted to do everything properly this time round.
I made the change from ZFS-FUSE to ZFS-on-Linux partly just to see how the different approach to ZFS compares and so I could use ZVOL volumes with iSCSI for virtual guests.
After the reinstall I noticed that my server had a high system load when I was copying data back to it. High as in 15+! The CPU and RAM usage was low and showed little activity. iostat showed that data reads and writes were quite low.
Some searching showed me that I needed to use sar to check my drives. If it’s not already on your system you need to install the sysstat package. I enabled and started sar and waited for some data to be collected (it’s not instant). After a few hours I used the ‘sar -d’ command to show me some disk stats:
root@vhost:/var/log/sysstat# sar -d
Linux 2.6.38-8-server (vhost) 28/06/11 _x86_64_ (2 CPU)
17:36:00 LINUX RESTART
Average: DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
Average: dev8-16 0.65 0.06 6.63 10.30 0.01 13.34 12.56 0.82
Average: dev8-0 38.71 2.98 4812.72 124.41 0.35 8.94 3.62 13.99
Average: dev8-32 0.65 0.03 6.63 10.30 0.01 15.55 14.32 0.93
Average: dev8-48 40.45 2.96 4813.06 119.07 0.56 13.77 3.99 16.12
Average: dev8-64 13.56 27.49 1069.82 80.93 2.87 211.46 68.78 93.25
Average: dev9-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: dev9-1 0.85 0.09 6.24 7.48 0.00 0.00 0.00 0.00
Average: dev251-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Average: dev251-1 0.04 0.09 0.29 10.00 0.00 35.00 15.11 0.06
Average: dev251-2 0.07 0.00 0.52 8.00 0.00 21.99 10.00 0.07
Average: dev251-3 0.68 0.00 5.43 8.00 0.01 17.26 11.11 0.75
The bit I’m interested in was the averages section at the bottom, specifically the %util column. This shows that dev8-64 is, on average, in use 93% of the time. Because it’s nearly at full capacity it might explain why I had high system loads. This device is causing a queue to form.
I needed to find out which exact device dev8-64 was. All I had to do was look for the corresponding numbers in /proc/diskstats:
root@vhost:/# cat /proc/diskstats
1 0 ram0 0 0 0 0 0 0 0 0 0 0 0
1 1 ram1 0 0 0 0 0 0 0 0 0 0 0
1 2 ram2 0 0 0 0 0 0 0 0 0 0 0
1 3 ram3 0 0 0 0 0 0 0 0 0 0 0
1 4 ram4 0 0 0 0 0 0 0 0 0 0 0
1 5 ram5 0 0 0 0 0 0 0 0 0 0 0
1 6 ram6 0 0 0 0 0 0 0 0 0 0 0
1 7 ram7 0 0 0 0 0 0 0 0 0 0 0
1 8 ram8 0 0 0 0 0 0 0 0 0 0 0
1 9 ram9 0 0 0 0 0 0 0 0 0 0 0
1 10 ram10 0 0 0 0 0 0 0 0 0 0 0
1 11 ram11 0 0 0 0 0 0 0 0 0 0 0
1 12 ram12 0 0 0 0 0 0 0 0 0 0 0
1 13 ram13 0 0 0 0 0 0 0 0 0 0 0
1 14 ram14 0 0 0 0 0 0 0 0 0 0 0
1 15 ram15 0 0 0 0 0 0 0 0 0 0 0
7 0 loop0 0 0 0 0 0 0 0 0 0 0 0
7 1 loop1 0 0 0 0 0 0 0 0 0 0 0
7 2 loop2 0 0 0 0 0 0 0 0 0 0 0
7 3 loop3 0 0 0 0 0 0 0 0 0 0 0
7 4 loop4 0 0 0 0 0 0 0 0 0 0 0
7 5 loop5 0 0 0 0 0 0 0 0 0 0 0
7 6 loop6 0 0 0 0 0 0 0 0 0 0 0
7 7 loop7 0 0 0 0 0 0 0 0 0 0 0
8 16 sdb 3003 2538 182874 89990 18182 14195 213086 306750 0 267500 396730
8 17 sdb1 130 521 2720 6140 1980 4 78 54410 0 59560 60550
8 18 sdb2 2820 1851 178402 80630 16200 14191 213008 252280 0 215220 332900
8 0 sda 26815 31523 13645033 224160 562106 155798 80700699 5615990 0 2966510 5839980
8 1 sda1 26620 28832 13640509 223670 562100 155798 80700698 5615960 0 2966150 5839460
8 9 sda9 19 0 152 20 0 0 0 0 0 20 20
8 32 sdc 2981 3058 167774 89820 18040 14337 213086 341810 0 299090 431630
8 33 sdc1 122 443 2526 6200 1979 5 78 110070 0 115240 116270
8 34 sdc2 2807 2448 163496 80480 16059 14332 213008 231680 0 193050 312160
8 48 sdd 48443 9400 13680566 866870 584715 131571 80700698 8202480 0 3230750 9068950
8 49 sdd1 48308 7215 13676769 866280 584715 131571 80700698 8202480 0 3230490 9068360
8 57 sdd9 65 226 1768 410 0 0 0 0 0 410 410
8 64 sde 925 451 115592 1557230 144973 5169 16824587 16876100 1 6962060 18433460
8 65 sde1 48 93 1368 520 86392 466 2819424 7021170 1 5682130 7022050
8 66 sde2 753 155 111608 1556630 58580 4703 14005155 9854920 0 2981230 11411320
9 0 md0 797 0 2158 0 23 0 54 0 0 0 0
9 1 md1 9556 0 338854 0 27322 0 202968 0 0 0 0
251 0 dm-0 263 0 2104 6690 0 0 0 0 0 3680 6690
251 1 dm-1 6901 0 267002 290730 2081 0 16272 140080 0 55190 430810
251 2 dm-2 342 0 2706 9710 1732 0 13816 22690 0 16970 32400
251 3 dm-3 1815 0 65162 35500 21636 0 172880 434700 0 216130 470190
As you can see dev8-64 is actually /dev/sde. In my case this was a 2GB Compact Flash card I was using as a cache device for my ZFS filesystem. I’d used it to help improve reliability on my old heavily deduped filesystem without buying new RAM but since my new setup used deduplication less I could safely remove it now. Proper SSD drives would be better as would using more devices and mirroring them. They all involve extra spending though which is what I’m trying to avoid as it’s only a home server built for convenience.
So you’ve bought your HD telly. It cost quite a lot of money.
Then you pay for a costly subscription service so you can receive HD content.
The problem is that after spending most of your day either sleeping or working you are only getting a few measly hours in which you can bask in the glow of HD.
Don’t you wish that there was a way to have HD vision ALL of the time? Well now you can:
I’ve never realised how limited my life has been with standard definition eyesight!
If only Hitler played about with Minecraft instead of writing Mein Kampf; He might have gone on to get the German population to build crazy structures from huge cube shaped blocks instead of pushing them to flatten Europe.
I was aware of Minecraft but I’d never looked at it or realised what it was about. Somebody I know pointed me towards a PC Gamer demo of the game, which I had up and running within a few minutes.
Confronted by nothing but a vast blocky landscape I was unsure of what to do. The square sun was shining brightly through the clouds, the trees looked err.. treeish and the beach in the distance was calling me to explore. Some angular sheep wandered near by but they ran off as I approached them. The sun moving through the sky reminded me I needed to find somewhere safe before nightfall came and the monsters came out….
The first thing you’ll discover about Minecraft is that everything in the game can be dug up, destroyed or moved. This leads to some novel approaches to problems you might face in the game. For example, It annoyed me to find that every morning there were monsters always lurking around by house. My solution was to build a moat by digging a trench around my house and filling it with water. No more monsters, but now the pigs like to bathe in it all the time.
The other main concept of the game is that of “crafting”. This is the ability to create new objects by mixing together the materials you find. I was doing it by trial and error but you can find recipes on the internet to make more advanced things. Some things can only be collected with the correct tools. I needed to make sure I had a stone pickaxe to be able to mine iron ore, my wimpy wooden pickaxe just wasn’t strong enough.
Getting materials often requires lengthy trips underground. The deeper you go the more likely you might find some useful ore or rare artefact. You’ll need to stock up on torches as it is dark down there and you’ll meet many monsters hiding in the corners. If you go too deep you might break through the mantle itself and reach a layer of lava!
The beauty of Minecraft is its simplicity. Get stuff, use stuff, place stuff, don’t die. Once you’ve figured them out you can make the game into an adventure of your choosing. It doesn’t take long before your brain starts coming up with crazy ideas you know you’re going to try. For example, I kept getting lost and it would take forever to find my way home, often being chased by monsters. I decided to build the tallest tower I could, this would then be visible from far away providing me with an easy way home. You can craft yourself a compass but that’s the easy way out. My huge engineering project filled me with pride once successfully completed.
My tower, visible in the distance.
I’ve since gone on to build bigger things and, by setting up my Minecraft server, have enlisted the help of my girlfriend to help with these projects (although I’ve had to switch the monsters off as she wouldn’t play with these lurking in the mines). If setting up your own server isn’t your cup of tea then there are plenty of public ones available. Be warned if you do visit though… they will make your lovingly crafted projects look pitiful in comparison!
The game is currently in a beta stage with new things being added over time. I’ve not had any problems with it as yet on my ageing system but you do need plenty of memory as it eats it by the bucket load. Since it uses Java it should run on any system with Java installed. I happily use it on my Windows XP and Ubuntu machines with no trouble.
Gotta go…. My ‘log flume down a waterfall starting at the top of a mountain’ idea isn’t going to build itself…
I’ve used ZFS in some form or other for a few years now, starting off with the OpenSolaris based Nexenta and moving over to Ubuntu Server as ZFS matured on Linux.
Recently I’d added a couple of new drives to the pool and all was good until after a reboot. After logging in to the server I could see the pool hadn’t mounted so I attempted it manually.
It was telling me the new drives I’d added were corrupt and the pool was offline. “No Problem” I though as I could just restore from backup and re-create the pool. The only problem with that is that my last backup is 6 months old*
I then noticed that the disk assignment was wrong. My ZFS array was looking for /dev/sdb and /dev/sdc when the new disks were now at /dev/sdf and /dev/sdg.
I then remembered using /dev/disk/by-id when I initially created the pool to avoid this issue but when adding the new drives I never gave it a thought.
To fix this without losing any data I did the following:
1) Rename your zpool.cache file
mv /var/lib/zfs/zpool.cache /var/lib/zfs/zpool.cache_old
2) Now import your pool again
zpool import -d /dev/disk/by-id/ poolname
It should now look through all of your disks and mount the pool as normal with each of the disks added by id.
Taadaa! Pool mounted and a full backup has now started.
*Yes, I know. How stupid of me… 
There’s a new Humble Bundle available from here:
For those of you who’ve not heard of the Humble Bundle before it’s a collection of games for sale. The major difference between this and any other online purchase is that YOU get to set the price you want to pay. If you feel like paying the full retail price (£35 – £40) you can, or you can pay less or you can pay more.
“Pay more! But who’d do that?”
The money you pay is, by default, split between the developers of the games and two charities:
You can change the ratios if you want so all of your money can go to the charity or all of it to the developers.
The games are DRM free, come with extra content such as soundtracks and can be downloaded and used on your Linux, Mac or Windows machines.
The Humble Bundle has also proven that Linux users would and will by games for the OS with donations from Linux users typically being more than double than those from Windows users.
You can’t lose. You get some great games, support some indie developers and get to feel all warm and fuzzy because you’ve given to charity too