Computers

One of the main bottlenecks when running high performance virtualization systems is the harddisk. Now, you could of course switch your entire system over to SSDs, but that is costly and you’ll end up with a lot less storage, or a massive RAID array.

Modern filesystems like ZFS have solved this problem by allowing for ‘hybrid’ systems. These use the traditional harddisks for persistent storage, and use SSD drives in front of them to cache the read and write queries. This way you get the best of both worlds. Nearly SSD performance and the storage size of a traditional drive.

At Flosoft.biz we use Proxmox to power our VPS offers, which uses LVM and EXT4 for it’s filesystem which doesn’t have a ‘SSD caching’ method built into it. Facebook seems to have had a similar issue, so they created FlashCache. Flashcache is a kernel module that allows you to add a block caching partition in front of any other partition of your system, resulting in an amazing speedup of your system.

After having spent a night or two on getting this to work on Proxmox 2, I decided to write a small tutorial here. I’d also like to thank @toxicnaan for his l33t hax0r skillz.

 

Updating your system

Get your system up to date and make sure you’ve got the latest Kernel.

apt-get update
apt-get dist-upgrade
apt-get install dkms build-essential git
reboot

 

Kernel Headers

You will now need to install the Kernel Headers for your Kernel so that you can compile the module. Make sure you install the correct version of the headers. These need to be the same as the Kernel you’re running.

uname -a # to get your kernel version
apt-get install pve-headers-2.6.32-17-pve # to install the headers for version 2.6.32-17

 

Get FlashCache

Now that we’ve got the Kernel tools, we can get FlashCache and build it.

git clone git://github.com/facebook/flashcache.git
cd flashcache/

make -f Makefile.dkms boot_conf
make install

 

Load FlashCache

Next we need to load FlashCache into our running Kernel and make sure it’s loaded upon boot.

modprobe flashcache
echo flashcache >> /etc/modules

 

Re-purposing the SSD drives

Now it’s time to find a new use for our SSD drives, namely as cache. You can skip this step if your server doesn’t have the SSD drives mounted as /var/lib/vz

umount /var/lib/vz
vgremove pve
pvremove /dev/md2

 

Re-purposing the 2 HDD drives

Now let’s prepare the 2 HDD drives to be used as the storage for /var/lib/vz.

umount /data
pvcreate /dev/md0
lvcreate -l 100%VG -n storage pve
mkfs.ext4 /dev/mapper/pve-storage

 

Creating the FlashCache partition

Now let’s create the FlashCache partition on the SSD drives & mount it.

flashcache_create -p back pvec-storage /dev/md2 /dev/mapper/pve-storage
mount /dev/mapper/pvec-storage /var/lib/vz
echo 1 > /proc/sys/dev/flashcache/md2+pve-storage/fast_remove

 

Editing /etc/fstab

Next step is to edit /etc/fstab and remove the /data and /var/lib/vz mounts. If you forget to do this (as I did for quite a while), your server will struggle to boot on it’s own, and you’ll end up with the datacenter techs thinking you’re an idiot 🙂

vi /etc/fstab

 

The init.d file

This next step is important. We need to add an init.d file to do some operations, like mounting the filesystem and cleaning it up. It will also unmount the drive before shutting down, as if you don’t do this, your kernel will freeze on shutdown. Make sure you edit your file according to your needs.

#!/bin/sh

# Start or stop Flashcache

### BEGIN INIT INFO
# Provides:          flashcache
# Required-Start:
# Required-Stop:     $remote_fs $network pvedaemon
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Flashcache SSD caching
# Description:       Flashcache SSD caching
### END INIT INFO

PATH=/bin:/usr/bin:/sbin:/usr/sbin

flashcache_start() {
if df -h | grep /var/lib/vz > /dev/null
then
echo "Flashcache allready running"
else
flashcache_load /dev/md2
mount /dev/mapper/pvec-storage /var/lib/vz
#mount /dev/mapper/pve-backup /mnt/backup
echo 1 > /proc/sys/dev/flashcache/md2+pve-storage/fast_remove
echo "Flashcache started"
fi
}

flashcache_stop() {
if df -h | grep /var/lib/vz > /dev/null
then
#umount /mnt/backup
umount /var/lib/vz
dmsetup remove pvec-storage
echo "Flashcache stopped"
else
echo "Flashcache not running"
fi
}

case "$1" in
    start)
flashcache_start
    ;;

    stop)
flashcache_stop
    ;;

    restart)
        $0 stop
        $0 start
    ;;
esac

exit 0

 

Enabling the init.d file.

Now we need to make the file executable and make sure it’s run on boot.

chmod +x /etc/init.d/flashcache
update-rc.d flashcache defaults

 

Give it a spin

Right, that should do it. Reboot your machine and see if it comes back.

reboot

If all went well, your drive should be mounted with FlashCache in between.

root@vh43:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
none                   32G  256K   32G   1% /dev
/dev/md1               10G  1.3G  8.2G  14% /
tmpfs                  32G     0   32G   0% /lib/init/rw
tmpfs                  32G     0   32G   0% /dev/shm
/dev/fuse              30M   12K   30M   1% /etc/pve
/dev/mapper/pvec-storage
                      1.8T  196M  1.7T   1% /var/lib/vz

You can also see the statistics of FlashCache by running:

cat /proc/flashcache/md2+pve-storage/flashcache_stats

That’s it! Your Proxmox system should now have it’s VMs on the FlashCache drive.

If you have any questions or feedback, just leave them below.

References:

Read more

Hey guys,

it’s been a while since my last post … I know … I guess my excuse is, I’ve been busy 🙂

Anyway, a friend of mine is trying to organize a Hackfest in Oulu, a nice town up north. I’m sure it’ll be an amazing event and I’m sure any help with setting it up will be appreciated.

So if you’re able to help, or just interested in joining an awesome hackfest in a different location than the usual ones, do check it out!

Oulu Open Hack, could it be done?

I’ll do my best to be there too!

–Florian

Read more

Red Alert 3 - Kirov reporting!We’re still months away from the release of Command And Conquer: Red Alert 3, but that’s not stopping EA from trying to work fans into a lather with these new, gorgeous screenshots of the series’ trademark Soviet war machine in action.

EA has yet to reveal a proper release date for the game, but they have mentioned that the title is planned to appear on the PlayStation 3 and Xbox 360 as well as its native PC platform.

These new screens look great and the water effects are gorgeous, but am I the only one noticing the Tesla Boats in the first picture? Who was the genius who signed off on adding electrical discharge weapons to a vehicle that floats on millions of gallons of ridiculously conductive fluid?

As you may know, I am a big fan of all the Command & Conquer games. But Red Alert was always my favourite. So, I can’t wait to get my hands on this.

Red Alert 3 - BuildingRed Alrt 3 - TanksRed Alert 3 - More shipsRed Alert 3 - Ships!

Read more

So DivX Corporation’s Stage6 has croaked. The service’s ‘goodbye, cruel word’ note says it was a victim of its own success, but that it proved ‘it’s possible to distribute true high definition video on the Internet’. What it really showed is how deliriously inefficient streaming video is, whether it’s high def or otherwise.

Stage6It cost at least $1m a month to run Stage6 with its 17.4 million unique users a month, whereas (at an informed guess) The Pirate Bay costs about $50,000 a month all-in for its 92.5 million. That’s $57,000 per million users for Stage6; $540 per million for The Pirate Bay (not including people using its tracker without visiting the site, which adds a lot of Mininova’s traffic as well, not to mention the other big indexes.) So at the very least, The Pirate Bay is a hundred and five times more efficient than Stage6 was.

But inefficiency is not the only reason the service is no more, while the vilified Pirate Bay, Mininova et al. are still with us. Stage6 was also a lot more illegal than a BitTorrent tracker — whether it pretended to be complying with the DMCA or not. Surprisingly under reported after the abrupt demise of the service was the 6th Feb US court ruling against DivX’s attempt to establish its protection under the DMCA’s safe harbour provisions ahead of a legal battle with Universal Music Group. My reading of the company’s consequent, speedy exit from the stage (and correct me if you think I’m wrong) is that Stage6 didn’t have the cash or confidence to test its luck any further. (How much this affects DivX as a whole remains to be seen. But only six days after the court decision, Jerome Vashisht Rota, the inventor of DivX and a major shareholder in DivX corporation, was openly dumping stock.)

It’s not hard to read the tea leaves. While GooTube (famously being sued by Viacom on pretty much the same grounds) probably won’t lose sleep, smaller players eating their lunches off of pirate content will be paying very close attention. VCs burning money on pushing streaming media to the masses will at least want to imagine some returns on their investment rather than the further expense of executives in the dock.

So why is the exit of Stage6 a step in the right direction? Because for all the hyperbole in the mainstream (and sometimes online) media about the YouTube or Google Video or Stage6 ‘revolution‘, the relationship to media they offer us is far too traditional. Come to this place. Be served your media (and suck down your advertising along with it). Go away again. Yes, we can upload material, but I’m not the only one who feels that this wasn’t the primary function of Stage6, even if it did distribute about 50,000 copies of STEAL THIS FILM II before its demise. No need to share, no need to understand the technology, no need to think. It’s what they called ‘lean back’ media: millions of people slouching thoughtlessly in front of an marketing-emitting portal.

The promise of P2P is a thorough breakdown of the kind of power that congeals in a portal like Stage6. A user-owned, user-operated infrastructure that doesn’t require massive investment, doesn’t by default allow oligarchs to make more money from us. A disruptive, mutable infrastructure that brings media to us in the context we choose, forcing a massive re-think about what, why and how we create — as individuals, as businesses, as a society.

It is lazy for us to rely at all on portals like Stage6, but worse than lazy, it’s dangerous. It suggests we don’t value the potential autonomy P2P offers us. Our old media masters profited from control of content: are we really so happy to swap them for new ones who profit from control of our eyeballs? However lazy we are, I think that most of us are able to see that that this isn’t a model that we want to encourage. The demise of Stage6 and the portals that will follow gives us cause to think about strengthening our infrastructures: and that can’t be a bad thing.

Source: Torrentfreak.com

Read more

Hey guys,

just recieved an IM from a friend with the link to this Picture. This would of been great @ FOSDEM and the XMPP DevCon. The Wifi would probably not have been that overloaded if people would of have respected this sign.

No Macs Allowed

Over and out.

Read more

Hey,

I just setup my SSL Certificate on this host, and updated my WordPress to use it. It’s a really nice certificate. Especially, for 19.99€ / year a Class2 Certificate and an additional IP is a great deal.

Website SSL1Not alot has changed at first sight. There is a new logo, but when you take a closer look, you will see that there is a Lock at the top, and the adressbar is yellow. Then there is also a lock at the bottom right. Both these locks indicate that the site is completely secure.

Website SSL2When you click on the lock, you get this information window, which states that the Website is secure and that it is a high-grade encryption. Only the best for my visitors! 🙂

Website SSL3The last window simply shows you all the information about the site owner, me!

As FlorianJensen.com now has an Exclusive IP, new site features will be available soon. The Store is currently being redone, but once it is ready, it will have alot of new features. Personally I encourage every site admin to get an SSL Certificate, as it is simply more secure, and you wouldn’t want to have your Network Administrator to see what you do. 🙂 That is why I even use an OpenVPN Connection when I am in public places. A good example is MSN. With a small tool like Wireshark, you can very easially capture all the data which is sent by MSN on the network. As this traffic is unencrypted, there is no problem for third-parties to read it. So use Jabber!

Read more

I just found this article on the Web. I find it very interesting that it is actually possible to do such things. For all the Users of the Flosoft.biz Webservices, we guarantee you, that the thing you are going to read, is NOT possible at our facilities.

Think your data’s secure. What about your data center?

The recent armed robbery of a Chicago-based co-location facility has customers hopping mad after learning it was at least the fourth forced intrusion in two years. They want to know how C I Host, an operator that vaunts the security of its data centers, could allow the same one to be penetrated so many times.

“I can’t believe a datacenter has been broken into that many times,” said Nick Krapf, president of Bloodservers.com, a startup game hosting provider, who said $15,000 worth of Dell servers were stolen in the October 2 heist. “What do you got to do to secure your facility for it not to happen? We’re pulling all our equipment from all their other facilities.”

In recent years, many IT administrators have found religion about installing security patches and deploying other measures such as intrusion prevention systems to keep criminals from accessing their systems and the data stored on them. The series of break-ins at C I Host is a reminder that safeguards must also extend to more mundane protections, including dead-bolt locks and steel cages.

CI Host likes to vaunt the security of its Chicago-based colocation facility, noting that safeguards include multiple layers of 24×7 security cameras, proximity card readers, biometric access controls and key pads, double-locking mantraps at data center entrance and 360-degree perimeter and roof surveillance. And yet, the same location has been the target of at least four burglaries or robberies since August 2005, according to police reports and former customers, some of whom say they lost sensitive data and hundreds of thousands of dollars in hardware.

Representatives from C I Host didn’t respond to emails requesting comment for this story.

In the most recent incident, “at least two masked intruders entered the suite after cutting into the reinforced walls with a power saw,” according to a letter C I Host officials sent customers. “During the robbery, C I Host’s night manager was repeatedly tazered and struck with a blunt instrument. After violently attacking the manager, the intruders stole equipment belonging to C I Host and its customers.” At least 20 data servers were stolen, said Patrick Camden, deputy director of news affairs for the Chicago Police Department.

The Chicago location has been hit by similar breaches in the past, according to police reports. One report detailing an occurrence on September 23, 2005, recounts a “hole cut through the wall coming out onto the hallway of third floor.” During a September 20, 2006 incident, an intruder “placed a silver + blk handgun to [victim’s] head and stated ‘lay down on the floor.'” The victim, a C I Host employee, was then blindfolded, bound with black tape and struck on the head with a weapon, according to the report.

To add insult to injury, C I Host representatives haven’t been particularly quick to alert customers of the robberies. It took them several days to admit the most recent breach, according to several customers who say they lost equipment. According to James F. Ruffer III, support people told him his server was down because the company had a problem with one of its routers. Krapf, the Bloodservers.com president, said he was told the same thing, as did several people recounting their experience on this forum.

“From a business owner perspective, my reputation is worth more to me than money,” said Ruffer. “The longer they waited the more money each particular person was losing. They should have been upfront and right on the ball.” ®

Original Story

It’s unbelievable what happens in the world of hosting. Concerning that, a friend of mine is currently working on a story about another host, which you’ll probably also find here in the next few weeks.

So stay tuned 🙂

Read more