A Shared GPU Cluster with Slurm and BeeGFS
A handful of GPU servers shared by a team tends to go through the same stages. At first people SSH into whichever machine looks idle. Then someone’s job gets killed because two people picked the same GPU. Then datasets end up copied onto every machine, each copy slightly different. A job scheduler fixes the first two; a shared filesystem fixes the third. This post walks through the setup I use: one login node plus a few GPU servers, with Slurm for scheduling, BeeGFS for a shared /home, UFW as the host firewall, and Ansible to keep every node consistent.
Migrating from WordPress to Hugo
I recently switched from WordPress to Hugo. This post outlines the reasons behind my migration and provides a step-by-step guide on how to make the transition. Rethinking Why Use WordPress I began using WordPress in 2009 when static site generators were not prevalent, and WordPress was the go-to CMS for nearly everyone. However, since my blog had infrequent updates but high readership, I started reconsidering the use of WordPress. Five years ago, I decided to convert the WordPress site into static HTML and uploaded it to a CDN to accelerate access speed for users in mainland China.
Signing commits using GPG in Git
Git is cryptographically secure, but it’s not foolproof. If you take work from others on the internet and want to verify that commits are actually from a trusted source, Git can sign and verify work with GPG. Generate a GPG Key To sign anything, you first need GPG installed and a personal key. Check whether you already have one: gpg --list-keys /home/hzxie/.gnupg/pubring.gpg ------------------------------ pub 4096R/3DBF9592 2016-02-15 uid Haozhe Xie (GPG key for GitHub) <noreply@haozhexie.com> sub 4096R/BFEB9969 2016-02-15 If the list is empty, generate a key. On GPG 2.1 and later, --full-generate-key lets you choose the algorithm and a 4096-bit RSA key, which GitHub recommends:
Build Tengine, MariaDB and PHP on Ubuntu 14.04
Introduction Nginx is one of the most popular web servers in the world and is responsible for hosting some of the largest and highest-traffic sites on the internet. Tengine is a branch of Nginx which is created by Alibaba Inc. MariaDB is a database server developed by some of the original authors of MySQL and offers drop-in replacement functionality. Note: This guide is from 2014 and builds every component from source on Ubuntu 14.04, which has long since reached end-of-life. On a current system you would install Tengine/Nginx, MariaDB, and PHP from the official APT repositories and manage them with systemd instead of the SysV init.d scripts shown below. It’s kept here as a historical walkthrough.
Getting Started with Hadoop 2.0
Apache™ Hadoop® is an open source software project that enables the distributed processing of large data sets across clusters of commodity servers. It is designed to scale up from a single server to thousands of machines, with a very high degree of fault tolerance. Rather than relying on high-end hardware, the resiliency of these clusters comes from the software’s ability to detect and handle failures at the application layer. Hadoop 1 popularized MapReduce programming for batch jobs and demonstrated the potential value of large scale, distributed processing. MapReduce, as implemented in Hadoop 1, can be I/O intensive, not suitable for interactive analysis, and constrained in support for graph, machine learning and on other memory intensive algorithms. Hadoop developers rewrote major components of the file system to produce Hadoop 2. To get started with the new version, it helps to understand the major differences between Hadoop 1 and 2.
Jenkins Installation and Configuration with Maven and GitHub
In this tutorial we install Jenkins and configure it to build a Maven project hosted on GitHub. Note: This post has been updated for Jenkins 2.5x running on Java 21. The original 2014 version deployed Jenkins as a WAR inside Tomcat 7 on Java 7 and walked through a UI that has since changed substantially. Modern Jenkins is self-contained, so you no longer need a separate servlet container, and Git/Maven support now ships with the suggested plugins. The setup below is therefore shorter than it used to be.
Building Vim from Source
Most Linux distributions ship a stripped-down Vim — often without +python3, +clipboard, or GUI support, and rarely the newest release. Building Vim yourself fixes all of that: you decide exactly which language interpreters and features get compiled in, and you get whatever version is on Vim’s master branch. The process is straightforward. This guide walks through it on a Debian-based distro such as Ubuntu. Note: This post has been updated from its original 2013 version to match how Vim is built today. The source has moved from vim.org’s Mercurial repository to GitHub, the Python 2 interpreter has been swapped for Python 3 and the GTK2 GUI for GTK3, and configure flags that newer Vim removed (such as --enable-sniff and the now-default --enable-multibyte) have been dropped.
Install PHP and PHPUnit on Windows/Ubuntu
Note: This post has been updated for PHP 8.5 and PHPUnit 13. The original 2013 version installed PHPUnit through PEAR (go-pear, pear channel-discover pear.phpunit.de), but PEAR has since been retired and PHPUnit no longer ships through it. The current approach is to install PHPUnit with Composer or as a standalone PHAR, both shown below. PHPUnit is the de-facto testing framework for PHP. This guide gets PHP and PHPUnit running on Windows and Ubuntu, then writes and runs a first test to confirm the setup works.