Building a Tiny PC Cluster for Parallel Computing

Building an Affordable PC Cluster for Parallel Computing

For researchers and data scientists with demanding machine learning workloads, the choice between costly cloud servers and simple local resources can often be a sticking point. In this practical report, one data scientist charts their journey to setting up a cluster of compact, low-cost PCs for parallel computing—demonstrating both the effectiveness of hands-on, homegrown solutions and sharing technical lessons applicable for a broad audience, including those across Europe.

Motivation: Why Run Your Own Cluster?

Many machine learning simulations require extensive compute time, making reliance on a single laptop impractical. Cloud computing offers an alternative, but its distributed resources can be difficult to conceptualise or optimise, especially for users eager to understand core infrastructure. Thus, creating a physical cluster of small PCs presents an accessible, transparent method for distributing workloads.

Hardware Choices and Setup

To keep costs down, second-hand Lenovo M715q Tiny PCs served as the cluster’s backbone. These devices offer a balance between compact form factor, affordability, and sufficient power for distributed computing workloads.

Step 1: Installing Ubuntu

Cluster nodes were provisioned with Ubuntu Server for stability and performance. The installation process was straightforward using a bootable USB stick. Attention to boot prioritisation and LAN connectivity ensured smooth setup—a testament to how open-source software and commodity hardware can combine into a reliable platform.

Step 2: Networking Configuration

To maintain clarity in orchestration, static IP addresses were assigned to each node (e.g., 192.168.1.101, 192.168.1.102, etc.) via the local router. Uniform addressing simplifies management and supports automation scripts.

Step 3: Passwordless SSH and Sudo

Efficient parallel computing depends on seamless remote communication. SSH keys were created and shared across all nodes, facilitating passwordless login—crucial for running distributed R scripts or automating package installations. Optional configuration of passwordless sudo reduced interruption during automated tasks or software updates across the cluster.

Step 4: Software Automation and R Deployment

A key element was the installation of R and required statistical packages on all nodes. Using simple shell loops enabled batch updates and installations via SSH, while scripting in R streamlined package management. Template simulation scripts, where each node utilises all available CPU cores via R's future package, ensured computational resources were fully leveraged.

Uploading scripts to nodes relied on automated tools (like scp) to distribute appropriately edited R files, ensuring each node undertook a unique portion of the computational workload.

Step 5: Task Distribution and Execution

Workloads were divided evenly across the cluster, and simple code substitutions allowed each node’s R script to tackle a separate segment of simulation iterations. With tmux sessions running RScripts in the background, the system proved robust—even if a local workstation disconnected, jobs continued uninterrupted.

Reliability, Results Extraction, and Performance Comparison

Results were harvested from each node via automated SCP retrieval and aggregated for analysis. Continuous logging and spot checks helped detect incomplete or failed tasks, allowing for reruns and workload redistribution as necessary.

Critically, multi-node deployment led to noticeable reductions in simulation times. For instance, combining tuned XGBoost and logistic regression models on a single quad-core took 3.29 hours, but the same workload shrank to 1.8 hours when split over three nodes. More modest model combinations saw even greater efficiency improvements.

Increasing cross-validation folds from 5 to 10 raised total compute time, but improved statistical bias without adverse impacts on variance—confirming widely cited machine learning principles in a reproducible, local setting.

Lessons, Opportunities, and Outlook

Key takeaways include:

• Effective use of SSH and scripting can automate much of the installation, operation, and result collection process.
• Careful random seeding ensures reproducibility in parallel simulations.
• Simple notification scripts or functions could enhance usability further (e.g., automated emails on task completion, time estimations, and error-handling).
• While this workflow is cost-efficient for individuals or small research groups, larger entities may still rely on dedicated GPU clusters or managed cloud services for scale.

The author highlights future opportunities: packaging the setup as reusable scripts, learning distributed computing tools like OpenMPI, and better monitoring or error recovery mechanisms.

European Context and Accessibility

While the project described is not Europe-specific, it embodies the European emphasis on open-source, democratised technology, and sustainable digital practices. Given current privacy and sovereignty debates around cloud data, small clusters operated locally enable greater data control and transparency, aligning with growing European enterprise and research interests.

---

Source: R-Bloggers: Setting Up A Cluster of Tiny PCs For Parallel Computing – A Note To Myself