I’ve been wanting to try out more LLMs in a simple and comfortable way for myself and, at the same time, allow others at home to start tinkering.

That’s why, when I got a day off (that is, 100% free and just for myself), I “hijacked” my colleague José (who was also interested) and grabbed a gaming laptop that’s been dying of laughter in a corner to set up a server that supports large language models (LLMs) using Debian as the operating system and Ollama for model management.

I have divided the process into two articles to focus on the important aspects of each phase:

1. Having a server with Debian on an old laptop,
2. Setting up a large language model (LLM) server with access from the local network,

I think anyone can imagine what it’s like to have an open and powered-on laptop at home just to use it from the local network… so I decided to configure Debian without a graphical environment, with system hibernation disabled to keep the laptop closed on a stand, upright, and with quite good ventilation.

Here’s how.

1. Installing Debian without a Graphical Environment

Start by installing Debian without a graphical environment and make sure to enable the SSH server during installation. This will allow us to manage the server remotely without the need for a desktop environment.

2. General System Configuration

Log in as the root user to perform the general system configuration. Keep in mind that some configurations and software installations must be done as the root user to have the necessary permissions. We will use the following command to switch to root:

su -

Then use the following script to configure Debian’s software sources:

#!/bin/bash

mv /etc/apt/sources.list.d/contrib.list /etc/apt/sources.list.d/contrib.list.bak

DIST=$(lsb_release -c -s)
echo "deb http://deb.debian.org/debian ${DIST} main contrib non-free-firmware non-free" >> /etc/apt/sources.list
echo "deb-src http://deb.debian.org/debian ${DIST} main contrib non-free-firmware non-free" >> /etc/apt/sources.list
echo "deb http://deb.debian.org/debian-security/ ${DIST}-security main contrib non-free-firmware non-free" >> /etc/apt/sources.list
echo "deb-src http://deb.debian.org/debian-security/ ${DIST}-security main contrib non-free-firmware non-free" >> /etc/apt/sources.list
echo "deb http://deb.debian.org/debian ${DIST}-updates main contrib non-free-firmware non-free" >> /etc/apt/sources.list
echo "deb-src http://deb.debian.org/debian ${DIST}-updates main contrib non-free-firmware non-free" >> /etc/apt/sources.list
apt -qy update

[!question] Why do we use a script?

If you notice, the script has a line that says:

DIST=$(lsb_release -c -s)

This line retrieves the version of Debian that we have installed and customizes the software sources for the installation we are configuring.

With the software sources now configured, we install sudo to facilitate actions that require superuser permissions since we can perform them from our non-privileged user in an atomic way (only for the accompanying command).

To install sudo, use the following command:

apt -qy install sudo

And, of course, we need to assign it to our user with:

usermod -aG sudo <user>

where <user> is replaced by your username.

With all this, we can now “close” the root session with:

exit

3. Configuring a Static IP on the Local Network

Since all of this is being done to access Ollama remotely… we need the laptop’s IP to be static, so… let’s configure it.

The IP is configured in the /etc/network/interfaces file and, just in case, let’s make a backup of the file with:

sudo cp /etc/network/interfaces /etc/network/interfaces.bak

Now we can be a bit clumsy, I mean, we can start tinkering, I mean, we can modify the file.

To do this, open it with nano (although if you prefer vim, make sure you know how to exit afterwards):

sudo nano /etc/network/interfaces

Find the network interface that interests you. It will be one of the lines that say:

iface ... dhcp

Change dhcp to static and below (mind the indentation!), write the following:

    address 192.168.1.100 # or the IP you prefer
    netmask 255.255.255.0
    gateway 192.168.1.1 # or the one corresponding to the IP you have set

As a small extra, you can change the default DNSs to CloudFlare’s (1.1.1.2 and 1.1.1.1) and/or Google’s (8.8.8.8 and 8.8.4.4). This is done in the /etc/resolv.conf file and you can resolve it (pun intended) with the following command:

sudo cp /etc/resolv.conf /etc/resolv.conf.bak && sudo echo -e "nameserver 1.1.1.2\nnameserver 1.1.1.1\nnameserver 8.8.8.8\nnameserver 8.8.4.4" > /etc/resolv.conf

[!question] Why change the default DNS?

Modifying the DNS servers allows us to use faster and more secure name resolution services than those provided by the internet service provider (ISP). This can improve browsing speed and offer an additional layer of privacy by using public DNS servers like Cloudflare’s (1.1.1.1) and Google’s (8.8.8.8).

In any case, to make the changes take effect, you need to restart the network service with:

sudo systemctl restart networking

4. Disable System Hibernation

As mentioned, I want to use the laptop as a server on the local network. I don’t want it to take up much space, so I’ll keep it closed and upright on a laptop stand.

By default, laptops go into hibernation when we close them, so we need to disable system hibernation to keep it running even when it’s closed.

To do this, first, as always, backup! In this case, the /etc/systemd/sleep.conf file:

sudo cp /etc/systemd/sleep.conf /etc/systemd/sleep.conf.bak

Then open the file:

sudo nano /etc/systemd/sleep.conf

Find the line that says [Sleep] and, just below it, add the following lines:

AllowSuspend=no
AllowHibernation=no
AllowSuspendThenHibernate=no
AllowHybridSleep=no

These lines may already exist, either commented out or with values set to yes. The important thing is that the file has the lines as I have provided.

With this, we now have a Debian server on an old laptop and can install any type of service. In our case, a service with Ollama and a web interface.

But that, as they say, is another story.