Self-Hosted SearXNG instance on OpenBSD

       974 words, 5 minutes

Some time ago, I discovered and used searx on OpenBSD . This worked quite well but there were a few annoying bugs that I couldn’t solve. Mainly using OpenSearch with Firefox and timeouts with some Big Tech search engines. After struggling enough, I decided to switch to SearXNG . It has some cons compared to SearX but, regarding my needs and beliefs, the pros win.

The original documentation for Linux is available here . I’m doing it on OpenBSD 7.3.

Preliminary steps

Set up a dedicated user:

# vi /etc/login.conf.d/searxng
searxng:\
        :openfiles=4096:\
        :tc=daemon:

# useradd -g =uid -c "SearXNG metasearch engine" \
  -L searxng -s /sbin/nologin -d /home/searxng   \
  -m -r 2000..2500 _searxng

Raise some system limits:

# vi /etc/sysctl.conf
kern.seminfo.semmni=1024
kern.seminfo.semmns=4096
kern.seminfo.semmnu=1024
kern.seminfo.semmsl=1024
kern.seminfo.semopm=1024
# egrep -v "^#" /etc/sysctl.conf | xargs sysctl

Installation

Install some minimum system dependencies:

# pkg_add git python%3.10 libxslt

Install SearXNG sources and prepare the environment:

# doas -u _searxng /bin/ksh -l

$ git clone https://github.com/searxng/searxng ~/src

$ python3 -m venv ~/pyenv
$ echo ". ~/pyenv/bin/activate" >> ~/.profile
$ ^D

Install the required Python modules:

# doas -u _searxng /bin/ksh -l

$ command -v python && python --version
/home/searxng/pyenv/bin/python
Python 3.10.11

$ pip install -U pip setuptools wheel pyyaml

$ cd ~/src
$ pip install -e .
(...)
Successfully built Brotli fasttext-predict lxml setproctitle uvloop MarkupSafe
(...)
Successfully installed Brotli-1.0.9 MarkupSafe-2.1.2 Werkzeug-2.3.4
anyio-3.6.2 async-timeout-4.0.2 babel-2.12.1 blinker-1.6.2 certifi-2022.12.7
charset-normalizer-3.1.0 click-8.1.3 fasttext-predict-0.9.2.1 flask-2.3.2
flask-babel-3.1.0 h11-0.12.0 h2-4.1.0 hpack-4.0.0 httpcore-0.14.7 httpx-0.21.2
httpx-socks-0.7.2 hyperframe-6.0.1 idna-3.4 itsdangerous-2.1.2 jinja2-3.1.2
lxml-4.9.2 markdown-it-py-2.2.0 mdurl-0.1.2 pygments-2.15.1
python-dateutil-2.8.2 python-socks-2.3.0 pytz-2023.3 redis-4.5.4 rfc3986-1.5.0
searxng-2023.5.10+cb1c3741 setproctitle-1.3.2 six-1.16.0 sniffio-1.3.0
typing_extensions-4.5.0 uvloop-0.17.0

Configuration

I’ll be using a configuration file that shall not be modified during source updates. It is then stored in $HOME.

# doas -u _searxng /bin/ksh -l

$ sed -e "s/ultrasecretkey/`openssl rand -hex 16`/g"   \
  ~/src/searx/settings.yml > ~/settings.yml

$ echo 'export SEARXNG_SETTINGS_PATH=~/settings.yml' \
  >> ~/.profile
$ . ~/.profile

$ vi ~/settings.yml

Check that everything works as expected:

$ python ~/src/searx/webapp.py
 * Serving Flask app 'webapp'
 * Debug mode: off
^C

Daemonize using Gunicorn

In my first attempt, I used uwsgi as I did with searx. But I find it a bit complicated to configure and not resource effective. So I decided to go with Gunicorn :

# doas -u _searxng /bin/ksh -l

$ pip install gunicorn
(...)
Successfully installed gunicorn-20.1.0

$ vi searxng.conf.py
# Server Socket ========================================================
bind = "127.0.0.1:8888"
backlog = 128

# Worker Processes =====================================================
workers = 1
threads = 2
keepalive = 5

# Process Naming =======================================================
proc_name = "searxng"

# Config File ==========================================================
wsgi_app = "searx.webapp"

# Server Mechanics =====================================================
chdir = "/home/searxng/src"

# Logging ==============================================================
accesslog = '-'
errorlog = '-'

syslog = True
syslog_addr = 'unix:///dev/log#dgram'
syslog_facility = 'daemon'
syslog_prefix = 'searxng'
#EOF

There is no startup file for Gunicorn. So let’s create one:

# vi /etc/rc.d/searxng
#!/bin/ksh

daemon="/home/searxng/pyenv/bin/gunicorn"
daemon_execdir="/home/searxng"
daemon_conf="/home/searxng/searxng.conf.py"
daemon_pidfile="/home/searxng/searxng.pid"
daemon_flags="-D -c ${daemon_conf} -p ${daemon_pidfile}"
daemon_user=_searxng

. /etc/rc.d/rc.subr

pexp="python3: gunicorn: master \[searxng\]"

rc_start() {
        rc_exec ". ~/.profile ; ${daemon} ${daemon_flags}"
}

rc_cmd $1
#EOF

# chmod 0555 /etc/rc.d/searxng
# rcctl enable searxng
# rcctl start searxng
searxng(ok)

Expose through a Reverse-Proxy (relayd)

You need, httpd(8) and acme-client(1) to get an SSL certificate and do the HTTP to HTTPS redirection. If you don’t know how to do this, their man pages are self-explanatory ;-)

Then configure relayd(8) with something like:

# vi /etc/relayd.conf
table <searxng> { 127.0.0.1 }

http protocol wwwtls {
  tls keypair <YOUR_FQDN>

  tcp { nodelay, socket buffer 65536 }

  match request header set "X-Forwarded-Proto" value "https"

  match response header set \
    "Strict-Transport-Security" value "max-age=31536000; includeSubDomains; preload"
  match response header set \
    "X-XSS-Protection" value "1; mode=block"
  match response header set \
    "X-Content-Type-Options" value "nosniff"
  match response header set \
    "Permissions-Policy" value "accelerometer=(),ambient-light-sensor=(),autoplay=(),camera=(),encrypted-media=(),focus-without-user-activation=(),geolocation=(),gyroscope=(),magnetometer=(),microphone=(),midi=(),payment=(),picture-in-picture=(),speaker=(),sync-xhr=(),usb=(),vr=()"
  match response header set \
    "Feature-Policy" value "accelerometer 'none';ambient-light-sensor 'none'; autoplay 'none';camera 'none';encrypted-media 'none';focus-without-user-activation ' none'; geolocation 'none';gyroscope 'none';magnetometer 'none';microphone 'none';midi 'none';payment 'none';picture-in-picture 'none'; speaker 'none';sync-xhr 'none';usb 'none';vr 'none'"
  match response header set \
    "Referrer-Policy" value "no-referrer"
  match response header set \
    "X-Robots-Tag" value "noindex, noarchive, nofollow"

  forward to <searxng>
}

relay www4tls {
        listen on $ext_ip port 443 tls

        protocol wwwtls

        forward to <searxng> port 8888 check http "/" code 200
}

From there, (re)start relayd(8) and you’re good to go.

Custom logo & favicons.

My daughter drew my a nice illustration to be used as a logo and favicon. Customizing SearXNG is not supported so a tweak has to be used.

As tweaks will be overwritten after each upgrade, I wrote a script to apply the modifications automagically:

# doas -u _searxng vi ~_searxng/update_img
#!/bin/sh
[[ ! -d ~/img ]] && mkdir ~/img
cp -p ~/img/* ~/src/searx/static/themes/simple/img/
rm ~/src/searx/static/themes/simple/img/{favicon,searxng}.svg
exit 0
#EOF

# chmod 0750 ~_searxng/update_img
# doas -u _searxng ksh -l ~_searxng/update_img

# rcctl restart searxng

Update searxng

SearXNG follows a rolling release model. This means that the latest commit on the master branch is to be considered the latest stable version. As there are a few steps to repeat, I just wrote a script to update.

# doas -u _searxng vi ~_searxng/update
cd ~/src
git fetch origin "HEAD"
git reset --hard "origin/HEAD"
pip install -U pip
pip install -U setuptools
pip install -U wheel
pip install -U pyyaml
pip install -U -e .
#EOF

# chmod 0750 ~_searxng/update

The upgrade is a simple combination of:

# doas -u _searxng ksh -l ~_searxng/update
# doas -u _searxng ksh -l ~_searxng/update_img

# diff -U2 ./src/searx/settings.yml settings.yml

# rcctl restart searxng
searxng(ok)
searxng(ok)

The End

From there, configure some more settings if you like. You may add or delete search engines. I kept the defaults. But I use a Redis cache and the hostname_replace plugin (to use invidious etc).

I didn’t look at building an OpenBSD port. The subset of Python plugins is way over my understanding yet. There seem to be really frequent changes / updates and I prefer to keep them in $HOME than wreck my whole /usr/local.

Related posts: