Offen Fair Web Analytics is a open and self hosted web analytics tool that treats operators and users as equal parties.

Packaging and testing | Offen
14 Sep 2020, Frederik Ring

Episode Six — Packaging and testing

It feels a little surreal to write this, but: this post marks the end of Milestone 6, which is the last one defined in our initial product plan defining the scope of our support by the NGI Zero PET initiative.

In these last weeks we focused on packaging and testing, which - who would have thought - uncovered some issues we didn't know about yet. But it also felt very rewarding to see the work of the last ~9 months paying off, now that we and others can deploy and use Offen Fair Web Analytics easily. Having designed Offen Fair Web Analytics as a self hosted solution from the start, we managed to establish a unique characteristic when comparing Offen Fair Web Analytics with other solutions out there: if you're looking to self host your analytics software, it won't get much easier. If you are unsure about that claim, check out the rest of this post to see what that actually means.

During Milestone 6 we have released the following versions:

As always, you can download the latest release from https://get.offen.dev or pull it from Docker Hub.

Achievements

Offen Fair Web Analytics runs almost everywhere

One of the design goals of Offen Fair Web Analytics is to make it really easy to install. Nevertheless, using self hosted software can be daunting if you haven't done it before, or you are not well versed with using the CLI. To help people unsure about what to do getting started, we published a lot of options and contributed to community resources:

In case you know of another great target for installing Offen Fair Web Analytics, let us know and we'll check out the option right away.

deb package for easier setup

Installing Offen Fair Web Analytics from our binary distributions hasn't been too complicated already, but starting with version 0.1.6 we now also ship Offen Fair Web Analytics as a deb package. This means, installation on Ubuntu or Debian servers is now as easy as:

curl -sSL -o offen.deb https://get.offen.dev/deb
sudo dpkg -i offen.deb
sudo systemctl enable offen
sudo systemctl start offen

We also updated our installation tutorials to reflect this. Packaging code lives in the offen/deb repository. A nice side effect of this is that we'd be pretty much ready to set up a repository for people to install Offen Fair Web Analytics using the apt package manager in case this is requested.

Helping others to install and run Offen Fair Web Analytics

While we've been doing a lot of test installations of Offen Fair Web Analytics ourselves in the past weeks, we've also seen others starting to install of Offen Fair Web Analytics. We've seen some expected teething troubles we've fixed along the way, but more importantly we've gained further insights on how people deploy and use Offen Fair Web Analytics, allowing us to further optimize for the relevant use cases.

Helping others to start contributing to Offen Fair Web Analytics

Offen Fair Web Analytics is free and open for everyone to use with no strigs attached. There is no business model like a hosted version or similar behind it. This is why it's important for us to open up the development of Offen Fair Web Analytics to the community now that we've set the foundation. To kick this off, we will be participating in this year's Hacktoberfest. Check out the relevant issues on our repository, forward this to whoever might be interested, or start hacking on Offen yourself right away. We are also happy about any kind of feedback on our roadmap.

Next up

Spreading the word

We've built a unique analytics software, and we sense the time is just about right for shifting paradigms when it comes to collecting and handling data. However, if noone knows about Offen Fair Web Analytics, noone can use it. Now that we have a working beta, we can start marketing Offen Fair Web Analytics more aggressively all across the internet. If you have ideas for a good fit in terms of audience, let us know.

Performance improvements

Securing the data collected as much as possible has always been (and will always stay) the most important aspect when developing Offen Fair Web Analytics. This makes having Offen Fair Web Analytics perform well on large datasets (i.e. sites with a lot of traffic) is hard. While it's already usable in these scenarios - and it's an obvious tradeoff where we have a clear priority - there is still room for improvement.

This is why we'd like to look into ways of making Offen Fair Web Analytics faster while still satisfying the strict privacy and security requirements we have. Check this issue for ideas about how we plan to implement this.

Defining the future of Offen Fair Web Analytics

This post is closing Milestone 6 out of 6. This means we have successfully built and shipped a working beta version of Offen Fair Web Analytics. It also means it's up to us to define what we want to do next.

The obvious path is further extending and maintaining Offen Fair Web Analytics, which we will continue to do. Yet, we'd also like to work on ideas about how we can make the tech, and also the ethical principles that Offen Fair Web Analytics is built upon available to the public. In the end, Offen Fair Web Analytics is a lot more than just an analytics tool.

Adding a second locale

Localizing software at an early stage is hard, so we developed Offen Fair Web Analytics as English-only up until now. Luckily, we already placed all the hooks need for localizing Offen Fair Web Analytics in our codebase. Now that Offen Fair Web Analytics has become more stable and mature, we think it's a good time to start adding a second locale. In our case, German is the obvious choice, so we are planning to make Offen Fair Web Analytics available in German as well in the next months. If you're interested in helping us localize Offen Fair Web Analytics in the language of your choice, we'd be happy to hear from you in the corresponding issue on GitHub. Vielen Dank.

Getting your hands dirty

Packaging Offen Fair Web Analytics as a snap

Snapcraft is a relatively recent way of packaging apps for use across all Linux distributions. While it has also received its fair share of criticism, it also provides a really simple way of packaging and distributing apps like Offen Fair Web Analytics.

To build a snap from Offen Fair Web Analytics, first install snapcraft. If you're on Ubuntu or Debian, you can use snap to install it:

snap install snapcraft

Next, you can create an empty directory where we build the snap:

mdkir -p offen-snap && cd offen-snap

The snap is defined in a file called snap/snapcraft.yaml :

mkdir snap && touch snap/snapcraft.yaml

You can define the snap in snap/snapcraft.yml like this:

name: offen
version: 0.1.5
summary: Offen
description: |
  Offen is an open alternative to common web analytics tools.
  Gain insights while your users have full access to their data.
  Lightweight, self hosted and free.
base: core18
confinement: strict
parts:
  offen:
    plugin: dump
    # this downloads the tarball from the official download URL
    source: https://get.offen.dev/v$SNAPCRAFT_PROJECT_VERSION
    source-type: tar
passthrough:
  # the layout connects the confined snap fs with the user's
  # default filesystem layout
  layout:
    /var/www/.cache: # this is the certificate cache
      bind: $SNAP/var/www/.cache
    /var/opt/offen: # this is where the database file will be stored
      bind: $SNAP/var/opt/offen
    /etc/offen/offen.env: # this is the configuration file
      bind-file: $SNAP/etc/offen/offen.env
apps:
  offen:
    command: offen-linux-amd64

The snap is now ready to be built:

$ snapcraft
Launching a VM.
snap "snapd" has no updates available                                           
...
Snapping |                                                                                
Snapped offen_0.1.5_amd64.snap
$

Now, you can install Offen Fair Web Analytics on your system from the built file using the snap command (the dangerous flag is needed as we did not sign the package):

$ snap install --dangerous offen_0.1.5_amd64.snap
Installed offen 0.1.5
$ which offen
/snap/bin/offen

And that's it, your system is now ready to use Offen Fair Web Analytics, but more importantly, this is another simple and discoverable way of installing it for less tech-savvy users. We'll keep our eyes open for when it's time to officially start publishing Offen Fair Web Analytics as a snap.

Feedback? Found a bug?

If you have any feedback, comment or bug report on this milestone release, we'd love to hear from you. Open an issue or send us an email at hioffen@posteo.de.