Werkstatt TechOps Report #1

ThoughtWorks Werkstatt Berlin hosts many different working groups, including several Cryptoparties, The Kids’ Hacker Club, and the Marx-Engels Werkshau group. In order for the groups to plan and stay in touch with each other in between their meetings at Werkstatt, we have implemented Werkstatt Groups, an online discussion forum based on NodeBB.

Creating a discussion channel for Werkstatt is tricky, since working group participants range from Tor project contributors, who are very knowledgable and concerned about technology and privacy issues, to kids, to political activists, who have other interests and areas of focus, and may be still learning about technology and privacy issues. So the Werkstatt Groups platform needs to be something that is usable across the spectrum, to be a place where privacy experts and privacy novices can intereact online.

Looking at the options available, a simple web forum became the most reasonable choice. With the many working groups at Werkstatt, managing dozens of mailing lists seems unworkable. Usenet, alas, has become entombed behind paywalls, and is inaccessable to most people, except through untrusted interfaces like Google Groups. Platforms that offer groups functionality like Facebook obviously have privacy issues, among many others, and old favourites like IRC and Jabber are not particularly suitable for asynchronous group discussion.

So how to set up a web forum that respects privacy? Run it on a Tor hidden service!

Before I explain how this was done, I need to start with a disclaimer: Werkstatt Groups makes no guarantees of privacy or anonymity, Tor is designed to provide anonymity. However, identifying all the possible ways in which the software running the forum may leak information is not easy, so use caution and report any issues or potential issues to us.

There are two ways to access this site, the recommended way is Tor Browser. Downloading and installing Tor Browser Bundle takes seconds and ensures that all your browser traffic goes over Tor and that your browser doesn’t leak any information and is difficult to fingerprint.

Using Tor Browser, you can access Werkstatt Groups using this url: http://vgnx2fk2co55genc.onion. Note HTTPS is not used, this is because the connection is already encrypted by Tor.

The other way of accessing it is by way of the public URL, http://groups.werkstatt.tw, which links to HTTPS when you access the forum. This is a reverse proxy running on a different server than the one that hosts the hidden service, accessing the hidden service over the tor network, thus making the site publicly accessible outside of the Tor network by way of a public url, while at the same time not revealing the location of the hidden service.

The NodeBB platform itself is a very dynamic, responsive platform which makes heavy use of websockets by way of socket.io, this is very advantageous over Tor, as a request to a hidden service needs to traverse 6 different servers, making page loads very expensive. Minimizing page loads by way of websocket requests compensates for this.

However, NodeBB also has some drawbacks, the platform uses Gravatar and Google Fonts, and socket.io includes a Flash fallback option, so a small Flash object is loaded in the site. All these issues are fixable, and are on our isssues list, however the best way to defend against these kinds of issues is to use Tor Browser. This way, even requests to Gravatar and Google Fonts go over Tor, and potentially dangerous plugins like Flash are blocked. However, JavaScript running in the browser is always a security concern, as exploits are possible. Also, NodeBB is beta software in very active development, and we are running the bleeding-edge head-of-branch, so expect glitches and some downtime.

OK, OK, so with all that out of the way, here is how the setup works. If all you want to do is use the forum, just get started here: http://groups.werkstatt.tw, however if you want to know how the setup works, keep reading. This assumes a relatively expert knowledge of server setup, including node, tor, nginx and iptables.


NodeBB installation instructions
for various platforms are available. However Werkstatt Groups uses git and npm to install NodeBB. So the steps are:

– Install and run Redis
– Pull head
git clone https://github.com/NodeBB/NodeBB.git
– Enter the directory, i.e:
cd /srv/http/NodeBB
– Build dependencies
npm install
– run the setup script
./nodebb setup
– change bind address in config.json to local interface only
"bind_address": "",
– If all that runs successfully, start NodeBB
./nodebb start
– Check the log
./nodebb log
This line should appear if all is good:
info: NodeBB is now listening on:

Tor Hidden Service

– Install and run Tor
– Configure the Tor hidden service in /etc/tor/torrc

HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 80

– restart Tor and find out onion address

cat /var/lib/tor/hidden_service/hostname

Your onion address will be different, of course.

To reduce the chances of the server revealing it’s address to other services based on outbound requests, add iptables rules to ensure that requests that come from the server go over Tor, here is an example of REDIRECT rules in the nat table OUTPUT chain configuration on Werkstatt Groups

-A OUTPUT -p icmp -j REDIRECT --to-ports 9040
-A OUTPUT -s xxx.xxx.xxx.xxx/32 -p tcp -m owner ! --uid-owner 43 -j REDIRECT --to-ports 9040
-A OUTPUT -p udp -m udp --dport 53 -j REDIRECT --to-ports 5353

‘xxx.xxx.xxx.xxx’ is the IP address of the hidden service and 43 is Tor’s userid, this means that all requests that originate fom this ip address that are not Tor itself are redirected over Tor’s Transparent Proxy, which I’ve configured to run on 9040. DNS Requests are redirected over Tor. For good measure, ping is short circuited as ell.

– Enable DNSPort and TransPort in /etc/tor/torrc

DNSPort 5353
TransPort 9040

Restart Tor again and visit the onion address in Tor Browser, you will see your NodeBB forum! Hooray!

Reverse Proxy

The reverse proxy runs nginx and tor.

In order to set up the public https server we need to use a different server. The IP address of the hidden service should not be listed anywhere, so it can not be used in your DNS zone.

So on this other server
– install and run tor
– install and run nginx
– make an ssl certificate and set up an https server with nginx
– set up proxy_pass in nginx for your onion node with websocket support, i.e.

proxy_pass http://vgnx2fk2co55genc.onion/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

Now we need to set up Tor to transparently proxy requests for Tor hidden services.

– In /etc/tor/torrc/

DNSPort 5353
TransPort 9040
AutomapHostsOnResolve 1

In addition to setting up the DNSPort and TransPort, The above code maps hidden services to the network, which we can then use in our iptables rules, as follows:

-A OUTPUT -p udp --dport 53 -j REDIRECT --to-ports 5353
-A OUTPUT -d -p tcp -j REDIRECT --to-ports 9040

And viola! Once you restart Tor and nginx, if you navigate to your https server, you should see your hidden service!

Questions and comments very welcome!

Leave a Reply