Tag Archives: p2p

One of the main problems in making a site public on a P2P network is its accessibility depends on a running server process on the initial host, until the files are seeded by other peers on the network. Fortunately, some of DAT’s developers operate a hosting service called ‘HashBase’, which we can use to make our sites persistent.

A site can be developed and uploaded to HashBase very easily from within Beaker’s integrated editor. Mine is a very basic site using Bootstrap.css. When done, review and publish the changes, so the finished article is displayed in the browser at the unique address.

Next, register an account with HashBase.io. There isn’t much in the way of account configuration here (it essentially provides a certain amount of storage associated with an account) so we can get straight to uploading the site’s files.

My site wasn’t accessible after my first attempt, and I assumed it just required time for the changes to propagate on the network. It turned out that dat.json needed to be modified in order to enable HashBase to host it – this isn’t mentioned on the Hashbase site, but instead in their documentation on GitHub. I learned the following lines must be added to dat.json:

This was enough to make the site reachable at dat://sapphire-dat.hashbase.io after re-uploading the archive. Reviewing and publishing changes in Beaker might cause dat.json to be reverted to its default, so it might be worth copying the above into another file called ‘template.json‘.

There are other recommended configuration options that might be important if we want to host something more than a static site. If you want to enable HTTPS, HashBase can sort the certificate provisioning if the following lines are also added:

I’ve hinted, in last week’s post, about the development of a feature that could help Minds.com evolve into a decentralised social network and bring P2P into the mainstream, thereby solving the growing privacy and censorship concerns that are associated with a centralised social network. This feature is based on an application layer protocol known as ‘DAT’. There are reasons to believe it’s likely to succeed where previous ideas failed: Since DAT works entirely at the application layer, and is implemented using Node.js, there’s very little effort or learning curve involved for developers and users of DAT applications. Web applications can be extended to support it, if the demand is there, using already published librariesthat are extensively documented.
For those who aren’t developers, there is a working browser that anyone, without technical skills or knowledge, can use to browse and publish sites on the DAT Web.

What is DAT?

DAT started life with the scientific community, which had a need for a more effective method of distributing, tracking and versioning data. In the conventional Web, data objects are moved, Web pages are deleted and domains expire – this is referred to as ‘content drift’. We’ve all come across an example of this in the form of ‘dead links’. When using a hyperlink to reference a data object or Web page, there is no guarantee that link would be valid at some point in the future. DAT was proposed as a solution to this.
But what does this have to do with censorship and privacy, you’re probably asking? The answer to this question is in how data is distrubuted, discovered and encrypted.

Merkle Trees, Hashing Algorithms and Public Key Encryption

The DAT protocol is essentially a real-world implementation of the Merkle Tree data structure, with the BLAKE2b and Ed25519 algorithms for identification, encryption and verification (other docs state that SHA256 is used as the hashing algorithm). It’s not necessary to understand this concept in order to develop DAT applications, since there are already libraries for implementing this, but for the curious, I reccommend reading Tara Vancil’s explanation first before moving on to the whitepaper.

An important point Vancil made was DAT is about the addressing and discovery of data objects, not the addressing of servers hosting those objects. Data objects are not bound to IP addresses or domains either. Each data object has its own address, and that address is determined by its cryptographic hash value – a file’s hash digest will be static, regardless of where it’s hosted. This is important, because we’re accustomed to thinking of the Internet/Web in terms of the client/server model, and proposed solutions for privacy and anti-censorship typically try to deal with the problem of decentralised host discovery in a peer-to-peer (P2P) network.

Some form of data structure is required to make the data objects addressable and to enable their integrity to be verified. A DAT peer-to-peer network uses Merkle Trees for this, where all ‘leaves’ and nodes contain the hash values of the data objects they represent, and the root node contains the hash digest of all its child nodes. In other words, as the whitepaper puts it, ‘each non-leaf node is the hash of all child nodes‘.
Not only does this provide a way of verifying the integrity of the data objects – the root node’s digest will change if there’s any modification to a data object represented in the tree – it provides the means to an efficient lookup system, as the root hash digest becomes the identifier for a dataset.

Obviously, this means clients would need to fetch the root node’s value for a given dataset from a trusted source, which might be one of many designated lookup peers on the network. If the client wanted a given data object, it wouldn’t need to fetch everything referenced under the root node, but just the root node value, the parent node of the requested objects, and the hash values of the other parent nodes.

Addressing, References and Security

Now, let’s get into the more specific aspects of how Merkle Trees are implemented in the context of DAT. All the ‘leaf’ nodes in the DAT Merkle Tree contain a BLAKE2b or SHA256 (depending on the docs being read) hash digest of the referenced object. All parent nodes contain the hash digest and a cryptographic signature. The signature is generated by creating Ed25519 keys for each parent node and using them to sign the hash digest.

When sharing a locally-created site in the Beaker browser, or viewing one already shared on the network, you might notice the URI following ‘DAT://’ is a long hexadecimal string. This is actually the Ed25519 public key of the archive containing the referenced object being shared, and it’s used to encrypt and decrypt the content. The corresponding private key is required to write changes to the DAT archive.
The public key is, in turn, hashed to generate a discovery key, which is used to find the data objects. This ensures no third-party can determine the public key of a private data object that hasn’t been publicly shared.

Beaker

The Beaker browser looks very much like the standard Firefox browser on the surface, and it can be used to browse both DAT:// and HTTP:// addresses. As we can see, DAT sites are rendered just as well as those on the conventional Web. The only problem is that, as with Tor and I2P, sites are hosted on machines that aren’t online 24/7, so many of them are unreachable at a given time.

From the Welcome dialogue, we can get straight to setting up a personal Web site dor publishing on the DAT Web. A default index page, script.js and styles.css are included ready for us to customise. In addition, Beaker allows us to share the contents of an arbitrary directory on the machine it’s running on.

Previously-created sites are available under the ‘Library‘ tab in the main menu. Sites that aleady exist will be listed under the ‘Your archives‘ section, and can be modified and/or published.

What happens to a published site when the local machine is offline? There is a method to keep a site accessible, by somehow getting another person or machine to ‘seed’ the data. This is a short-hand way of saying another person could fetch a copy of the site and re-share it over the network. Seeding happens automatically as a user is actively browsing a DAT site.

The Node.js Modules

Several Node.js modules provide libraries that developers can use to implement DAT features in their applications.

hypercore: A component for creating and appending feeds, and verifying the integrity of data objects. The API exposes a number of methods under the ‘feed’ namespace for reading, writing and querying feeds.

hyperdrive: This is a distributed filesystem for P2P. One of the design principles is to reproduce, as closely as possible, the APIs as the core Node.js filesystem component, thereby making it transparent to application developers. This module enables a local file system to be replicated on other machines.

dat-node: A high-level component that developers could use to bring together other DAT modules and build DAT-capable applications.

hyperdiscovery: Module for network discovery and joining. Running two instances of a hyperdiscovery module will result in a given archive key being replicated.

dat-storage: The DAT storage provider. Used for storing secret keys, among other things, using the hyperdrive filesystem.

In conjunction with Electron.js and Node.js, the above modules can be used to develop a DAT-enabled desktop application, of which Beaker is just one example.

Node Discovery in Practice

Two components are used for this: discovery-channel and discovery-swarm. The discovery-channel component searches BitTorrent, DNS and Multicast DNS servers for peers, and advertises the address/port of the local node. Therefore, it is based on the bittorrent-dht and dns-discovery modules. Using discovery-channel, the client can join channels, terminate sessions, call handlers on session initiation and fetch a list of relevant channels. The network-swarm module uses discovery-channel to connect with DAT peers and control the session.

Share this:

Like this:

Given how easy it is to get started with I2P, I’m surprised it hasn’t got anywhere near the same attention as Tor. Like Tor, I2P is an overlay network that creates a multi-layered outbound tunnel through a series of ‘routers’, and each router along a path can only decrypt one layer. A separate inbound tunnel is created for inbound traffic. In theory, the payload is only accessible to the endpoints. The routing and addressing of I2P peers is also decentralised.
For the users, getting started is a simple matter of downloading and running the client/server software, and waiting for it to build a list of reachable clients.

Pseudonymity
I used the word ‘pseudonymity’ intentionally, and it has a specific meaning – most of us want an online identity, but it’s not always desirable to associate it with our offline lives. As it stands, with most people using social media on the clear Web, it’s fairly trivial to identify who posted what, since profiles are mostly based around real names and real world identities. If you’re using FaceBook, Google and generally social media, there’s a good chance the service providers have built a fairly detailed profile of you – the town of residence, the sites you browse, the people you hang out with, etc., regardless of how much of that info is public. Censorship and surveillance has never been easier, when almost everything’s in the hands of Google, FaceBook, Twitter, etc. So, there are two issues that result in a gross information asymmetry: the triviality and low cost of mass surveillance, and the centralisation of our means to communicate. I2P is a solution to both of these.

Pseudonymity is about creating a profile and identity, but keeping it separate from our real world, identities. That separation could safeguard freedom of expression by enabling the sharing or challenging of controversial ideas without fear of recrimination. Thankfully this is already possible with I2P, Tor, (carefully selected) VPNs and a little understanding of how to sanitise browser traffic. With Tor and I2P, it is possible to have a domain, site and email address that are separate from the clear Web.

Starting I2P
Although the I2P software is more or less usable straight out of the box (at least for Windows), it works much better after waiting about 30 minutes for the client/server to build its list of peers.

When the I2P application is installed and run, it starts the default Web browser with the GUI loaded. This is where users can restart the proxy’s tunnels, view the network status and access some of the hidden services. Clicking the logo at the top-left will toggle the interface between the hidden services menu and the management console.

Next, the browser must be configured to route traffic through the local I2P proxy. Go to Internet Options or wherever the connection settings are for the browser, and point the proxy settings to 127.0.0.1 on port 4444. If you’ve got multiple proxies installed on a machine for different things, the FoxyProxy extension for Firefox provides an easy way of switching between them.

One of the weaknesses of Tor and I2P is they mask only the IP addresses of the clients, and not much else – this is why they won’t guarantee anonymity on their own. The payload might still include identifying information, and ideally you’d have something for stripping that identifying information before it leaves the local network. It should be possible to chain I2P and Privoxy, in the same way we might do for Tor, in order to strip potentially identifying information from browser traffic. It’s also possible to use Lynx, with the proxy addresses for HTTP and HTTPs configured in lynx.cfg.

As with the .onion services, not all the I2P services are available at a given moment. Some are offline and others take a while to reach.

Firewall Rules
Apparently firewall configuration isn’t necessary, but I added inbound and outbound rules anyway, just to see whether it made a difference. This can be done by opening the Windows Firewall GUI, and selecting ‘New Rule…’. Here we want to create a rule for a program (‘Rule that controls connections for a program’).

Find the I2Psvc.exe file in Windows Explorer (C:\Program Files\i2p\) and paste its location into the New Inbound Rule window.

Proceed with the default options and do the same for the outbound rule.

Email
Of course, an anonymous email account is a necessity, and we have two options: either register a clearweb account over a VPN connection or use an I2P email service. Here I’ve registered a test account with mail.i2p, which is administrated by Postman HQ (hq.postman.i2p).
The downside with this is that mail can only be routed through the I2P network, not from a clearweb mail account to the mail.i2p server or vice versa.

Personal Site
A hidden service on the I2P domain is referred to as an ‘eepsite’. Each user can create their own site/service using the local proxy as a Web Server. As I’ve mentioned, this is likely how social media profiles and groups would be hosted in a few decades.
The server directory is found at C:\Users\[user name]\AppData\Roaming\I2P\eepsite.

To put this online, click the Hidden Services Manager link under the I2P Internals section, and start the I2P Web server.

After the Web server is fully operational, which takes a few minutes, the local I2P address is found in the Hidden Services Manager, under Edit Server Settings. Changes to this address should eventually propagate through the I2P addressing system.

Like this:

It’s around this time of year that INFOSEC vendors and bloggers post their industry predictions for the year to come. Here I’ll tell of something I’m pretty confident will happen in roughly ten years.

OpenVPN is still working nicely, by the way, but the fact it’s become a necessity for getting decent Internet shows that I’m missing the boat in not following up my dissertation project with something tangible. Currently I haven’t gone much further than designing the communications protocol for this, creating a nice interface in VB.NET and outlining roughly how it should work in theory.
I revisited one of my sources, a research paper, ‘P2P social networking for the rest of US‘, authored back in 2009 by Zoe Antoniou, Sacha Zyto and Dimitris Kalofonus, who outlined a joint project between the Nokia Research Centre and MIT to develop a prototype Peer-to-Peer social network called MyNet. There are also two publicly-availablepapers (actually PDFs) for this: ‘MyNet: a Platform for Secure P2P Personal and Social Networking Services‘ and ‘A Hybrid P2P/Infrastructure Platform for Personal and Social Internet Services‘.

Whenever ‘Peer-to-Peer’ or P2P networking gets mentioned, the average person would think piracy, illegal file-sharing, BitTorrent and… whatnot. This isn’t really fair, as the term simply refers to a serverless, point-to-point, way of communicating. The Internet should actually now be almost entirely Peer-to-Peer. Except for the need to centrally manage stuff in enterprise networks, the main reason we still have a client-server Internet is because of technological limitations that used to exist – the IPv4 address space was badly assigned and exhausted some time ago, and personal computers didn’t always have the resources to host content for multiple users. The latter point was also made by the authors of the MyNet paper.
P2P is intrinsically a good thing, a more efficient, decentralised and secure way of communicating, and it’s where we need to go in order for the Internet to survive as a free and open communications medium.

At the core of the MyNet idea are the following concepts:
(1) – Users create Personal Networks of their devices, and these Personal Networks interlink to form Social Networks. This seems either a development of the Personal Area Network of the 1990s, or the Microsoft Active Directory feature (the name of which I can’t recall) that ties multiple devices to a given user.
(2) – One-to-many relationship, which I think refers to hosting content on a personal device.
(3) – Routing is based around the Personal and Social networks, but the researchers don’t explain how in the first journal paper.
(4) – Users share content by issuing ‘Passlets’ to each other. This would probably work in a similar way to ACLs and tokens in Microsoft Windows and Active Directory.
(5) – Social networks can be, or are, built offline, by using Near Field Communication to exchange details between users’ Personal Networks.

MyNet (MIT and Nokia Research Center, 2009)

So the advantages to this are obvious – users have total control over what they access and share, and they won’t have to give that control to a third-party if the ‘Passlet’ thing works as it should. It also uses out-of-band communications (NFC) for some of its authentication.

This idea looks very promising. In fact, I’m quite certain that something like this will emerge in maybe 10-5 years, and become another phenomenon just like FaceBook did. It might even become popular as the email client. Why? Because it’s the only way peoples’ need to share and openly communicate could be balanced with their need for privacy and security. We also know that encryption, Tor and VPNs – which almost nobody bothered with 15 years ago – are becoming mainstream for a reason. The client-server way of doing things is outdated.
If MyNet could be made to work at the application layer of a typical device, it would only take an initial handful of average Joes to install an ‘app’.

Usability, Convenience and Ubiquity
The paper’s abstract recognises one of the classic problems encountered when developing something like this:‘The user experience of current P2P Personal and Social networking systems does not meet the usability needs of the technically naïve users. This is the motivation behind MyNet, a P2P platform that enables non-expert users to easily organize their resources and share them in their immediate social neighborhood.’

As the authors pointed out, it’s already possible to configure an average home network to function like a MyNet Personal Network, but it’s unreasonable to expect the average person to do so. Most people are consumers, and this is the underlying reason why numerous privacy and anti-censorship technologies, most of them really innovative, failed to become mainstream.

So there we have ‘business requirements’ – we’re supposed to be developing solutions, not creating new problems. The solution must work with minimal effort, and using the resources on a typical PC, smartphone or laptop. As an example, OpenVPN works because it’s just a matter of installing an application and importing a text file, and it quickly solves a problem most people will encounter as more sites get blocked in the UK. Something like FreedomBox, on the other hand, is still proof-of-concept because most of us aren’t going to spend time and effort buying the hardware. These are requirements that should be central to the design.

Following that comes the design stage: what can developers re-use in creating a solution? This question was central to my dissertation, largely because my own engineering and programming skills are passable at best. We could do quite a lot with IPv6, and there are already various libraries available for IPsec, sockets, encryption and key exchange. Everything can hopefully be bundled and deployed anywhere.
As for MyNet, the developers made use of C++ and Python libraries, and apparently used Remote Procedure Calls for the underlying stuff.

Routing and Infrastructure
The real specifics of how the data would be routed between devices is addressed in ‘MyNet: a Platform for Secure P2P Personal and Social Networking Services‘.

This is where I’ll describe my own alternative for P2P comms, which was actually inspired by The Second Internet (Lawrence Hughes). It goes something like this: Every device could have its own IPv6 address. In fact, as there are something like 10e25 IPv6 addresses to every square centimetre of the Earth’s surface, there’s no technical reason why each device can’t have a considerably large block of IP addresses. We wouldn’t need NAT, so Peer-to-Peer IPsec tunnels should be possible between remote devices, between the endpoints. While that protects the confidentiality of communications, the channels still wouldn’t be secure unless they’re resilient against discovery, filtering and disruption. The solution: the IPv6 Layered Security Protocol that buries the data under several layers of encryption and tunnels it through a channel hidden from third-parties. As far as I’m aware, I’m the first to come up with a fairly solid (but still unrefined) idea for an IPv6 ‘darknet’ or ‘stealth network’, with Lawrence Hughes, Sam Bowne and lecturers at Newport university laying much of the groundwork.

What’s interesting is Arbor Networks cited a study that claimed P2P accounted for roughly 60% of IPv6 traffic in 2009, along with its own measurements that put the figure at 61% in late-2010 across North America and Europe. So it does look like P2P might be the killer application of IPv6, although the current activity could be attributed to a lack of firewall configuration for IPv6. The P2P OS blog has an interesting post describing how this might work during the transition to IPv6, if it’s going to be done in stages between now and 2030.

It’s been a while since I last posted an update, largely because the project’s been on hold for the last six weeks. Basically the second year of the course was mainly about theoretical stuff, like policies, compliance, management, legislation, etc., and the third year got very technical (and practical) from day one. And it’s not a bad thing either, as I expect any infosec professional to have at least some experience and a decent understanding of enterprise network and server configuration. So, that’s my excuse.

Roughly a month ago I had the basic secure messaging client application working, and hopefully I can get that communicating with the network. Later it can be modified for audio and video comms, and perhaps even a social network could be built around it someday.

Getting hold of the equipment for the development stage won’t be a problem, as I initially expected. I now have a carrier grade routing system at my disposal, which means the countermeasures can be tested with a collection of Cisco 2800 routers, an Adtran Atlas 550 Integrated Access Device (IAD), and TCP and IP filtering layers. The Adtran is what’s going to simulate the ISP and Internet.

Routing system, minus the cabling

By the end of January 2013, the whole thing should simulate multiple clients communicating between networks, tunneling their comms through whatever interception and filtering exists between them. It’ll be a form of P2P communication, but there’ll be nothing to mark it out to ISPs as such.

Before that happens, I’ll need to somehow configure the routing system, which must be done via serial ports and Telnet sessions, which is apparently quite easy.

Categories

Profile

My name is Michael, and I’m a software developer specialising in clinical systems integration and messaging (API creation, SQL Server, Windows Server, secure comms, HL7/DICOM messaging, Service Broker, etc.), using a toolkit based primarily around .NET and SQL Server, though my natural habitat is the Linux/UNIX command line interface.
Before that, I studied computer security (a lot of networking, operating system internals and reverse engineering) at the University of South Wales, and somehow managed to earn a Masters’ degree. My rackmount kit includes an old Dell Proliant, an HP ProCurve Layer 3 switch, two Cisco 2600s and a couple of UNIX systems.
Apart from all that, I’m a martial artist (Aikido and Aiki-jutsu), a practising Catholic, a prolific author of half-completed software, and a volunteer social worker.