First working Ethereum C++ cross-builds

TL;DR

Ethereum C++ cross-builds were birthed into the world on Saturday 28th November 2015.   This is a really important milestone, because it opens the door to near-universal portability of Ethereum.

PiZero

A cross-built eth running on a Raspberry Pi Zero, courtesy of Martin Brook

As George was kind enough to say recently:

The work that @doublethink_co are doing on Ethereum is important and you should check it out! – George Hallam

George

I have time to read more details …

The Light Client (LES) sub-protocol is still being refined.   When we have working LES implementations the flood-gates will really open on the range of devices which can run Ethereum nodes.

Getting to Light Client on mobile/wearable is the ultimate goal of all the work which doublethinkco is doing, with the Samsung Gear S2 smartwatch as the flagship target device.

Zsolt Felfoldi gave a very interesting presentation on his Light Client progress at the recent devcon1 conference.   The YouTube video of his presentation is unavailable at the moment, but will likely be available again in an edited form soon.

LightClientDevcon

To my knowledge there is no C++ implementation of LES underway yet.  That was an intentional decision.  No doubt the C++ implementation will get underway as soon as the details of the sub-protocol are settled, and the go implementation has been given a decent workout.

Bob Summerwill and Anthony Cros have been working on Ethereum C++ cross builds since September, after the ETHDEV team came to the conclusion that the work-in-progress on Light Client needed to be rebooted as a sub-protocol, rather than as extensions to the existing ETH protocol.

That change of plan left us with no real way to contribute directly to Light Client development. We needed to get ourselves out of the way while that design work was brewing and to do something else which was useful and would help the ultimate end-goal.

So we took a detour, and focused on porting the existing C++ client code to as many mobile and wearable devices as we could, with a primary focus on Linux over Android and iOS.

02 hotel porter_std

Having the existing full client available in as many places as possible means that we can just “turn on” Light Client on numerous devices as-and-when the C++ implementation of the new sub-protocol is available.

Perhaps we can even help with that Light Client work directly ourselves?

That was our original intention, but we were just too early for that to be practical.   Maybe it makes sense now?

Our detour has gone very much as we expected. That earlier article also talks about what cross-builds are and why they are important.

We realized quite early that our needs were massively aligned with John Gerryts (EthEmbedded), who needed to cross-builds for the final phase of his Devgrant for embedded devices.   So we’ve been working closely together most of the time, and especially in the last week or so, as we have had binaries to share.

Martin Brook has just joined the party too.  He’s one of the main Sailfish OS porters, and his input and testing has been very useful.

Martin

So why is the C++ client so important?

We’ve been working with both the go and C++ development teams, but because of Péter Szilágyi‘s excellent work with xgo, there has been close to no need for us to do any work there.

peter

Why not just leave things to the go team?

It’s because C/C++ is the foundation of all systems programming and performance critical applications, and it has been that way since the very first UNIX releases, and will likely be that way for many years to come. C/C++ is always the first language on any new platform so code written in C/C++ has universal reach, all the way down to the very smallest of devices.

We are finally starting to get some innovation and competition in the systems programming space with go and rust, but it is exceedingly unlikely that those languages will ever make a significant dent in the wearable, embedded and IoT space.

With the renaissance of Modern C++, it is possible to write C++ which is maintainable and understandable as long as you are disciplined and stick to best practices.   It isn’t easy, and elements of C++ like the pre-processor, compilation model, lack of cross-platform build system and lack of cross-platform packaging system make C++ “not fun” to work with.  Bob has been doing it for nearly 20 years though.  Not a problem.

Doesn’t Moore’s Law mean that this kind of “close to the metal” performance doesn’t matter anymore?

Nope, because we’re running into Amdahl’s Law, which essentially states that the potential speedups for a given application are ultimately limited by the work which can only happen in a serial manner.   Having 16 or 32 cores doesn’t help if your application isn’t of a type which can benefit from massive parallelism.

In addition, raw performance is going to be critical for the resource-constrained devices which are going to be spawning in their billions in the next few years with the growth of IoT and smart devices.

 

Why the focus on mobile Linux, not Android?

Isn’t Android an open source platform, and shouldn’t that be the primary focus?

Android is really not as open as Google would like you to think (excellent article on that a few years back), and Google are obviously a nefarious data-farming corporation in general.   Replicant are trying to “free” Android, but it’s a losing battle.   All mobile devices have serious privacy and security issues.

Everybody already knows about Apple’s control-freak nature.  Ethereum may or may not ever be allowed into the App Store.   They have got form.  Don’t be surprised if they ban iOS and watchOS applications which use Ethereum.  It is a mortal threat to their monopolistic control of all commerce and applications on THEIR platform.   We’ll port to iOS and watchOS, but it might never be shippable.

Anyway, to get to the point, we’re nearly done. We’ve got our first running C++ code-compiled builds working now, and will extend that across the rest of the test matrix in the coming days and weeks (months? I hope not!)

So what have you built?

Dockerfiles which let you cross-build ARM binaries which are already known to be working on Raspberry Pi 2, Raspberry Pi Zero, Odroid XU3 and Wandboard Quad. We are working towards working binaries for a very significant range of devices.

docker

Everything is there for whoever wants to use it the
doublethinkco/cpp-ethereum-cross repo on Github, which includes the full test matrix in the README.

What is left to do?

“Just work”, namely …

… then Bob will get automated builds and automated testing running on the mobile/wearable home test farm he has been building.

20151120_083926

Following the go team’s lead, we are likely to start publishing cross-build binaries too, as soon as they are in a stable enough state for that to be worthwhile. It would be very useful to have matching cross-build binaries for the forthcoming eth-1.2.0 release, for example.

If you have further mobile/wearable/embedded devices which you think need some Ethereum love, please tweet us and we’ll see what we can do!

Bob Summerwill, 30th November 2015

Github repo README revamped

Check out our webthree-umbrella-cross Github repo.

Yesterday I spent some time improving that front-page so that the project is close to self-describing. We’re getting very close to having working binaries, and are attacking multiple platforms in parallel.   We have (currently broken) releases which anybody is welcome to use to help to with our debugging efforts.

We’re working with EthEmbedded to get this cross-build infrastructure working for single-board-computers as well as for our mobile/wearable target devices.    Join us in the Gitter porting channel if you would like to help out, or just follow along.

Our new-and-improved DOT dependency graph is in a Pull Request for merge back to the main ethereum/webthree-umbrella repo.

Jolla Phone has arrived at doublethinkco!

I know I’m a bit late to the party, and that it is November 2015, not November 2013, but my Jolla Phone has arrived in Vancouver!

 

 

I had to order the phone indirectly, for delivery to my parent’s house in the UK, and then re-mailed from there.   I have heard some whispers of 3G working for people in Canada, and will report back on whether I can get that going with my Rogers provider, or if I’m tied to 2G.

The company could use some good news, and I hope my purchase and continuing intention to bring Ethereum to Sailfish will raise a faint smile on some weary faces.   A familiar name popped up to help with that process yesterday, with the arrival of Martin Brook (vgrade) who is trying a Sailfish native C++ build in parallel with our ongoing C++ cross-build efforts.  Great to have him helping!

Ironically, getting the Sailfish cross-build working on the Jolla Phone is likely my quickest route to a Tizen Gear S2 version.   Because the whole platform is built for developers, not for dumb consumers!   No GDB on the Gear S2, for example.

Best wishes, Jolla and sailors!

Mobile/wearable test farm takes shape

Laura Kalbag from Ind.ie did me a huge favor recently when she pointed me to the superb DeviceLab from Vanamco.   As a background task I have been putting together a home-based mobile Linux test farm, which I will be using for automated testing of our Ethereum ports.

Before:

20151015_092228

After:

20151120_083926

And all of this is going to sit behind the pfSense box I built, when I finish the OS setup and cabling on that.    I might revisit my wireless router setup at the same time.

So what have I got racked up on those two boards right now?

Left hand board (mainly Tizen)

Form factor Device OS Version
Smartwatch Gear 2 Tizen 2.2.1
Smartwatch Gear S Tizen 2.3.0
Smartphone Samsung Z1 Tizen 2.3.0
Smartphone Samsung Z3 Tizen 2.4.0
Developer phone Samsung RD-PQ Tizen 2.3.0
Smartphone Nokia N9 MeeGo 1.2 (Harmattan)
Developer phone Nexus 4 Sailfish 1.??

Right hand board (a very random selection)

Form factor Device OS Version
Smartphone Nokia Lumia 920 Windows Phone 8.0
Smartphone Geeksphone Peak Firefox OS 1.0
Smartphone Blackberry Z10 BB10 10.0
Smartphone iPhone 3GS iOS 5.0
Smartphone iPhone 5 iOS 5.0
Smartwatch Samsung Gear Live Android Wear 4.4
Smartwatch Apple Watch Sport watchOS 1.0

And what else do I have which isn’t hooked up yet?

Form factor Device OS Version
Developer phone Samsung TM1 Tizen 2.4.0 Beta
Smartwatch Samsung Gear S2 Tizen 2.3.1
Smartphone Meizu MX4 Ubuntu Edition Ubuntu Touch ??.??
Smartphone Jolla Phone Sailfish 2.0
Tablet Jolla Tablet Sailfish 2.0
Tablet Nexus 7 Android  ?
Tablet Samsung Tab S Android  ?
Tablet iPad 3 iOS ?
SBC Raspberry Pi Raspbian ?
SBC Odroid-XU4 Tizen 3.0
SBC Intel NUC Tizen 3.0

And completely unrelated to this effort, I also have my Commodore 64 and Atari VCS to get connected!

 

 

We got on screen at #devcon1 on the final day

Many thanks to John Gerryts for crediting Anthony and myself as contributing to EthEmbedded project, as shown on the “DEVgrants Update” slides at the Ethereum devcon1 conference today.

We got on-screen!   Somewhat over-credited as part of the EthEmbedded team, but we’ll take it 🙂

DoubleThinkOnScreen

It was fantastic to hear of the additional $100K funding from Omise.   We will be applying for a grant for our cross-compilation and porting work 🙂

Porting Ethereum – geth running on Samsung Gear S2 smartwatch. eth coming soon

Back in early June of this year I made a build of ethereumj-android on my Mac, ran it on my Samsung Galaxy S6 and made a video which was pretty popular on the Ethereum sub-reddit at the time:

That was just my first step on a journey towards a Light Client for the Samsung Gear S2 smartwatch (and various other mobile/wearable Linux devices).

Samsung-Gear-S2-AH-faces-1-1600x1067

That aim has come closer to fruition this week, with a working (full node) geth on device, and getting an eth executable on device.   We’re still working through runtime issues on the eth executable, but it’s very close now.

Nearly all of the credit for geth porting lies with Péter Szilágyi and his majestic xgo CGO cross-compilation solution.   We really didn’t have any work to do on geth itself.   There is still a sporadic crash to chase, but it’s workable.

peter

The block-chain download is slow on the smartwatch, but the main issue is storage space, with only 4Gb of on-device storage.   The Morden test-network fits, but the Frontier main network is already too big.   There is no big surprise here.   The need for a Light Client is the motivator this whole project, but getting the full client working on device is a good step in that direction, because the infrastructure is now ready and waiting for the Light Client protocol to take shape.   Geth is particularly important to get working, because it is the Ethereum client which will get Light Client support first.

While it is great to see a working client on-device, the larger progress which we’ve made which is on the cusp of completion is adding cross-compilation support for eth, the C++ client.   Having C++ cross-compilation is critical for resource-constrained platforms which cannot support compilation on-device.   The performance and universal reach of C++ will also be very important in bringing Ethereum to the very smallest IoT and embedded platforms.

Anthony Cros is sub-contracting for doublethinkco, and has done 90%+ of the work.  The C++ side has been quite the battle!    I’d just like to publicly thank Anthony for all of the blood, sweat and tears he has poured into this work over the last couple of months.   I know he will be very happy and relieved when we get eth running 🙂

anthony-cros

Tizen has been a test-case for us, because it doesn’t support native compilation even on mobile, let alone wearable.   We tried to get that working, but it was a dead-end.   So then we flipped to cross-builds, first on the TM1 developer phone, and then just in the last days on the Gear S2 itself.

samsung-gear-s2-press-169

The cross-compilation support consists of:

So it looks like this porting tangent has gone much as I expected it might when we started on it just under two months ago.   We will continue to work with EthEmbedded and SlockIt to ensure that the C++ cross-builds are working for all platforms of interest, and to get the cross-build support upstreamed.   We’ve not started on Sailfish OS and Ubuntu Phone ports, but they will be very similar to Tizen, and may even “just work”.

After that, I hope that we can make some more direct contributions towards the Light Client implementations, starting with geth as Zsolt described at devcon1 yesterday:

Screen Shot 2015-11-13 at 12.20.58 AM

When those come to fruition we can all go and build some cool DAPPS for smartwatches and the 50 billion smart-devices which allegedly will be making their way into the world by 2020.

For months, I have been trying to push in the direction which SlockIt teased and delighted us all with a devcon1 today – bringing block-chain technology to small Internet-connected devices:

Wearables, IoT, smart-devices, embedded systems.   We are heading into a very connected future.   I want that future to be decentralized and under our control.

Hold on to your hats!   We’ll be there soon 🙂

Porting Ethereum – Tizen cross-builds. Update by Anthony Cros, 14th October 2015

NOTE:   You can follow along with our day-to-day progress on the Ethereum Porting Gitter channel.


Ethereum_logo_bwlogo-Tizen-Pinwheelanthony-cros

Some progress! On both fronts: go-ethereum and cpp-ethereum.

Our efforts lately have been focused on cross-compiling these two projects targeting an ARM architecture.

go-ethereum

The easier one first. As Bob mentioned in his latest post, Péter Szilágyi from the go-ethereum team has created the great xgo tool, that allows cross-compiling go-ethereum to different platforms (though it is not limited to cross-compilation for Ethereum, it is a general tool).

Using xgo and the instructions from its README file, I was able to generate a binary for geth quite easily and without any change (xgo notably leverages the power of Docker).   I was then able to run this binary on a Samsung TM1 developer phone (with Tizen O.S.), a process I will detail in an upcoming post. The binary worked well up until a point during block synchronization (~block 90K) where it failed, though nothing indicated it was an issue with the platform itself (more likely the particular version of geth I had used for compilation). I will rebuild and report on the result soon!

tony:~/tizen-sdk/tools[130] $ ./sdb push ~/.tony/program/go/go1.4.2/bin/geth-linux-arm /home/developer
pushed geth-linux-arm 100% 19MB
1 file(s) pushed. 0 file(s) skipped.
/home/tony/.tony/program/go/go1.4.2/bin/geth-linux-arm 4142 KB/s (20639213 bytes in 4.865s)
tony:~/tizen-sdk/tools[0] $ ./sdb shell
sh-4.1$ cd /home/developer/
sh-4.1$ ls
geth-linux-arm testupload.txt
sh-4.1$ ls -l
total 20160
-rwxr-xrwx 1 developer developer 20639213 Sep 29 18:49 geth-linux-arm
-rwxrwxrwx 1 developer developer 0 Jul 3 16:55 testupload.txt
sh-4.1$ ./geth-linux-arm
=======================================
Disclaimer of Liabilites and Warranties
=======================================

THE USER EXPRESSLY KNOWS AND AGREES THAT THE USER IS USING THE ETHEREUM PLATFORM AT THE USER’S SOLE
RISK. THE USER REPRESENTS THAT THE USER HAS AN ADEQUATE UNDERSTANDING OF THE RISKS, USAGE AND
INTRICACIES OF CRYPTOGRAPHIC TOKENS AND BLOCKCHAIN-BASED OPEN SOURCE SOFTWARE, ETH PLATFORM AND ETH.
THE USER ACKNOWLEDGES AND AGREES THAT, TO THE FULLEST EXTENT PERMITTED BY ANY APPLICABLE LAW, THE
DISCLAIMERS OF LIABILITY CONTAINED HEREIN APPLY TO ANY AND ALL DAMAGES OR INJURY WHATSOEVER CAUSED
BY OR RELATED TO RISKS OF, USE OF, OR INABILITY TO USE, ETH OR THE ETHEREUM PLATFORM UNDER ANY CAUSE
OR ACTION WHATSOEVER OF ANY KIND IN ANY JURISDICTION, INCLUDING, WITHOUT LIMITATION, ACTIONS FOR
BREACH OF WARRANTY, BREACH OF CONTRACT OR TORT (INCLUDING NEGLIGENCE) AND THAT NEITHER STIFTUNG
ETHEREUM NOR ETHEREUM TEAM SHALL BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY OR
CONSEQUENTIAL DAMAGES, INCLUDING FOR LOSS OF PROFITS, GOODWILL OR DATA. SOME JURISDICTIONS DO NOT
ALLOW THE EXCLUSION OF CERTAIN WARRANTIES OR THE LIMITATION OR EXCLUSION OF LIABILITY FOR CERTAIN
TYPES OF DAMAGES. THEREFORE, SOME OF THE ABOVE LIMITATIONS IN THIS SECTION MAY NOT APPLY TO A USER.
IN PARTICULAR, NOTHING IN THESE TERMS SHALL AFFECT THE STATUTORY RIGHTS OF ANY USER OR EXCLUDE
INJURY ARISING FROM ANY WILLFUL MISCONDUCT OR FRAUD OF STIFTUNG ETHEREUM.

Do you accept this agreement? [y/N] y

I1004 23:21:26.410705 9630 flags.go:414] WARNING: No etherbase set and no accounts found as default
I1004 23:21:26.411956 9630 database.go:71] Alloted 16MB cache to /home/developer/.ethereum/chaindata
I1004 23:21:26.454467 9630 database.go:71] Alloted 16MB cache to /home/developer/.ethereum/dapp
I1004 23:21:26.475128 9630 backend.go:291] Protocol Versions: [62 61], Network Id: 1
I1004 23:21:26.476165 9630 backend.go:333] Blockchain DB Version: 3
I1004 23:21:26.480377 9630 chain_manager.go:248] Last block (#0) d4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3 TD=17179869184
I1004 23:21:26.546630 9630 cmd.go:125] Starting Geth/v1.2.0/linux/go1.5.1
I1004 23:21:26.547729 9630 server.go:311] Starting Server
I1004 23:21:26.568572 9630 udp.go:207] Listening, enode://95b46a352806ebd5e97ec6bdcdcb9ea425eed48c95f78963df264c9985cba1aa72c658482951025322a8db49b88ae3eeb2314a0995df8d636de0276259bdaa5b@108.162.185.124:30303
I1004 23:21:26.569580 9630 backend.go:559] Server started
I1004 23:21:26.570892 9630 ipc_unix.go:109] IPC service started (/home/developer/.ethereum/geth.ipc)
I1004 23:21:26.571563 9630 server.go:552] Listening on [::]:30303
I1004 23:21:26.574707 9630 nat.go:111] mapped network port udp:30303 -> 30303 (ethereum discovery) using NAT-PMP(192.168.1.1)
I1004 23:21:26.588775 9630 nat.go:111] mapped network port tcp:30303 -> 30303 (ethereum p2p) using NAT-PMP(192.168.1.1)
I1004 23:21:36.715515 9630 downloader.go:275] Block synchronisation started
meanwhile:
sh-4.1$ pwd
/home/developer/.ethereum
sh-4.1$ find
.
./nodes
./nodes/000001.log
./nodes/LOCK
./nodes/LOG
./nodes/MANIFEST-000000
./nodes/CURRENT
./geth.ipc
./nodekey
./chaindata
./chaindata/LOG.old
./chaindata/MANIFEST-000008
./chaindata/LOCK
./chaindata/000010.ldb
./chaindata/LOG
./chaindata/CURRENT
./chaindata/000007.log
./dapp
./dapp/000002.log
./dapp/MANIFEST-000003
./dapp/LOG.old
./dapp/LOCK
./dapp/LOG
./dapp/CURRENT
sh-4.1$ find -type f | tr "\n" "\0" | du -cs --files0-from=-
1392 ./nodes/000001.log
4 ./nodes/LOCK
8 ./nodes/LOG
8 ./nodes/MANIFEST-000000
8 ./nodes/CURRENT
8 ./nodekey
8 ./chaindata/LOG.old
8 ./chaindata/MANIFEST-000008
4 ./chaindata/LOCK
1892 ./chaindata/000010.ldb
8 ./chaindata/LOG
8 ./chaindata/CURRENT
892 ./chaindata/000007.log
4 ./dapp/000002.log
8 ./dapp/MANIFEST-000003
8 ./dapp/LOG.old
4 ./dapp/LOCK
8 ./dapp/LOG
8 ./dapp/CURRENT
4288 total

cpp-ethereum

Much more manual work here, but after much struggle I was able to successfully cross-compile the libweb3core library. I created a set of scripts that should allow anyone to reproduce this process (in fact please do and report on any issue encountered!).
For it to work, I had to use a number of hacks along the way however:

  1. code – Hardcoding Boost‘s project-config.jam file to use my gcc cross-compiler (I tried many different flags/environment variables for bootstrap.sh but none ever worked/were picked up)
  2. code – Commenting out the CXXFLAGS-march=native flags from CryptoPP‘s Makefile. Likewise I found no cleaner way to unset this flag (which you must for it to succeed).
  3. code – I was never able to get the ARCHIVE_INSTALLS_DIR flag to get picked up during configuration for JsonCpp, instead having to rely on DESTDIR during the installation phase (not necessarily a big issue, but still frustrating)
  4. I ended up having to create a ~/$HOME/cryptopp symlink to CryptoPP’s installation directory for libweb3core‘s CMake file to be able to detect CryptoPP’s headers, no other combinations ever seemed to work. This on top of making CMAKE_INCLUDE_PATH point to the source directory. Removing either results in a failure to detect said headers (note that no such issue occured for the detection of the library itself).
  5. code – Disabling of MiniUPNP, which has apparently broken backward compatibility in its latest version and for which I wasn’t able to find a working version (typically because of failure to include upnp-dev.h, which would have also required hacking their CMake file).
  6. code – Removal of erroneous -lOFF artifacts from the CMake configuration phase of libweb3core, a bug I have brought to the attention of the cpp-ethereum team.

If anyone is aware of workarounds for these or can even suggest a more straighforward/correct way of getting to the same result, please contact me on gitter.

So overall it’s quite a lot of pieces that need to work together in order to create such a binary. Next step will be building other necessary components for eth to run. After which I will hopefully be able to test the C++ client on the same Z3 phone!

Stay tuned for more progress!