Voodooalert+x264 Logo
US+UK Flagge DE+AT Flagge
Change language:
 

x264 benchmark compilation & installation guide for Haiku

Haiku Logo

System prerequisites

To be able to compile and install the x264 benchmark on Haiku, a set of basic software needs to be present on the system. That includes:


At the time of writing of this article, installation on Haiku release 3 is impossible, because that release version has only GCC 2.95 to offer, which is ancient. To make it work, you will need to use a development or "nightly" build of Haiku with GCC4 included, or a newer version if available. You can either choose to use a [hybrid image with GCC4] as the primary compiler and GCC2 as the secondary one, or you may want to try a [pure GCC4 image], which will however not support any legacy Haiku software compiled with the older GCC2. For convenience, besides the installation CD, the Haiku developers also have some VM images to offer, which can be useful for testing purposes.

First things first, we need to build the yasm assembler, without that piece of software we would need to forget about most assembler optimizations, so we really want to make yasm work. First please download the assembler:

Before we can start compiling something, we need to ensure that the correct, newer compiler will be used. To make sure that's the case, run the following command:

After that, just unpack the archive like this: tar -xzvf ./yasm-1.2.0.tar.gz and edit the file config/config.guess afterwards, you may use the built-in Haiku editor "Pe" or "StyledEdit" for that. Please look for the following text in that file:

BePC:BeOS:*:*)
  echo i586-pc-beos
  exit 0 ;;

After that spot please add the following code to allow the configure script to properly correlate the platform Haiku to the base OS type BeOS:

BePC:*Haiku*:*:*)
  echo i586-pc-beos
  exit 0 ;;

Now that that's done, the configure script should run properly, so let's fire it up, and let's also compile and install yasm after that:

Now the assembler should be available which can be verified by running yasm --version. Now we may continue with libav and x264.

The benchmarking software itself

To be able to use x264, we will need the decoders of libavcodec (in short libav) and of course x264 itself, which will be linked to libav during compilation. Additionally of course we will need the actual benchmark input video, download mirrors for that one are provided at the bottom of the x264 [benchmark results list]. If desired, you may also use ffmpeg/ffms as a replacement for libav.

For simplicities sake we will assume, that the files are now in /boot/home/x264src/. Additionally we will assume that the yasm assembler is readily available. On the terminal we change into that directory and begin with libav. Please unpack the archive: tar -xzvf <filename.tgz>, after which you will find a new libav folder. Change to that folder to begin with the necessary modifications and configurations:

The first thing we need to do is to set the proper environment variables. It is of utmost importance to really know your processor and its features to not make any mistakes here and even more so later in this guide. In our case I will assume that what we've got is an Intel Core 2 processor with SSSE3 as its latest instruction extension. So our hypothetical processor features MMX, SSE, SSE2, SSE3 and SSSE3. That results in the following options for our C/C++ compiler (documentation about stuff like the processor marches is available [here]):

Additionally to that we will also need to set some Haiku-specific linker flags, since some of the operating systems functions are not hidden within libraries that configure knows from Linux or other Unix systems. So some required functions reside in libnetwork and the entire posix thread functionality that we need to support multiple CPU cores and threads is implemented in libroot instead of libm or libpthread. To fix that, we set the following linker flags:

Please note that you should not terminate your session or close your terminal window after setting CFLAGS, CXXFLAGS and LDFLAGS. If you do so, you will need to set them again after opening a new terminal window!

Now let's start the libav configure script, targetting our hypothetical Intel Core 2 processor:
As you can see here, we're giving one hell of a lot of options to the script, not like e.g. in Linux. The reason for this is the broken CPU detection of configure on Haiku, which will mostly not detect any or entirely wrong instruction extensions. So we manually disable all extensions that our hypothetical processor cannot support. At the time of writing, libav can support the following instruction extensions, that we need to enable and disable manually here, according to the processor at hand (Attention! Virtual machine hypervisors may also affect available instruction extensions!):

So you really need to pay attention here, enable only the correct instruction extensions and disable all the rest! Unfortunately, even if you do this properly, configure might still fuck everything up, even if you tell it what to do exactly. To make sure nothing bad happened, you should manually check some of the newly created files after configure but before starting to compile. Potentially wrong instruction extension configurations would need adjustments in the following files:
The best thing would be to search for keywords like "MMX" or "AVX" and so on through all the three files. Those keywords will pop up in multiple locations for each file, because EXTERNAL and INLINE assembler are configured sperately, and there will also be a global setting for each flag in each file. So you should be careful to catch them all, if you miss just a single one, you will need to do all the configuration and compilation again! If you miss AMD 3DNow! for instance, and build that assembler code on an Intel Core 2, this will never work, as the processor cannot speak 3DNow! plain and simple.

There is also one more modification that you will need to apply in config.mak, here we shall change an error behaviour to warning to make the code compile, so we will allow implicit function declarations. To do just that, look for the following string in config.mak, and remove it:
Since the build scripts will look for the GNU coreutil "env" in a hardcoded path, we will have to make sure that it can be found where the scripts are looking for it:

Now we can finally compile:

With libav set up we can begin to build and link x264 itself. Please unpack the x264 archive: tar -xjvf last_x264.tar.bz2 and change into the newly created x264 subdirectory. This might also be a good time to recheck whether our CFLAGS/CXXFLAGS and LDFLAGS are still set (export | grep -i flags should give us a good idea about whether this is the case).

Before we can go on, we need to prepare the x264 configure script for Haiku and ensure, that a clean link to libav can be established. To achieve all that, several things need to be done:

We need to make sure that x264 will be using posix threads, just like libav before it. If x264 does BeOS threads instead, the mixture can never work. Also, we need to set some additional linker flags that would otherwise be ignored. Please look for the following part in the configure script:

case $host_os in
  beos*)
    SYS="BEOS"
    define HAVE_MALLOC_H
    ;;

Right after that part of the code, please add the following to ensure proper detection of the Haiku OS and to set said additional linker flags, so that x264 can find all operating system functions that it will need:

  *haiku*)
    SYS="HAIKU"
    define HAVE_MALLOC_H
    LDFLAGS="$LDFLAGS -lroot -lnetwork"
    ;;

With that we're however not done just yet, linking x264 correctly is all nice, but we will still need to tell configure which threading model there is to be found in "libroot". Please look for the following spot in the code:

case $SYS in
  BEOS)
    thread="beos"
    define HAVE_BEOSTHREAD
    ;;

Right after that, please add the following:

  HAIKU)
    thread="posix"
    libpthread="-lroot"
    ;;

Now that we're linking against libroot where we will find our precious threading functions, we should remove an additional check for libm, which is usually responsible for that stuff. Look for the following string:

for lib in -lpostproc -lavcodec -lavcore -lswscale --lavutil -lm -lz -lbz2 $libpthread -lavifil32; do

All you need to do here is to replace -lm with -lroot! That finalizes the modifications in the configure script. But before you cheer, we're still not done yet.

Usually, x264 is trying to use a Linux/Unix function called nice() to lower its thread priorities at runtime - a function that does not exist on Haiku! So we need to do something about that, otherwise the source code will not compile because of that missing function. Since the priority lowering is not critical, not even really important we can just remove that part from the x264 source code with no big side effects. So please open the file common/osdeps.h and look for the following spot in that header source:

define x264_lower_thread_priority(p) { UNUSED int nice_ret = nice(p); }

Just replace that whole line with:

define x264_lower_thread_priority(p)

By doing that, x264 will cease to try and lower its own threads priorities and we no longer depend on functions which are not present on Haiku. Enough with all the modifications already, we should be all set for configuration, compilation, linking and installation:

Before we can really start the compilation, we need to make sure that the linker can actually find libav, since the configure script will not re-use the prefix for the linkers search path. For that, we edit config.mak, and look for the following code far at the bottom:

LDFLAGSCLI = -L. -lavformat -lavcodec -lswscale -lavutil -lroot -z -lbz2 -lroot -lswscale -lavutil

We add the new linker search path as follows:

LDFLAGSCLI = -L. -L/boot/common/lib -lavformat -lavcodec -lswscale -lavutil -lroot -z -lbz2 -lroot -lswscale -lavutil

Now compilation can start:

After doing all that, hopefully without major fuckups, you can check x264s functionality by running x264 --version. If that works and if x264 shows that it's linked against libswscale and libavformat, then everything worked fine, unless you accidentally built some assembler for an instruction extension your CPU doesn't have without noticing. Otherwise, you may now download the benchmark script:

That file launchbenchmark.sh should be copied into a dedicated benchmarking subfolder that we can just create if there isn't one yet: mkdir -p /boot/home/x264benchmark. Besides the script you should also have the input video stream elephantsdream_source.264 in that folder.

Now unfortunately x264 will use a way of CPU core detection that might very well work fine on Linux and Unix, but will fail on BeOS and with it on Haiku. That results in x264 having strictly no idea how many logical processors there are in the system. When that happens, x264 will only launch a single worker thread, and hence will only load one logical processor core, even if we have multithreading support on board. Luckily, there is an easy fix for this, we just need to add a little something to our launchbenchmark.sh, telling x264 how many worker threads it's supposed to spawn. For that, we should follow the old x264 rule:

So, for a CPU with 4 cores, that would make 6 threads. For a CPU with 4 cores and hyper threading (=8 logical CPUs) that would make 12 threads. If by any chance you have a triple core that simple old formula won't work of course, but in that case just round up to 5 threads. The maximum number of workers is by the way 32, at least at the time of writing, so you can't spawn more worker threads than that. So to make that little change to the benchmark, open up launchbenchmark.sh with StyledEdit, and add the option after both occurences of --cqm flat:

Here, n is the amount of worker threads, so e.g. --threads 12 for a system with 8 logical CPUs. Now with that, nothing should stand in the way of using multiple CPUs, cores and hyper threads anymore!

So now you can basically just launch the benchmark, so just enter the following line after ensuring that the script is executable (chmod +x ./launchbenchmark.sh) and while of course sitting in /boot/home/x264benchmark:

That could look somewhat like this (click to enlarge):

Haiku Screenshot

After completion, the result can be seen right next to the "Real" string.

Depending on the host machine that might take some hours, in extreme cases even days, weeks or months. If you have problems with this guide or if there are any additional questions about the x264 benchmark on Haiku, please refer to the [according forum thread] and ask there. The forum is german, but if you ask in english, you will still get a reply, at least by me. :)

[Back to the x264 benchmark results list]

[Comment on this guide]