Arch Linux User Repository

I confirm that the package building is broken now. And the decision to replace libraries with *-static is strange as well

Can't build with llvm7 too: fatal error: llvm/Support/VirtualFileSystem.h: No such file or directory

P.S. Understand - need clang-7 too.

However, llvm dependency must be replaced with llvm7 since it does not build with LLVM 8 (currrent) for some reason

After a week of attempts I happened to build it without arrow and lousy breaking the whole system [whatever]z[whatever]-static packages

# $Id: $
# Maintainer: Dmitry Bilunov <kmeaw@yandex-team.ru>

pkgname=ch
pkgver=19.4.0.49
pkgrel=1
pkgdesc='An open-source column-oriented database management system that allows generating analytical data reports in real time'
arch=('i686' 'x86_64')
url='https://clickhouse.yandex/'
license=('Apache')
depends=('poco' 'boost' 'jemalloc' 'xxhash' 'thrift' 're2' 'double-conversion' 'unixodbc' 'openssl' 'icu' 'llvm' 'gtest' 'gsasl' 'zstd' 'zlib' 'lz4' 'libxml2' 'brotli'  'lld' 'clang' 'libdaemon')
makedepends=('cmake' 'patchelf')
source=(
    https://github.com/yandex/ClickHouse/archive/v$pkgver-stable.tar.gz
    https://github.com/google/cctz/archive/4f9776a.tar.gz
    https://github.com/edenhill/librdkafka/archive/363dcad.tar.gz
    https://github.com/mfontanini/cppkafka/archive/860c90e.tar.gz
    https://github.com/aklomp/base64/archive/a27c565.tar.gz
    https://github.com/ClickHouse-Extras/libhdfs3/archive/bd6505c.tar.gz
    https://github.com/ClickHouse-Extras/protobuf/archive/1273537.tar.gz
    libunwind.patch
)
md5sums=(
    'eb7be28304b18fe3c5aaf5871bea598f'
    '5323f7ba2565a84a80a93edde95eb4fe'
    '858f1e9d37c064a1e5ce99f24ed9cf95'
    '40d91626049ab15bf3bbe75fa00bf52d'
    'e98c6b94f39d6947c83eb554aeb618e2'
    '2fa16f04d0d533cbb5797bfd45b058fd'
    'd63e8036385a27113365a40eb6bfe75e'
    'f3f60b75abf8d6f21de74db6e88e1e7b'
)
backup=('etc/clickhouse-client/config.xml' 'etc/clickhouse-server/config.xml' 'etc/clickhouse-server/users.xml')
install=$pkgname.install

prepare() {
  cd ClickHouse-$pkgver-stable
  sed -e 's/mysqlxx common\(.*\) \(\${Z_LIB}\)/mysqlxx \2 common\1/' -i libs/libmysqlxx/CMakeLists.txt

  sed -e 's/mysqlxx common\(.*\) \(\${Z_LIB}\)/mysqlxx \2 common\1/' -i libs/libmysqlxx/CMakeLists.txt
  patch -p1 < ../libunwind.patch
  mkdir -p contrib/cctz contrib/librdkafka contrib/cppkafka contrib/base64 contrib/libhdfs3 contrib/protobuf
  rm -rf contrib/{cctz,librdkafka,cppkafka,base64,libhdfs3,protobuf}/*
  mkdir -p contrib/boost/boost

  mv ../librdkafka-363dcad*/* contrib/librdkafka/
  mv ../cppkafka-860c90e*/* contrib/cppkafka/
  mv ../cctz-4f9776a*/* contrib/cctz/
  mv ../base64-a27c565*/* contrib/base64/
  mv ../libhdfs3-bd6505c*/* contrib/libhdfs3/
  mv ../protobuf-1273537*/* contrib/protobuf/
  for dir in contrib/*/; do
    rmdir $dir &> /dev/null || true
  done
  mkdir -p build
}

build() {
    cd ClickHouse-$pkgver-stable/build
    cmake -D CMAKE_BUILD_TYPE:STRING=Release -D USE_STATIC_LIBRARIES:BOOL=False -D ENABLE_TESTS:BOOL=False -D USE_INTERNAL_POCO_LIBRARY:BOOL=False -D USE_INTERNAL_BOOST_LIBRARY:BOOL=False -D ENABLE_JEMALLOC:BOOL=True -D USE_INTERNAL_JEMALLOC_LIBRARY:BOOL=False -D ENABLE_MYSQL:BOOL=True -D USE_INTERNAL_MYSQL_LIBRARY:BOOL=False -D USE_INTERNAL_THRIFT_LIBRARY:BOOL=False -D USE_INTERNAL_LIBUNWIND_LIBRARY:BOOL=False -D USE_INTERNAL_ZSTD_LIBRARY:BOOL=False -D USE_INTERNAL_LZ4_LIBRARY:BOOL=False -D USE_INTERNAL_RE2_LIBRARY:BOOL=False -D USE_INTERNAL_DOUBLE_CONVERSION_LIBRARY:BOOL=False -D DOUBLE_CONVERSION_ROOT_DIR=/usr -D USE_INTERNAL_UNIXODBC_LIBRARY:BOOL=False -D USE_INTERNAL_GTEST_LIBRARY:BOOL=False -D USE_INTERNAL_CAPNP_LIBRARY:BOOL=False -D USE_INTERNAL_ZLIB_LIBRARY:BOOL=False -D USE_INTERNAL_LLVM_LIBRARY:BOOL=False -D USE_INTERNAL_SSL_LIBRARY:BOOL=False -D USE_INTERNAL_LIBGSASL_LIBRARY:BOOL=False -D USE_INTERNAL_LIBXML2_LIBRARY:BOOL=False -D USE_INTERNAL_BROTLI_LIBRARY:BOOL=False -D USE_INTERNAL_PROTOBUF_LIBRARY:BOOL=True -DCMAKE_CXX_FLAGS=-D__GLIBCXX_USE_CXX11_ABI=0 -D SPLIT_SHARED_LIBRARIES:BOOL=True .. 

    cmake --build . --target clickhouse
}

package() {
    cd ClickHouse-$pkgver-stable/build
  mkdir -p $pkgdir/etc/clickhouse-server/ $pkgdir/etc/clickhouse-client/
  mkdir -p $pkgdir/usr/bin/
  mkdir -p $pkgdir/usr/lib/systemd/system
  cp dbms/programs/clickhouse $pkgdir/usr/bin/clickhouse-client
  cp ../dbms/programs/server/config.xml ../dbms/programs/server/users.xml $pkgdir/etc/clickhouse-server/
  ln -s /usr/bin/clickhouse-client $pkgdir/usr/bin/clickhouse-server
  patchelf --remove-rpath $pkgdir/usr/bin/clickhouse-client
  cp ../dbms/programs/client/clickhouse-client.xml $pkgdir/etc/clickhouse-client/config.xml
  for lib in libclickhouse{-{benchmark,{clien,forma,performance-tes}t,{co{mp{ile,resso},pie},obfuscato,serve}r,extract-from-config,local}-lib,{_{aggregat,tabl}e,}_functions,_{com{mon_{config,io,zookeeper},pression},dictionaries{,_embedded},parsers,storage{_kafka,s_system}}}.so; do
    libsrc=$(find dbms/ -name "$lib")
    libdst=$lib.$pkgver
    cp ${libsrc:?$lib not found} $pkgdir/usr/lib/$libdst
    patchelf --remove-rpath $pkgdir/usr/lib/$libdst
    patchelf --replace-needed $lib $libdst $pkgdir/usr/bin/clickhouse-client
  done
  for lib in lib{base64,c{ctz,ommon,ppkafka},daemon,{dbm,string_util}s,mysqlxx,pocoext}.so; do
    libsrc=$(find contrib/ libs/ dbms/ -name "$lib")
    libdst=libclickhouse-${lib#lib}.$pkgver
    cp ${libsrc:?$lib not found} $pkgdir/usr/lib/$libdst
    patchelf --remove-rpath $pkgdir/usr/lib/$libdst
    patchelf --replace-needed $lib $libdst $pkgdir/usr/bin/clickhouse-client
  done
  sed -e 's:/opt/clickhouse:/var/lib/clickhouse:g' -i $pkgdir/etc/clickhouse-server/config.xml
  sed -e '/listen_host/s%::<%::1<%' -i $pkgdir/etc/clickhouse-server/config.xml
  cp ../debian/clickhouse-server.service $pkgdir/usr/lib/systemd/system
}

It tries to build arrow package, which is very strange - replaces half of the system packages to aur versions of them (like zlib-statc) and doesn't even compile. Does this one really need it?

19.3.5-1 doesn't build:

tmp/tmp.JefkzNrImZ/clickhouse/clickhouse/src/ClickHouse-19.3.5-stable/dbms/programs/clang/Compiler-7.0.1/driver.cpp:15:10: fatal error: clang/Driver/Driver.h: No such file or directory
 #include "clang/Driver/Driver.h"

Also building with persistent ccache could noticeably speed up subsequent builds.

rbelio, you have several options:

1. Try to find out where your resources are wasted and utilize them better. It could be slow disk I/O, slow filesystem (check fs mount options), only one core used (check MAKEFLAGS in /etc/makepkg.conf). Sometimes installing "ninja" helps to build ClickHouse faster.

2. Build the package on a faster box, then transfer the binary. Or use some distributed tools, like distcc or dist-clang.

3. Do not apply full system updates too often to avoid frequent rebuilds. However this means taking risk of broken packages and introducing security vulnerabilities.

4. (Not recommended.) You can hack your binary to point to another library with patchelf --replace-needed. Anything could break after this hack but sometimes it would work. Maybe this is an acceptable use case for non-production environments.

kmeaw, thanks. I have another question, is there a way to mitigate the long compilation time everytime the libraries get updated? It takes me at least a couple of hours of compilation (over i7 dell laptop with ssd and 16gb). After reading more about arch I finally understood that partial update is not allowed and old libraries are always deleted when new libraries are available to the package manager.

rbelio, you cannot just link to a libicuuc.so instead of a libicuuc.so.62, because ".62" is there for a reason: the SONAME field value is "libiccuc.so.62", which means that anything else like "libiccuc.so.61" is not ABI-compatible with libiccuc.so.62. Everytime the SONAME is changed, you need to rebuild all the binaries depending on it. You can use something similar to Gentoo's revdep-rebuild to automate the process.