Arch Linux User Repository

After the long discussion in dev-group of Telegram, the most straight forward way to use clickhouse is the usage of the official build.

After the long discussion in dev-group of Telegram, the most straight forward way to use clickhouse is the usage of the official build.

curl: (22) The requested URL returned error: 404 Not Found
==> ERROR: Failure while downloading https://github.com/apache/arrow/archive/87ac6fd.tar.gz
    Aborting...
Error downloading sources: clickhouse

I confirm that the package building is broken now. And the decision to replace libraries with *-static is strange as well

Can't build with llvm7 too: fatal error: llvm/Support/VirtualFileSystem.h: No such file or directory

P.S. Understand - need clang-7 too.

However, llvm dependency must be replaced with llvm7 since it does not build with LLVM 8 (currrent) for some reason

After a week of attempts I happened to build it without arrow and lousy breaking the whole system [whatever]z[whatever]-static packages

# $Id: $
# Maintainer: Dmitry Bilunov <kmeaw@yandex-team.ru>

pkgname=ch
pkgver=19.4.0.49
pkgrel=1
pkgdesc='An open-source column-oriented database management system that allows generating analytical data reports in real time'
arch=('i686' 'x86_64')
url='https://clickhouse.yandex/'
license=('Apache')
depends=('poco' 'boost' 'jemalloc' 'xxhash' 'thrift' 're2' 'double-conversion' 'unixodbc' 'openssl' 'icu' 'llvm' 'gtest' 'gsasl' 'zstd' 'zlib' 'lz4' 'libxml2' 'brotli'  'lld' 'clang' 'libdaemon')
makedepends=('cmake' 'patchelf')
source=(
    https://github.com/yandex/ClickHouse/archive/v$pkgver-stable.tar.gz
    https://github.com/google/cctz/archive/4f9776a.tar.gz
    https://github.com/edenhill/librdkafka/archive/363dcad.tar.gz
    https://github.com/mfontanini/cppkafka/archive/860c90e.tar.gz
    https://github.com/aklomp/base64/archive/a27c565.tar.gz
    https://github.com/ClickHouse-Extras/libhdfs3/archive/bd6505c.tar.gz
    https://github.com/ClickHouse-Extras/protobuf/archive/1273537.tar.gz
    libunwind.patch
)
md5sums=(
    'eb7be28304b18fe3c5aaf5871bea598f'
    '5323f7ba2565a84a80a93edde95eb4fe'
    '858f1e9d37c064a1e5ce99f24ed9cf95'
    '40d91626049ab15bf3bbe75fa00bf52d'
    'e98c6b94f39d6947c83eb554aeb618e2'
    '2fa16f04d0d533cbb5797bfd45b058fd'
    'd63e8036385a27113365a40eb6bfe75e'
    'f3f60b75abf8d6f21de74db6e88e1e7b'
)
backup=('etc/clickhouse-client/config.xml' 'etc/clickhouse-server/config.xml' 'etc/clickhouse-server/users.xml')
install=$pkgname.install

prepare() {
  cd ClickHouse-$pkgver-stable
  sed -e 's/mysqlxx common\(.*\) \(\${Z_LIB}\)/mysqlxx \2 common\1/' -i libs/libmysqlxx/CMakeLists.txt

  sed -e 's/mysqlxx common\(.*\) \(\${Z_LIB}\)/mysqlxx \2 common\1/' -i libs/libmysqlxx/CMakeLists.txt
  patch -p1 < ../libunwind.patch
  mkdir -p contrib/cctz contrib/librdkafka contrib/cppkafka contrib/base64 contrib/libhdfs3 contrib/protobuf
  rm -rf contrib/{cctz,librdkafka,cppkafka,base64,libhdfs3,protobuf}/*
  mkdir -p contrib/boost/boost

  mv ../librdkafka-363dcad*/* contrib/librdkafka/
  mv ../cppkafka-860c90e*/* contrib/cppkafka/
  mv ../cctz-4f9776a*/* contrib/cctz/
  mv ../base64-a27c565*/* contrib/base64/
  mv ../libhdfs3-bd6505c*/* contrib/libhdfs3/
  mv ../protobuf-1273537*/* contrib/protobuf/
  for dir in contrib/*/; do
    rmdir $dir &> /dev/null || true
  done
  mkdir -p build
}

build() {
    cd ClickHouse-$pkgver-stable/build
    cmake -D CMAKE_BUILD_TYPE:STRING=Release -D USE_STATIC_LIBRARIES:BOOL=False -D ENABLE_TESTS:BOOL=False -D USE_INTERNAL_POCO_LIBRARY:BOOL=False -D USE_INTERNAL_BOOST_LIBRARY:BOOL=False -D ENABLE_JEMALLOC:BOOL=True -D USE_INTERNAL_JEMALLOC_LIBRARY:BOOL=False -D ENABLE_MYSQL:BOOL=True -D USE_INTERNAL_MYSQL_LIBRARY:BOOL=False -D USE_INTERNAL_THRIFT_LIBRARY:BOOL=False -D USE_INTERNAL_LIBUNWIND_LIBRARY:BOOL=False -D USE_INTERNAL_ZSTD_LIBRARY:BOOL=False -D USE_INTERNAL_LZ4_LIBRARY:BOOL=False -D USE_INTERNAL_RE2_LIBRARY:BOOL=False -D USE_INTERNAL_DOUBLE_CONVERSION_LIBRARY:BOOL=False -D DOUBLE_CONVERSION_ROOT_DIR=/usr -D USE_INTERNAL_UNIXODBC_LIBRARY:BOOL=False -D USE_INTERNAL_GTEST_LIBRARY:BOOL=False -D USE_INTERNAL_CAPNP_LIBRARY:BOOL=False -D USE_INTERNAL_ZLIB_LIBRARY:BOOL=False -D USE_INTERNAL_LLVM_LIBRARY:BOOL=False -D USE_INTERNAL_SSL_LIBRARY:BOOL=False -D USE_INTERNAL_LIBGSASL_LIBRARY:BOOL=False -D USE_INTERNAL_LIBXML2_LIBRARY:BOOL=False -D USE_INTERNAL_BROTLI_LIBRARY:BOOL=False -D USE_INTERNAL_PROTOBUF_LIBRARY:BOOL=True -DCMAKE_CXX_FLAGS=-D__GLIBCXX_USE_CXX11_ABI=0 -D SPLIT_SHARED_LIBRARIES:BOOL=True .. 

    cmake --build . --target clickhouse
}

package() {
    cd ClickHouse-$pkgver-stable/build
  mkdir -p $pkgdir/etc/clickhouse-server/ $pkgdir/etc/clickhouse-client/
  mkdir -p $pkgdir/usr/bin/
  mkdir -p $pkgdir/usr/lib/systemd/system
  cp dbms/programs/clickhouse $pkgdir/usr/bin/clickhouse-client
  cp ../dbms/programs/server/config.xml ../dbms/programs/server/users.xml $pkgdir/etc/clickhouse-server/
  ln -s /usr/bin/clickhouse-client $pkgdir/usr/bin/clickhouse-server
  patchelf --remove-rpath $pkgdir/usr/bin/clickhouse-client
  cp ../dbms/programs/client/clickhouse-client.xml $pkgdir/etc/clickhouse-client/config.xml
  for lib in libclickhouse{-{benchmark,{clien,forma,performance-tes}t,{co{mp{ile,resso},pie},obfuscato,serve}r,extract-from-config,local}-lib,{_{aggregat,tabl}e,}_functions,_{com{mon_{config,io,zookeeper},pression},dictionaries{,_embedded},parsers,storage{_kafka,s_system}}}.so; do
    libsrc=$(find dbms/ -name "$lib")
    libdst=$lib.$pkgver
    cp ${libsrc:?$lib not found} $pkgdir/usr/lib/$libdst
    patchelf --remove-rpath $pkgdir/usr/lib/$libdst
    patchelf --replace-needed $lib $libdst $pkgdir/usr/bin/clickhouse-client
  done
  for lib in lib{base64,c{ctz,ommon,ppkafka},daemon,{dbm,string_util}s,mysqlxx,pocoext}.so; do
    libsrc=$(find contrib/ libs/ dbms/ -name "$lib")
    libdst=libclickhouse-${lib#lib}.$pkgver
    cp ${libsrc:?$lib not found} $pkgdir/usr/lib/$libdst
    patchelf --remove-rpath $pkgdir/usr/lib/$libdst
    patchelf --replace-needed $lib $libdst $pkgdir/usr/bin/clickhouse-client
  done
  sed -e 's:/opt/clickhouse:/var/lib/clickhouse:g' -i $pkgdir/etc/clickhouse-server/config.xml
  sed -e '/listen_host/s%::<%::1<%' -i $pkgdir/etc/clickhouse-server/config.xml
  cp ../debian/clickhouse-server.service $pkgdir/usr/lib/systemd/system
}

It tries to build arrow package, which is very strange - replaces half of the system packages to aur versions of them (like zlib-statc) and doesn't even compile. Does this one really need it?

19.3.5-1 doesn't build:

tmp/tmp.JefkzNrImZ/clickhouse/clickhouse/src/ClickHouse-19.3.5-stable/dbms/programs/clang/Compiler-7.0.1/driver.cpp:15:10: fatal error: clang/Driver/Driver.h: No such file or directory
 #include "clang/Driver/Driver.h"

Also building with persistent ccache could noticeably speed up subsequent builds.

rbelio, you have several options:

1. Try to find out where your resources are wasted and utilize them better. It could be slow disk I/O, slow filesystem (check fs mount options), only one core used (check MAKEFLAGS in /etc/makepkg.conf). Sometimes installing "ninja" helps to build ClickHouse faster.

2. Build the package on a faster box, then transfer the binary. Or use some distributed tools, like distcc or dist-clang.

3. Do not apply full system updates too often to avoid frequent rebuilds. However this means taking risk of broken packages and introducing security vulnerabilities.

4. (Not recommended.) You can hack your binary to point to another library with patchelf --replace-needed. Anything could break after this hack but sometimes it would work. Maybe this is an acceptable use case for non-production environments.